[B]How to use Sitemap for WebCrawling[/B]

How to use Sitemap(its available in robots.txt) for WebCrawling using any OpenSource. Please help me in this regard

Thanks.

Double post. Reply in the original thread: http://www.java-forums.org/advanced-...bcrawling.html

@OP: please don't post a question twice. Also consider giving some context about the site and the framework being used (if any) so people can see where the Java language fits in.