I need to develop the crawler which will crawl all the product link and price from a site . I heard that its more efficient and fast to use Sitemap which is mention in robots.txt file but I am not able to figured out how to use sitemap for crawling. please someone help me how to use Sitemap for crawling.