Results 1 to 3 of 3
  1. #1
    amitvit014 is offline Member
    Join Date
    Nov 2011
    Posts
    4
    Rep Power
    0

    Default How to use Sitemap for WebCrawling

    How to use Sitemap for WebCrawling using any OpenSource.

  2. #2
    Norm's Avatar
    Norm is online now Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    17,901
    Rep Power
    25

    Default Re: How to use Sitemap for WebCrawling

    Can you provide a link to that product?

  3. #3
    pbrockway2 is offline Moderator
    Join Date
    Feb 2009
    Location
    New Zealand
    Posts
    4,585
    Rep Power
    12

    Default Re: How to use Sitemap for WebCrawling

    From a (now closed) double post (here): "How to use Sitemap(its available in robots.txt) for WebCrawling using any OpenSource."

    I don't think we're dealing with some actual piece of software here.

Similar Threads

  1. Webcrawling www.stock-forecasting.com
    By devasia1000 in forum Networking
    Replies: 0
    Last Post: 02-12-2011, 12:25 PM
  2. webcrawling and harvesting
    By locke19 in forum New To Java
    Replies: 2
    Last Post: 04-14-2010, 03:53 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •