Hi

I have a task to use stormcrawler for crawling a list of web urls.

Gone through the Getting Started page in
XML Code:
http://stormcrawler.net/getting-started/
.

Till the You can run the topology in local mode I have done.

run the jar file with apache-storm-1.2.3, (unable to execute with windows cmd, so I used pythonscript) the command I executed is
Java Code:
C:\Users\Documents\apache-storm-1.2.3\apache-storm-1.2.3\bin>python storm.py jar C:\Users\Documents\ZZZCrawler\16-6-2021\stormcrawler\target\stormcrawler-1.0-SNAPSHOT.jar com.mycompany.crawler.CrawlTopology -conf C:\Users\Documents\ZZZCrawler\16-6-2021\stormcrawler\crawler-conf.yaml -local --sleep 30000
wait for a long time nothing is happening.

StormCrawler is used to Crawl websites, do any one have gone through this, how to get the content of website.

It would be very much helpful.

Gone through lots of docs and YouTube videos, most of them used Elastic search, I did the same too, but there also only URLs are listing not getting the content.

If the stormcrawler has no capability to do the same, then I need to search another alternative.


Thank in advance.