Crawl site, find new articles and then email HTML
Hie guys, I need some advice from the senior members.
I want to write a program that will crawl about 7 websites, search for new articles, copy that content, and then send it to an email address.
I'm still at the stage of trying to create a mental picture of the application and to find ways of how things will be done.
When a new article is found pulling out the data will be quite straight forward. The question I have is this though, how can I use Java to find out if a new article has been posted. These are rather simple sites where a new link pops up if an article is posted.
Secondly I would also appreciate some suggestions as to how to implement this whole thing. I would like to get my program to run these checks every 30 minutes.