Results 1 to 5 of 5
- 01-01-2010, 01:37 PM #1
Member
- Join Date
- Jan 2010
- Posts
- 3
- Rep Power
- 0
Creating a crawler for a particular website
Hello everyone,
I am trying to develop a crawler for a website (nextag.com), the idea is to take the search query and pass it on to nextag and then get the results of the website in an array.
I am not sure even on how to get started, I tried reading tutorials but they were not much of a help for this particular problem. Any help will be greatly appreciated.
Thanks
- 01-01-2010, 01:58 PM #2
- Join Date
- Jul 2007
- Location
- Colombo, Sri Lanka
- Posts
- 11,374
- Blog Entries
- 1
- Rep Power
- 18
Basically are you want to develop a search engine for your web site, is it?
- 01-01-2010, 02:18 PM #3
Member
- Join Date
- Jan 2010
- Posts
- 3
- Rep Power
- 0
In a manner, yes.
The website has its own search feature, however, I want that the user should pass the query through the program I have developed. I should be able to get total number of results and page-wise results in an array.
So suppose user passes the parameter "mobile phone, 2" then I should be able to list the results given by NexTag.com for "mobile phone" on page number #2 of the results page.
- 01-01-2010, 05:20 PM #4
Member
- Join Date
- Jan 2010
- Posts
- 3
- Rep Power
- 0
Is anybody coming to my rescue?
- 01-02-2010, 03:10 AM #5
- Join Date
- Jul 2007
- Location
- Colombo, Sri Lanka
- Posts
- 11,374
- Blog Entries
- 1
- Rep Power
- 18
How about the following resources. I've customize few of them used several times. May those things not exactly what you want, but it's possible to modified them.
Open Source Search Engines in Java
Similar Threads
-
crawler doesn't start threads.
By Pierced1 in forum Threads and SynchronizationReplies: 2Last Post: 09-28-2009, 08:02 PM -
set up tomcat on website
By hunterbdb in forum Java ServletReplies: 1Last Post: 04-22-2009, 06:52 AM -
Writing to a website
By Shawn in forum New To JavaReplies: 6Last Post: 03-20-2009, 12:32 PM -
How to download website (Get all link in website)
By finalmem in forum Advanced JavaReplies: 0Last Post: 11-12-2008, 08:43 AM -
Search Engine , Web Crawler
By sahil.ansari in forum Advanced JavaReplies: 5Last Post: 07-21-2008, 01:53 AM


LinkBack URL
About LinkBacks
Reply With Quote

Bookmarks