How to download temporary files as well as web page

[baiwen]

I am writing an application using JDK. I use URL class to create URLConnection to download a web page to parse out some strings and etc.

It works fine if I only parse string in its html page.

As we all know, using IE to open a web page will download all temporary files at "Temporary Internet Files" folder, image file, mp3, and etc

How can I write Java application by using URL/URLConnection/HttpURLConnection classes to not only download a web but all media or objects used by the page and be able to access to those objects WITHOUT opening IE?

I have been searching and reading all over the place and could not find any clue. I would really appreciate it if some experts share some tips here of how to do it.

Thanks

[emceenugget]

basically anything that's a "src" attribute must be downloaded. there's probably more, but that's a start. just parse the html and when you come across an attribute you need to download, download that as well.

[toadaly]

You are basically implementing the guts of a browser. This is not a trivial task, since you must first parse the HTML (IMHO, the built-in HTML parsing code in the current Java release is almost worthless for this purpose), and then familiarize yourself with the voluminous W3C HTML specs to figure out when you need to request objects.

However, there are tools out there such as Internet Download Manager that already have such functionality. Why not just use one of those?