reading URL using java through proxy server

[asheesh]

hi all
i made a java application through user can access webpage through proxy server, but this works for only some pages like (http://www.iita-conference.org,yahoo.com,iiitk.ac.in etc) it not works for like rediff.com, ieeeexplore,orkut etc. i also post the code plz tell me where is the problem? why it not works for all URLs?
if possible plz send code too.

package ieee;

import java.io.InputStream;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.PasswordAuthentication;
import java.net.URL;
import java.util.Properties;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
*
* @author sknavin
*/
public class Main {

/**
* @param args the command line arguments
*/
private static String loadPage(URL parURL) throws Exception {

Properties sysProps = System.getProperties();
sysProps.put("proxySet", "true");
sysProps.put("proxyHost", "172.31.1.3");
sysProps.put("proxyPort", "8080");

Authenticator authenticator = new Authenticator() {

public PasswordAuthentication getPasswordAuthentication() {
return (new PasswordAuthentication("userID","password".toCharA rray()));
}
};
Authenticator.setDefault(authenticator);

long lTime = System.currentTimeMillis();
StringBuffer sb = new StringBuffer("");
String http = parURL.toString();
// Create a connection.
HttpURLConnection urlConnection =
(HttpURLConnection) parURL.openConnection();
urlConnection.setRequestMethod("GET");
urlConnection.setDoOutput(true);
urlConnection.setDoInput(true);
urlConnection.setRequestProperty("USER-AGENT", "Mozilla/2.02Gold (WinNT; I)");
//
urlConnection.setRequestProperty("Content-type", "application/x-www-form-urlenCcoded");
urlConnection.setAllowUserInteraction(true);

urlConnection.connect();

InputStream in =
((HttpURLConnection) urlConnection).getInputStream();
int length = urlConnection.getContentLength();
for (int n = 0; n < length; n++) {
sb.append((char) in.read());
}

return sb.toString();
}

public static void main(String[] args) {
try {
// type your intended url here
URL url = new URL("http://www.iita-conference.org/");
String s = loadPage(url);
System.out.println("S=" + s);


} catch (Exception ex) {
Logger.getLogger(Main.class.getName()).log(Level.S EVERE, null, ex);
}


}
}

[DonCash]

Hello asheesh,

Welcome to the Java Forums.

It seems to me this is because the sites are blocking connectings from programs that are not web browsers. What you need to do is to trick these sites into thinking your program is a web browser.

Try this:

Code:

import java.net.*;
import java.io.*;

class httpconnect {

public static void main(String[] args) throws Exception {

URL url = new URL("http://www.anywebsite.com/");

URLConnection spoof = url.openConnection();

spoof.setRequestProperty(
"User-Agent", "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)" );

BufferedReader in = new BufferedReader(new InputStreamReader(spoof.getInputStream()));

String strLine = "";

while ((strLine = in.readLine()) != null) {

        System.out.println(strLine);
		
    }
in.close();
  }
}

Note this bit of code:

Code:

spoof.setRequestProperty( 
"User-Agent", "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; H010818)" );

This is what tricks the sites into thinking you are using a web browser.

I'm sure this will fix your problem!