Results 1 to 2 of 2
  1. #1
    quad64bit's Avatar
    quad64bit is offline Moderator
    Join Date
    Jul 2009
    Location
    VA
    Posts
    1,323
    Rep Power
    7

    Default Set SystemProperty "http.agent"

    I've seen this question posted before including here, but nowhere with an answer.

    I am trying to parse information from a webpage -- however, I keep getting a 403 error due to the user agent being invalid.

    When I set:
    System.setProperty("http.agent", "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_7; en-us) AppleWebKit/530.19.2 (KHTML, like Gecko) Version/4.0.2 Safari/530.19");

    Java appends itself to the end: Java/1.5.0_19

    Since the order of precedence is right to left, java supersedes everything else and the website denies access.

    Is there any way to prevent java from appending it's versioning information?

    I saw talk of custom http interaction but no explanation. Thanks for your advice!

  2. #2
    quad64bit's Avatar
    quad64bit is offline Moderator
    Join Date
    Jul 2009
    Location
    VA
    Posts
    1,323
    Rep Power
    7

    Default

    Hey guys, so I did some more experimentation, and found that the problem I was having can be bypassed by writing a custom http request, which is actually super easy. For anyone else having the same problem in the future, here is my solution:
    Java Code:
    public ConnectionTest(){
            try {
                Socket s = new Socket("www.imdb.com", 80);
                System.out.println("Socket Opened");
                BufferedWriter out = new BufferedWriter(new OutputStreamWriter(s.getOutputStream()));
                BufferedReader in = new BufferedReader(new InputStreamReader(s.getInputStream()));
                System.out.println("Streams Created");
                out.write("GET /title/tt0479884/ HTTP/1.1\n" +
                        "Host:www.imdb.com\n" +
                        "User-agent: Mozilla/4.0\n" +
                        "Connection: close\n\n");
                out.flush();
                while(true){
                    String line = in.readLine();
                    if(line == null)
                        break;
                    System.out.println(line);
                }
    
            } catch (UnknownHostException ex) {
                Logger.getLogger(ConnectionTest.class.getName()).log(Level.SEVERE, null, ex);
            } catch (IOException ex) {
                Logger.getLogger(ConnectionTest.class.getName()).log(Level.SEVERE, null, ex);
            }
        }
    }

Similar Threads

  1. Java, Military Format using "/" and "%" Operator!!
    By sk8rsam77 in forum New To Java
    Replies: 11
    Last Post: 02-26-2010, 03:03 AM
  2. problem at "http://localhost:8080"
    By tushar88 in forum New To Java
    Replies: 2
    Last Post: 07-08-2009, 06:50 PM
  3. Replies: 2
    Last Post: 01-24-2009, 06:56 PM
  4. System.setProperty("http.agent"..) appends Java/1.6
    By wwuster in forum Advanced Java
    Replies: 1
    Last Post: 10-25-2008, 03:20 PM
  5. Replies: 1
    Last Post: 10-20-2008, 07:35 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •