Results 1 to 2 of 2
  1. #1
    jlew is offline Member
    Join Date
    Nov 2007
    Location
    Vermont
    Posts
    2
    Rep Power
    0

    Question Http Buffer issue... chunked?

    I am writing this little java code (not an applet, but used as its own jar file) that talks to a web server using http but at times gets weird responses with VERY LONG LINES without line returns.

    Java Code:
    Socket socket    = new Socket( websitehost, 80);
    OutputStream os   = socket.getOutputStream();
    PrintWriter out   = new PrintWriter( os, true );
    BufferedReader in = new BufferedReader( new InputStreamReader( socket.getInputStream() ) );
    
    //Talk to Server
    out.println("GET " + URL_PREFIX + "&data=" + request + " HTTP/1.1");
    out.println("Host: " + websitehost);
    out.println("User-Agent: " + agent);
    out.println("Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5");
    out.println("Accept-Language: en-us,en;q=0.5");
    out.println("Accept-Encoding: deflate");
    out.println("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7");
    out.println("Keep-Alive: 300");
    out.println("Connection: keep-alive");
    out.println("Referer: " + referer);
    out.println("Cookie: pass=" + PASSW + "; email=" + EMAIL);
    out.println();
    
     // read the response
    boolean loop = true;
    StringBuffer sb = new StringBuffer(8096);
    
    while (loop) {
      if ( in.ready() ) {
         int i=0;
         while (i!=-1) {
            i = in.read();
            sb.append((char) i);
         }
         loop = false;
      }
      
      Thread.currentThread().sleep(50); //Don't remember why that is there
      }
    
       socket.close();
       return sb.toString();
    The problem is this website from time to time gives out HUGE lines of code with no breaks, and for some reason (don't know if it is the string buffer or something) when the line gets so many characters it adds in a line break, a few digits and anther line break and then it is reading again as if nothing happened

    XML Code:
    HTTP/1.1 200 OK
    Date: Sun, 21 Oct 2007 18:08:53 GMT
    Server: Apache/1.3.33 (Debian GNU/Linux) mod_throttle/3.1.2 mod_gzip/1.3.26.1a mod_fastcgi/2.4.2
    Pragma: no-cache
    Expires: Tue, 26-Oct-2000 12:00:00
    Vary: *
    Connection: close
    Transfer-Encoding: chunked
    Content-Type: text/html; charset=ISO-8859-1
    
    ...
    
    <img sr
    2000
    c=http://
    And with that img sr break there are more then this, but im just showing you this because you don't need to see the entire output to get the idea.

    Anyone have any ideas? Im thinking maybe its in the encoding returned "chunked" im not sure any ideas or solutions would be great.

    I need to be able to send my own headers that is why i am using this method... i have to emulate browsers and junk's headers in this script as well as get the data.

  2. #2
    jlew is offline Member
    Join Date
    Nov 2007
    Location
    Vermont
    Posts
    2
    Rep Power
    0

    Default

    The problem has to do with chunked data... Does anyone know a simple fix on how to tell it to read the chunked data and ignoring the chunked size flags. I was looking up this issue and what that \n2000\n is some sort of flag or something that when converted to hex it tells the next chunk size. Like in my code right now i see 2000 then the begin of html, after x number of bytes or chars or something it stops sends me a new flag saying how big the next chunk will be and then continues... Does anyone know how to handle chunked input?

    I can't seem to find a really good strait forward answer on how to do this without having to include a bunch of server library which i don't have.

Similar Threads

  1. [SOLVED] Save the current buffer to file
    By Azndaddy in forum New To Java
    Replies: 2
    Last Post: 03-29-2008, 08:46 AM
  2. Replies: 0
    Last Post: 03-11-2008, 01:25 PM
  3. Trouble with Buffer Sizing
    By Jeff in forum New To Java
    Replies: 3
    Last Post: 02-07-2008, 02:43 PM
  4. Help with String Buffer
    By mathias in forum AWT / Swing
    Replies: 1
    Last Post: 08-07-2007, 07:52 AM
  5. how to set the value of BUFFER SIZE
    By oregon in forum Advanced Java
    Replies: 1
    Last Post: 08-06-2007, 04:16 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •