[jlew]

I am writing this little java code (not an applet, but used as its own jar file) that talks to a web server using http but at times gets weird responses with VERY LONG LINES without line returns.

Code:

Socket socket    = new Socket( websitehost, 80);
OutputStream os   = socket.getOutputStream();
PrintWriter out   = new PrintWriter( os, true );
BufferedReader in = new BufferedReader( new InputStreamReader( socket.getInputStream() ) );

//Talk to Server
out.println("GET " + URL_PREFIX + "&data=" + request + " HTTP/1.1");
out.println("Host: " + websitehost);
out.println("User-Agent: " + agent);
out.println("Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5");
out.println("Accept-Language: en-us,en;q=0.5");
out.println("Accept-Encoding: deflate");
out.println("Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7");
out.println("Keep-Alive: 300");
out.println("Connection: keep-alive");
out.println("Referer: " + referer);
out.println("Cookie: pass=" + PASSW + "; email=" + EMAIL);
out.println();

 // read the response
boolean loop = true;
StringBuffer sb = new StringBuffer(8096);

while (loop) {
  if ( in.ready() ) {
     int i=0;
     while (i!=-1) {
        i = in.read();
        sb.append((char) i);
     }
     loop = false;
  }
  
  Thread.currentThread().sleep(50); //Don't remember why that is there
  }

   socket.close();
   return sb.toString();

The problem is this website from time to time gives out HUGE lines of code with no breaks, and for some reason (don't know if it is the string buffer or something) when the line gets so many characters it adds in a line break, a few digits and anther line break and then it is reading again as if nothing happened

HTML Code:

HTTP/1.1 200 OK
Date: Sun, 21 Oct 2007 18:08:53 GMT
Server: Apache/1.3.33 (Debian GNU/Linux) mod_throttle/3.1.2 mod_gzip/1.3.26.1a mod_fastcgi/2.4.2
Pragma: no-cache
Expires: Tue, 26-Oct-2000 12:00:00
Vary: *
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html; charset=ISO-8859-1

...

<img sr
2000
c=http://

And with that img sr break there are more then this, but im just showing you this because you don't need to see the entire output to get the idea.

Anyone have any ideas? Im thinking maybe its in the encoding returned "chunked" im not sure any ideas or solutions would be great.

I need to be able to send my own headers that is why i am using this method... i have to emulate browsers and junk's headers in this script as well as get the data.