Results 1 to 6 of 6
  1. #1
    jodyflorian is offline Member
    Join Date
    May 2008
    Posts
    6
    Rep Power
    0

    Default Fetch Webpage - not working

    Hi,

    I'm determined to understand streams backwards - I'm getting somewhere but reached a hurdle. For some reason the code below doesn't retrieve the entire html page - only what looks like the http header. I'm a web developer coming from PHP so finding it quite a steep learning curve. I know there are simpler ways of doing this, but I want to get to grips with streams and this problems is bugging me!

    Any help would be brilliant - thanks!

    The output that the code produces is:

    HTTP/1.1 200 OKDate: Thu, 09 Oct 2008 07:13:28 GMTServer: Microsoft-IIS/6.0MicrosoftOfficeWebServer: 5.0_PubX-Powered-By: PHP/5.2.6Content-type: text/htmlContent-Length: 9752

    Note regarding the code - this site wouldn't let me post because the code includes a url - replace "DOT" with "." and "HTTP" with http : / /

    Java Code:
    public class Application 
    {
    	public static void main(String[] args) 
    	{
    		JFrame jf = new JFrame();
    		JTextArea jta = new JTextArea(6, 100);
    
    		jta.setText("Enter more text to see scrollbars");
    		JScrollPane jsp = new JScrollPane(jta);
            
    		jf.add(jsp);
    		jf.pack();
    		
    		jf.pack();
    		jf.setVisible(true);
    
    		String nl = "\r\n";
    		
    		try
    		{
    //			URL url = new URL("HTTPdiscodevilsDOTcoDOTuk/");
    			
    			
    			Socket skt = new Socket("discodevilsDOTcoDOTuk", 80);			InputStream is = skt.getInputStream();
    			//InputStream is = url.openStream();
    			OutputStream os = skt.getOutputStream();
    			OutputStreamWriter osw = new OutputStreamWriter(os);
    			InputStreamReader isr = new InputStreamReader(is);
    			BufferedReader br = new BufferedReader(isr);
    			osw.write("GET / HTTP/1.1"+nl);
    			osw.write("HOST: discodevilsDOTcoDOTuk"+nl);
    			osw.write(nl);
    			osw.flush();
    			String all = "";
    		    	String s;
    		    	while (!(s = br.readLine()).equals("")) {
    //		    	while ((s = br.readLine())!=null) {
    //		    	while (br.ready()) {
    //		    		s = br.readLine();
    		    		all = all + s;
    		    	}
    			skt.close();
    			jta.setText(all);
    		}
    		catch (IOException ioe)
    		{
    			jta.setText(
    				"OOPS! Sock has a hole \n"
    				+ ioe.getClass()+nl
    				+ ioe.getMessage()+nl
    			);
    		}
    		finally
    		{
    			
    		}
    	}
    }

  2. #2
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    SW Missouri
    Posts
    17,308
    Rep Power
    25

    Default

    Do you understand the format of an HTML response?
    There is an empty line between the header and body.
    Does your code handle that?
    Can you explain what the logic is in the following:
    while (!(s = br.readLine()).equals("")) {
    all = all + s;

  3. #3
    jodyflorian is offline Member
    Join Date
    May 2008
    Posts
    6
    Rep Power
    0

    Default

    Uh thank you Norm!!

    The line:

    Java Code:
    		    	while (!(s = br.readLine()).equals("")) {
    is meant to see if there are any more lines to be read.

    I see now how this line stops the reading when the empty line between head and body is read and tested.

    My question now then is how can I determine when the end of the page is received? Is there a socket or stream related method I can call, replacing the condition in the above line of code? As far as I'm aware the http response doesn't have a delimiter for the end.

    Thanks again

  4. #4
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    SW Missouri
    Posts
    17,308
    Rep Power
    25

    Default

    see if there are any more lines to be read.
    The input stream classes have methods that say number of bytes available or if you are at the end of file.

  5. #5
    jodyflorian is offline Member
    Join Date
    May 2008
    Posts
    6
    Rep Power
    0

    Default

    Thanks.

    So far I've found that sub classes of InputStream have the method available() which would tell me how many bytes are left in the stream. However Readers are different - according to the Java Reference, BufferedReader's readLine() should return null when the end is reached. Yet the new while code has a problem:

    This works: (found by accident!!)

    Java Code:
    		jta.setText("reading...");
    		while ((s = br.readLine()) != null) {
    	    		all = all + s + "\n";
    			jta.setText(all);
    	    	}
    		skt.close();
    		jta.setText(all);
    But this doesn't: (nothing is displayed beyond "reading...")

    Java Code:
    		jta.setText("reading...");
    		while ((s = br.readLine()) != null) {
    	    		all = all + s + "\n";
    			//jta.setText(all);
    	    	}
    		skt.close();
    		jta.setText(all);
    I'm feeling really stupid right about now but can anyone tell me what I'm missing?? It's obviously staring me in the face! I just don't understand why:

    1) The second snippet doesn't work.
    2) The first snippet doesn't produce a large amount of rubbish - the first line, the first two lines, the first three lines and so on.

    Thanks!
    Last edited by jodyflorian; 10-10-2008 at 09:58 AM.

  6. #6
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    SW Missouri
    Posts
    17,308
    Rep Power
    25

    Default

    second snippet doesn't work.
    Can you explain?
    Try debugging your code by adding println(s) to show each read of s.
    first snippet doesn't produce a large amount of rubbish
    Not sure what that means


    What is the difference between the two snippets? The setText is commented out.

Similar Threads

  1. Secure Webpage for Logins
    By chandpuri in forum Advanced Java
    Replies: 3
    Last Post: 09-22-2009, 05:42 PM
  2. save a webpage.....?
    By 05bit044 in forum Networking
    Replies: 1
    Last Post: 08-04-2008, 04:49 PM
  3. Replies: 8
    Last Post: 05-28-2008, 07:00 AM
  4. How to fetch an arralist element (i need the code)
    By raj reddy in forum JavaServer Pages (JSP) and JSTL
    Replies: 7
    Last Post: 04-21-2008, 03:28 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •