Page 1 of 4 123 ... LastLast
Results 1 to 20 of 63
Like Tree5Likes

Thread: Full GC did not clear socket instances

  1. #1
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Full GC did not clear socket instances

    I have a socket programming running and below is my snippet of codes. Yesterday or the day before I had a full GC and I notice all the instances of the socket did not decrease compare to my last sample capture before the full GC and today I tried jmap:live I notice there is quite a huge drop.

    Java Code:
    BoneCP connectionPool = null;
      class ConnectionHandler implements Runnable {
        Connection dbconn = null;
        public void run() { // etc
         BufferedWriter writeBuffer = null;
         BufferedReader readBuffer = null;
         String capturedMessage="";
         try{
            dbconn = connectionPool.getConnection();
            dbconn.setAutoCommit(false);
            while ((nextChar=readBuffer.read()) != -1){          
              capturedMessage += (char) nextChar;
              if (nextChar == '*'){
              
              }
         }
         catch (SocketTimeoutException ex){
               ex.printStackTrace();
         }
         catch (IOException ex){
               ex.printStackTrace();
         }
         catch (Exception ex){
               ex.printStackTrace(System.out);
         }    
         finally{
            try{
             if ( dbconn != null ){
               dbconn.close();
             }
             else{
              System.out.println("dbConn is null in finally close");
             }
            }
            catch(SQLException ex){
                ex.printStackTrace();
            }
            try{
              if ( writeBuffer != null ){
                writeBuffer.close();           
              }
              else{
                System.out.println("writeBuffer is null in finally close");
              }
            }
            catch(IOException ex){
                ex.printStackTrace(System.out);
            }
           }
          }

    Before Full GG below are the values of jmap. I compare this 12: 44684 5004608 java.net.SocksSocketImpl (Beforee full GC) and 12: 48536 5436032 java.net.SocksSocketImpl (After full GC)

    I cant put the full jmap result so I have attached zip the 2 document one is before and after the Full GC ?
    Attached Files Attached Files

  2. #2
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    What sockets?
    Are you talking about the ones your connections use?
    If so why do you think they will be GCed?
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  3. #3
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Tolls,
    I am not sure I see all the socket instance such as java.net.SocksSocketImpl,java.net.SocketInputStrea m , java.net.Socket, java.net.SocketOutputStream keep growing. I am worried is my socket being closed properly or not? I dont want to end up with OOM error. So I am worried the instances are growing even after FGC. I really dont know whether they will be FGCed or not.

  4. #4
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    But what socket?
    Is it the database connection?

    I assume it is, because that's the code that you've posted.
    So why do you think they should be being closed?
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  5. #5
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Tolls,
    I mean the client server socket not the db socket. I am monitoring the jmap values and they keep growing even though I have closed it via this statement writeBuffer.close(); in the finally block ?

  6. #6
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    You really need to pick up the nack of posting relevant code, then.

    Anyway, from what I can see above you never close the BufferedReader.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  7. #7
    gimbal2 is offline Just a guy
    Join Date
    Jun 2013
    Location
    Netherlands
    Posts
    4,365
    Rep Power
    6

    Default Re: Full GC did not clear socket instances

    I thought this looked familiar. Also asked here, weeks ago:

    https://forums.oracle.com/thread/2606733
    "Syntactic sugar causes cancer of the semicolon." -- Alan Perlis

  8. #8
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Tolls,
    I am confuse and lost here. If you see this two article. 1. Here suggested by EJP just close the output stream will be sufficient to close the whole socket java - What is the difference between closing Input/OutputStream and closing Socket directly? - Stack Overflow "You should close the outermost output stream you have created from the socket. That will flush it. Closing either the socket or the input stream doesn't do that so it isn't adequate. Having closed that output stream you don't need to do anything else." and 2. This tutorial says close everything Reading from and Writing to a Socket (The Java™ Tutorials > Custom Networking > All About Sockets). So what should we do close all or just close outputstream ?

  9. #9
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    And when you changed your code so that it closed the streams and then the socket?

    Or look at the code for Socket (or the associated OutputStream class of Socket) and see what happens on a close().

    EJP is correct (as usual) that closing the output stream closes the socket, just from looking at the code.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  10. #10
    gimbal2 is offline Just a guy
    Join Date
    Jun 2013
    Location
    Netherlands
    Posts
    4,365
    Rep Power
    6

    Default Re: Full GC did not clear socket instances

    That 'close outermost output stream' refers to the following situation; If you have a nested setup, say;

    Java Code:
    FileOutputStream outputStream = new FileOutputStream(somefile);
    BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(outputStream));
    then you MUST close writer which is the OUTER most element in the chain, and not close outputStream directly which is the INNER most in the chain. If you close outputStream directly, data that was buffered by writer may not be flushed. I've run into that one myself once before.
    "Syntactic sugar causes cancer of the semicolon." -- Alan Perlis

  11. #11
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    I thought that was what EJP was talking about until I read that SO thread.
    Closing the output stream (which is a SocketOutputStream?) does result in the socket being closed. The stream class knows which socket it belongs to, so can ensure it is closed off.

    Though, as I said, that says nothing about being garbage collected.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  12. #12
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Gimbal,
    So in my scenario based on your experience what is the right way to close in the finally block? Must I close the input,output and the socket in this sequence?

  13. #13
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Tolls,
    So you also interpret what EJP said that Closing the output stream (which is a SocketOutputStream?) is good enough rite. What I want a confirmation is that will this lead to any kind of socket leakage and what I asking is that during the FGC there numbers remain high? Will number keep growing and is this normal ?

  14. #14
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    I looked at the code.
    You do realise the source code is all available with the JDK?

    It shows that closing the output stream closes the socket.

    If they're not being collected then it's possible you are holding onto something somewhere.
    The badly edited snippet above gives us no information at all.

    You should be able to put together a test case which just has your socket and streams and nothing else, opening and closing them as you do in the code you are trying to "fix".
    Post that here (assuming it shows the same sort of gc output) and then we can talk about how that actual code is working.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  15. #15
    gimbal2 is offline Just a guy
    Join Date
    Jun 2013
    Location
    Netherlands
    Posts
    4,365
    Rep Power
    6

    Default Re: Full GC did not clear socket instances

    Quote Originally Posted by newbie14 View Post
    Dear Tolls,
    So you also interpret what EJP said that Closing the output stream (which is a SocketOutputStream?) is good enough rite. What I want a confirmation is that will this lead to any kind of socket leakage and what I asking is that during the FGC there numbers remain high? Will number keep growing and is this normal ?
    Yes, it is enough as has been said countless times to you already. But forum threads are no kind of confirmation, I mean you're asking people who you don't even know. There is but one true confirmation: proving it yourself. You need to do that no matter what people here answer to validate that was is said is actually the truth.

    If you have a leak somewhere (in stead of you simply misinterpreting the data you see), then letting the application run for sufficient enough time should cause the system to run out of handles for a new socket. So to get proof I suggest a load test by trying to continuously create and close connections over a time period of say an hour. If that does not fail, I have little reason to believe there is a resource leak somewhere.
    "Syntactic sugar causes cancer of the semicolon." -- Alan Perlis

  16. #16
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Tolls,
    Below is the full codes where I have done tests with.

    Java Code:
    public class cs8000 {
       class ConnectionHandler implements Runnable {
    
        private Socket receivedSocketConn1;
        ConnectionHandler(Socket receivedSocketConn1) {
             this.receivedSocketConn1=receivedSocketConn1;
        }
         Connection dbconn = null;
         Date connCreated = null;
         
         public void run() { // etc
         BufferedWriter writeBuffer  = null;
         BufferedReader readBuffer  = null;  
         String message="";
         try {
             writeBuffer  =  new BufferedWriter(new OutputStreamWriter(receivedSocketConn1.getOutputStream()));
             readBuffer  = new BufferedReader(new InputStreamReader(receivedSocketConn1.getInputStream()));
         
             int m = 0, count=0;
             int nextChar=0;
         
                 System.out.println( "\n\n\n THE device"+" "+ receivedSocketConn1.getInetAddress() +":"+receivedSocketConn1.getPort()+" IS CONNECTED ");
    	     receivedSocketConn1.setSoTimeout(1000);
    	     
    	        while ((nextChar=readBuffer .read()) != -1) 
    	         {	           	  
    	           	  message += (char) nextChar;  
    	           	   //n = n + (char) m;
    	           	  //  n = new StringBuffer().append((char)m).toString();
    	         	  int i = message.indexOf("GET");
    							if(i != -1) { 
    								break;
    							}
    				      if (nextChar == '*')
    				      {
    				      	
    				      }
    	         	
    	         }
    	         //closeConnection();
    	         System.out.println( "\n\n\n THE device close connection"+" "+ receivedSocketConn1.getInetAddress() +":"+receivedSocketConn1.getPort()+" IS CONNECTED ");
    	     
    	         System.out.println("\n\nDevice Closed The Connection Properly");
    	      }
    	      catch (SocketTimeoutException ex)  
    	      { 
    	           System.out.println("MyError:SocketTimeoutException has been caught in in the main first try");
    	           ex.printStackTrace();
    	      }  
    	      
    	      catch (Exception ex)  
    	      { 
    	           System.out.println( "\n\n\n THE device had exception problem"+" "+ receivedSocketConn1.getInetAddress() +":"+receivedSocketConn1.getPort()+" IS CONNECTED ");
    	     
    	           System.out.println("MyError:Exception has been caught in in the main first try");
    	           ex.printStackTrace(System.out);
    	      }      
    	      finally
    	      {
    	        try 
    	       	{
    	            System.out.println( "\n\n\n THE device is in finally"+" "+ receivedSocketConn1.getInetAddress() +":"+receivedSocketConn1.getPort()+" IS CONNECTED ");
    	     
    	            if ( writeBuffer  != null ) 
    		        {
    		          	writeBuffer .close();
    		        }
    		        else 
    		        {
    		        	System.out.println("MyError:writeBuffer  is null in finally close");
    		        }
    	        }
    	        catch(IOException ex){
    	           System.out.println("MyError:IOException has been caught in w in finally close");
    	           ex.printStackTrace(System.out);
    	        }
    	      }
          }
        
       
       }// main function close
       public static void main(String[] args) {
          new cs8000();
       }
       cs8000() { // default constructor     
          try 
          {
    			   final ServerSocket serverSocketConn = new ServerSocket(8000);				
    			   while (true) 
    					{
    						try 
    						{
    					            Socket socketConn1 = serverSocketConn.accept();
                                    new Thread(new ConnectionHandler(socketConn1)).start();			            
    						}
    						catch(Exception e)
    						{
    							System.out.println("MyError:Socket Accepting has been caught in main loop."+e.toString());
    						    e.printStackTrace(System.out);
    						}
    					}
          } 
          catch (Exception e) 
          {
             System.out.println("MyError:Socket Conn has been caught in main loop."+e.toString());
             e.printStackTrace(System.out);
             //System.exit(0); 
          }
       }
    }
    I run a client with a for loop which send to this server close and open again. Below is my jmap results. The number remain high till I do a jmap histo:live else they keep growing. I took the jmap on intilisation of the server, immediately after the 50 sockets done, waited 5 minutes and finally after the jmap histo:live.

    On start of the socket

    90: 25 2800 java.net.SocksSocketImpl
    138: 23 1104 java.net.SocketInputStream
    139: 22 1056 java.net.SocketOutputStream
    220: 20 480 java.net.InetSocketAddress
    255: 19 304 java.net.Socket$2
    256: 19 304 java.net.Socket$3
    257: 19 304 java.net.SocksSocketImpl$3

    Immediately after the test
    3: 77 8624 java.net.SocksSocketImpl
    82: 75 3600 java.net.SocketInputStream
    99: 76 2432 java.net.Socket
    113: 37 1776 java.net.SocketOutputStream
    211: 10 400 java.net.SocketTimeoutException
    282: 9 144 java.net.Socket$2
    283: 9 144 java.net.Socket$3
    480: 1 24 java.net.ServerSocket


    5 Minutes after the test
    53: 77 8624 java.net.SocksSocketImpl
    82: 75 3600 java.net.SocketInputStream
    82: 75 3600 java.net.SocketInputStream
    99: 76 2432 java.net.Socket
    113: 37 1776 java.net.SocketOutputStream
    211: 10 400 java.net.SocketTimeoutException
    282: 9 144 java.net.Socket$2
    283: 9 144 java.net.Socket$3
    480: 1 24 java.net.ServerSocket


    Using jmap histo:live
    62: 26 2912 java.net.SocksSocketImpl
    89: 24 1152 java.net.SocketInputStream
    90: 24 1152 java.net.SocketOutputStream
    114: 25 800 java.net.Socket
    428: 1 24 java.net.ServerSocket

  17. #17
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    All the above is showing is that stuff only gets garbage collected if the gc needs to (ie it is going to run out of space).
    That's why you have a difference between histo:live and the "5 Minutes after the test" ones. That's the ones waiting to be gc'd.

    Do as gimbal suggests and just hit the server for an hour (or less) and see if it falls over.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  18. #18
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Tolls,
    Yes this test I did is on a smaller machine. The other machine (production machine) I set the memory to minimum and maximum as 512Mb which I know is still quite small though. So maybe that is why when the last FGC took place it did not clear drastically all the sockets is it? I am still wondering why when we close the sockets and yet they remain shown in the jmap histo this kind of confusing for newbie like me and make me worried that something really wrong here. Below is how I run my test now and based on gimbal's advice how to maintain this for 1 hour ?

    Java Code:
    public class stress1 extends Thread {
    public static void main(String[] argv) {
        try {
            for (int i = 0; i < 50; i++) {
            Socket socket = new Socket("192.168.2.102", 8000);
            PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
            BufferedReader in = new BufferedReader(new InputStreamReader(
            socket.getInputStream()));
            out.println("$ACTEST35,30061104075130528955N10024852E000068*03A1*");
            System.out.println(in.readLine() + i);
            out.close();
            in.close();
            socket.close();
            }
        } catch (Exception e) {
            e.printStackTrace();
       }
      }
    }

  19. #19
    newbie14 is offline Member
    Join Date
    Feb 2010
    Posts
    63
    Rep Power
    0

    Default Re: Full GC did not clear socket instances

    Dear Gimbal,
    Agree with what you said off course I dont take things blindly as you see below based on Tolls suggestion I ran experiment and came out with some values. I agree I dont know how to interpret them well to decide is it leaking or not. By letting to run the application for some time also need to factor the memory available rite? At times it can fail due to insufficient memory too? So how to differentiate a genuine memory limitation and leakage?

  20. #20
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,224
    Rep Power
    20

    Default Re: Full GC did not clear socket instances

    Quote Originally Posted by newbie14 View Post
    Dear Tolls,
    Yes this test I did is on a smaller machine. The other machine (production machine) I set the memory to minimum and maximum as 512Mb which I know is still quite small though. So maybe that is why when the last FGC took place it did not clear drastically all the sockets is it? I am still wondering why when we close the sockets and yet they remain shown in the jmap histo this kind of confusing for newbie like me and make me worried that something really wrong here. Below is how I run my test now and based on gimbal's advice how to maintain this for 1 hour ?

    Java Code:
    public class stress1 extends Thread {
    public static void main(String[] argv) {
        try {
            for (int i = 0; i < 50; i++) {
            Socket socket = new Socket("192.168.2.102", 8000);
            PrintWriter out = new PrintWriter(socket.getOutputStream(), true);
            BufferedReader in = new BufferedReader(new InputStreamReader(
            socket.getInputStream()));
            out.println("$ACTEST35,30061104075130528955N10024852E000068*03A1*");
            System.out.println(in.readLine() + i);
            out.close();
            in.close();
            socket.close();
            }
        } catch (Exception e) {
            e.printStackTrace();
       }
      }
    }
    Use a while(true) loop.
    Possibly stick a Thread sleep in there so you don't end up queuing up connection attempts.
    Keep running until you decide to stop. Use your watch. Go for lunch. Have (lots) of coffee. Play table tennis. Do some knitting.

    How do you know it did a full garbage collection?

    The other thing you could so is take a heap dump and analyse that so you can see where these sockets etc are coming from.
    That will show you how much is being held onto.
    gimbal2 likes this.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

Page 1 of 4 123 ... LastLast

Similar Threads

  1. Replies: 16
    Last Post: 08-08-2013, 05:34 PM
  2. Socket Programming ERROR : Socket not connected
    By vishrut_n_shah in forum Networking
    Replies: 0
    Last Post: 11-04-2011, 10:32 AM
  3. What's the best way to clear a Socket?
    By Joel in forum New To Java
    Replies: 2
    Last Post: 06-22-2011, 06:30 PM
  4. Replies: 1
    Last Post: 07-02-2010, 02:18 PM
  5. append response to the request from Socket and write to another socket
    By vaibhav_singh_vs@yahoo.co in forum Networking
    Replies: 3
    Last Post: 04-17-2009, 08:02 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •