Results 1 to 8 of 8
  1. #1
    Ms.Ranjan is offline Member
    Join Date
    Jun 2008
    Posts
    85
    Rep Power
    0

    Default Java Heap Space : ArrayList<Integer>

    Dear All,

    I am expected to process 50,000,000 records from a database. I take 100,000 records at a time store it in ArrayList<Integer> at a time in order to process. I use 64 bit windows machine and I have increased the heap space as much as possible

    java -Xms500m -Xmx1500m Codename

    I choose ArrayList instead of Hashmap since it not a specific search for records but rather looping through the whole list. Few research study in google advised me to choose arraylist over hashmap.

    But I still get java.lang.OutofMemoryError: Java heap space. What is so weird is I get this error after processing every 5,000,000 records..which I dont understand at all...

    But anyway the question is using ArrayList<Long> over ArrayList<Integer> will it improve the performance. As far as I know values stored in the ArrayList when it is a number and needs more bit space we have to use Long instead of Integer. Is my understanding right? Please correct me if I wrong.. And if you have suggestion for my heap space problem I would really appreciate..
    Last edited by Ms.Ranjan; 03-15-2011 at 02:44 PM.

  2. #2
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,086
    Rep Power
    20

    Default

    You are (somehow, somewhere) holding onto those references, and more.

    Not knowing what it is your system is supposed to do, or what it is doing, I can't say more than that.

    As for Long or Integer, that depends on whether you need to store longs or integers. Performance is irrelevant at this point.

  3. #3
    Ms.Ranjan is offline Member
    Join Date
    Jun 2008
    Posts
    85
    Rep Power
    0

    Default

    Thank you tolls. Here is the code snippet
    Java Code:
    ArrayList<Integer> id=new ArrayList<Integer>();
    
    public static void main(String args[])
    {
            while(condition){
    		 id=new ArrayList<Integer>(); 
    		 id=d.search();	//gets 100,000 records	 
    		 d.processlogicindatabase(id);
    		 id=null;
    		 }
    }
    making the list =null and intializing the list again will not help loose the reference..is the approach correct

  4. #4
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,086
    Rep Power
    20

    Default

    You don't need the list at all outside of the loop so simply declare it inside the loop.

    Without knowing what either search() or process...() do I can't say at all what is going on.

    My suggestion is to take heap dumps throughout the run and stick them through jhat or Eclipse MAT to see what is being built up and where.

  5. #5
    lovelesh is offline Senior Member
    Join Date
    Nov 2010
    Location
    Delhi
    Posts
    135
    Blog Entries
    1
    Rep Power
    0

    Post

    It seems to be a memory leak in either search method or processlogicindatabase method.

    Re-visit these methods, I suggest.

  6. #6
    ozzyman's Avatar
    ozzyman is offline Senior Member
    Join Date
    Mar 2011
    Location
    London, UK
    Posts
    797
    Blog Entries
    2
    Rep Power
    4

    Default

    Please use these variables to track the heap space to see where it is being used up.

    Java Code:
    // Get current size of heap in bytes
    long heapSize = Runtime.getRuntime().totalMemory();
    
    // Get maximum size of heap in bytes. The heap cannot grow beyond this size.
    // Any attempt will result in an OutOfMemoryException.
    long heapMaxSize = Runtime.getRuntime().maxMemory();
    
    // Get amount of free memory within the heap in bytes. This size will increase
    // after garbage collection and decrease as new objects are created.
    long heapFreeSize = Runtime.getRuntime().freeMemory();

  7. #7
    Cbani is offline Member
    Join Date
    Jan 2010
    Posts
    90
    Rep Power
    0

    Default

    Java Code:
    ArrayList<Integer> id=new ArrayList<Integer>();
    
    public static void main(String args[])
    {
            while(condition){
    		 id=new ArrayList<Integer>(); 
    		 id=d.search();	//gets 100,000 records	 
    		 d.processlogicindatabase(id);
    		 id=null;
    		 }
    }
    rather code it this way

    Java Code:
    //if id is a method level variable then refer it to null; else leave it the way it is
    
    ArrayList<Integer> id;
    public static void main(String args[])
    {
            while(condition){
    		  id=d.search();	//gets 100,000 records	 
    		 d.processlogicindatabase(id);
    		 id=null;
    		 }
    }

  8. #8
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,086
    Rep Power
    20

    Default

    Quote Originally Posted by ozzyman View Post
    Please use these variables to track the heap space to see where it is being used up.

    Java Code:
    // Get current size of heap in bytes
    long heapSize = Runtime.getRuntime().totalMemory();
    
    // Get maximum size of heap in bytes. The heap cannot grow beyond this size.
    // Any attempt will result in an OutOfMemoryException.
    long heapMaxSize = Runtime.getRuntime().maxMemory();
    
    // Get amount of free memory within the heap in bytes. This size will increase
    // after garbage collection and decrease as new objects are created.
    long heapFreeSize = Runtime.getRuntime().freeMemory();
    No. When tracking possible memory leaks you use heap dumps.
    The above will give you little useful information, never mind it involving scattering special code throughout your app.

    Quote Originally Posted by Cbani View Post
    Java Code:
    ArrayList<Integer> id=new ArrayList<Integer>();
    
    public static void main(String args[])
    {
            while(condition){
    		 id=new ArrayList<Integer>(); 
    		 id=d.search();	//gets 100,000 records	 
    		 d.processlogicindatabase(id);
    		 id=null;
    		 }
    }
    rather code it this way

    Java Code:
    //if id is a method level variable then refer it to null; else leave it the way it is
    
    ArrayList<Integer> id;
    public static void main(String args[])
    {
            while(condition){
    		  id=d.search();	//gets 100,000 records	 
    		 d.processlogicindatabase(id);
    		 id=null;
    		 }
    }
    And the above will achieve nothing in terms of memory usage, bar saving a few bytes on empty arraylists, which should be garbage collected anyway.

    In addition, it's little different to the original code in that "id" is still unecessarily declared outside the loop.

Similar Threads

  1. Java Heap space error using ArrayList.
    By AcousticBruce in forum New To Java
    Replies: 13
    Last Post: 01-03-2011, 01:06 PM
  2. Java Heap Space
    By a.r.viswanath@gmail.com in forum Eclipse
    Replies: 2
    Last Post: 07-02-2010, 05:34 AM
  3. Java heap space,
    By babyboban in forum Advanced Java
    Replies: 18
    Last Post: 01-14-2010, 12:22 PM
  4. Java Heap Space
    By sandeeprao.techno in forum Advanced Java
    Replies: 19
    Last Post: 10-30-2008, 11:27 AM
  5. Java heap space?
    By javanewbie in forum New To Java
    Replies: 1
    Last Post: 06-24-2008, 06:55 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •