Java Heap Space : ArrayList<Integer>

[Ms.Ranjan]

Dear All,

I am expected to process 50,000,000 records from a database. I take 100,000 records at a time store it in ArrayList<Integer> at a time in order to process. I use 64 bit windows machine and I have increased the heap space as much as possible

java -Xms500m -Xmx1500m Codename

I choose ArrayList instead of Hashmap since it not a specific search for records but rather looping through the whole list. Few research study in google advised me to choose arraylist over hashmap.

But I still get java.lang.OutofMemoryError: Java heap space. What is so weird is I get this error after processing every 5,000,000 records..which I dont understand at all...

But anyway the question is using ArrayList<Long> over ArrayList<Integer> will it improve the performance. As far as I know values stored in the ArrayList when it is a number and needs more bit space we have to use Long instead of Integer. Is my understanding right? Please correct me if I wrong.. And if you have suggestion for my heap space problem I would really appreciate..

[Tolls]

You are (somehow, somewhere) holding onto those references, and more.

Not knowing what it is your system is supposed to do, or what it is doing, I can't say more than that.

As for Long or Integer, that depends on whether you need to store longs or integers. Performance is irrelevant at this point.

[Ms.Ranjan]

Thank you tolls. Here is the code snippet

ArrayList<Integer> id=new ArrayList<Integer>();

public static void main(String args[])
{
        while(condition){
		 id=new ArrayList<Integer>(); 
		 id=d.search();	//gets 100,000 records	 
		 d.processlogicindatabase(id);
		 id=null;
		 }
}

making the list =null and intializing the list again will not help loose the reference..is the approach correct

[Tolls]

You don't need the list at all outside of the loop so simply declare it inside the loop.

Without knowing what either search() or process...() do I can't say at all what is going on.

My suggestion is to take heap dumps throughout the run and stick them through jhat or Eclipse MAT to see what is being built up and where.

[lovelesh]

It seems to be a memory leak in either search method or processlogicindatabase method.

Re-visit these methods, I suggest.

[ozzyman]

Please use these variables to track the heap space to see where it is being used up.

// Get current size of heap in bytes
long heapSize = Runtime.getRuntime().totalMemory();

// Get maximum size of heap in bytes. The heap cannot grow beyond this size.
// Any attempt will result in an OutOfMemoryException.
long heapMaxSize = Runtime.getRuntime().maxMemory();

// Get amount of free memory within the heap in bytes. This size will increase
// after garbage collection and decrease as new objects are created.
long heapFreeSize = Runtime.getRuntime().freeMemory();

[Cbani]

ArrayList<Integer> id=new ArrayList<Integer>();

public static void main(String args[])
{
        while(condition){
		 id=new ArrayList<Integer>(); 
		 id=d.search();	//gets 100,000 records	 
		 d.processlogicindatabase(id);
		 id=null;
		 }
}

rather code it this way

//if id is a method level variable then refer it to null; else leave it the way it is

ArrayList<Integer> id;
public static void main(String args[])
{
        while(condition){
		  id=d.search();	//gets 100,000 records	 
		 d.processlogicindatabase(id);
		 id=null;
		 }
}

[Tolls]

Please use these variables to track the heap space to see where it is being used up.

// Get current size of heap in bytes
long heapSize = Runtime.getRuntime().totalMemory();

// Get maximum size of heap in bytes. The heap cannot grow beyond this size.
// Any attempt will result in an OutOfMemoryException.
long heapMaxSize = Runtime.getRuntime().maxMemory();

// Get amount of free memory within the heap in bytes. This size will increase
// after garbage collection and decrease as new objects are created.
long heapFreeSize = Runtime.getRuntime().freeMemory();

No. When tracking possible memory leaks you use heap dumps.
The above will give you little useful information, never mind it involving scattering special code throughout your app.

ArrayList<Integer> id=new ArrayList<Integer>();

public static void main(String args[])
{
        while(condition){
		 id=new ArrayList<Integer>(); 
		 id=d.search();	//gets 100,000 records	 
		 d.processlogicindatabase(id);
		 id=null;
		 }
}

rather code it this way

//if id is a method level variable then refer it to null; else leave it the way it is

ArrayList<Integer> id;
public static void main(String args[])
{
        while(condition){
		  id=d.search();	//gets 100,000 records	 
		 d.processlogicindatabase(id);
		 id=null;
		 }
}

And the above will achieve nothing in terms of memory usage, bar saving a few bytes on empty arraylists, which should be garbage collected anyway.

In addition, it's little different to the original code in that "id" is still unecessarily declared outside the loop.