Memory is not getting released on RHEL Linux Closely related to Java application

[arajendran]

when I put load of 200 users on basic Airthmetic example page that gets shipped with Tomcat. Via free -m command on Linux, I can see the memory consumption (2gb of memory is consumed outta 4gb).
But as soon as the load on page is reduced and if nobody is using that application ..the free -m doesnt show the boost in free memory it still shows memory is occupied and its gradually @ slow pace its showing increase in free memory. I am not able to get this behaviour ..Only when I stop my tomcat it shows the actual good Available memory i.e approx 4gb. please suggest some settings

I wanted to know when my actual Webapplication with DB connectivity in Linux gets Live and is used by 200 users. Memory issue will affect its performance ..!

[Tolls]

The JVM grabs the memory off the underlying system.
It does not release that memory back except very slowly (there was a time it never released it back).

When you start a JVM up you tell it how much memory it is allowed to use (-Xmx, or the default) and therefore it can use up to that amount. If you have assigned it 2Gb of avaiable memory, then don't be surprised when it takes that and keeps hold of it.

[arajendran]

Thanks Tolls for your reply....
We haven't assigned any memory to JAVA_OPTS or CATALINA_OPTS in catalina.sh. We are using the default catalina.sh.
Its still consuming that much amount of memory mentioned above and releasing very slowly.

[Tolls]

There's a default value for the JVM based on the assumed type of machine you are using.
In your case probably a server (these days that tends to be the case as it's based, I think, on number of processors).
That default is usually around 1Gb, but that might have changed.

If this is a live server you should specify the amount you want to give Tomcat (and associated webapps).

Anyway, whatever you end up specifying then the JVM will quite likely grab that at some poiont and only slowly release it, as you are currently seeing.

[arajendran]

Hi Toll,

Thanks for your valuable time and the knowledge you have shared on the forum. Would like to know whether is there anyway using which we can expedite the release. Thanks.

Regards,
Ajay

[arajendran]

here's One more scenario
I created a webapplication via Eclipse with only one jsp i.e index.jsp. In this jsp page.. I have only typed in "just a Jsp" inside Body tag nothing else and I exported a war and deployed it to tomcat. Now I asked my Software tester colleague to just put a load of 200 users on this jsp ..During the load I was monitoring memory via free -m command

here's free -m the output
.................................................. .............................
On Tomcat Start
1)
-/+ buffers/cache: used 251 free 3697


.................................................. .............................
Below is the memory status when the load was complete
3)
-/+ buffers/cache: used 1727 free 2221


.................................................. .............................
below is when the Load was Completed and after few mins memory status is still :(

-/+ buffers/cache: used 1723 free 2225


the used 1723 never reduces !!!!!
For my real Web application I've been asked such Question .. will the consumed memory ever reduces?
In btw the load.. when I fired the command ps aux it showed 41 % memory consumed by java ...Can Somebody please help me on this ?

ps: for 200 users to work we have set xmx and xms value to 1536m (coz it wasnt taking the default value, 1024m or just 512m and load was failing) load worked successfully with xmx and xms value to 1536m
*Environment*
Red Hat Linux
JDK1.6
Tomcat6
4g Ram

[JosAH]

Can Somebody please help me on this ?

Operating systes are kind of stupid when it comes to memory allocation and releases, i.e. they treat memory as a stack; they can give more memory from the top of the stack and it has to be given back to the OS so that it fits on that stack top again. See the man pages for the brk and sbrk calls. If a process has free memory somewhere 'in the middle' it can't release it to the OS because it has to be a consecutive chunk with the memory on the stack. The JVM shuffles memory around and when/if a 'feasible' chunk of free memory becomes avaalable it can be returned to the OS. The trouble here is that only the garbage collector can check this but if the Java process doesn't do anything the garbage collector isn't invoked and no possible memory can be returned to the OS. The advantage is that this memory can be reused by the Java process without any system calls where the Java virtual machine has to beg for more memory from the OS but the disadvantage is that the JVM is using 'too much' memory at a certain moment in time. The other advantage is that we're talking virtual memory here.

kind regards,

Jos

[Tolls]

here's One more scenario

And?
I don't see the problem.
If you'd given the application a hundred Mb then you'd get some garbage collection going on, but you gave it lots such that it never had to do a gc in the time you were monitoring it.

Either your application fits on the server or it doesn't.
Either your server has the memory to support your application or it doesn't.
So what is the problem with Java occupying the memory you have allocated to it?
It's not as if anything else is using it...

[arajendran]

Thanks JosAH for your reply ..yes! memory is getting reused for subsequent load of 200 users.

[arajendran]

Thanks Tolls..my application was not taking load of 200 users ..the tests were failing ..only when I increased the heap size ..it passed successfully. My prob was the java occupying memory is there but will it release ever!! I waited hours to see ..also we went on looking for some kinda parameters that shall help to free the used memory :P
nothing so far we've found ..but we observed that on subsequent loads.. its getting re-utilized :)

[Tolls]

Of course it is.
That memory is memory the OS has given the JVM to use.
That is the JVMs Heap (+ other bits).
You need to understand that the JVM is simpyl an application sat on top of the OS.
It manages its own memory, which is a great slab of memory from the OS.
You don't use OS-related memory commands to see what memory your app is using, you use JVM-related memory commands (profiling, heap monitoring, that sort of thing).

There is no reason to have the JVM release memory back to the OS, frankly.