Issue with Java Heap Space

[charan reddy]

Hi Experts,
We have developped our Scheduer framework which basically schesdules the java classess at specified frequency. This Scheduler framework is nothing but similar to BEA Java Method server.

The problem i am facing is JAVA HEAP SPACE issue. After one or two days after scheduler is started , While invokation of Jobs(ie schedule java programs) starts throwing JVM out of heap space ERROR ...Infact I am relaesing all the resources then and there.

Also i specifed min 512MB and Max 1GB runtime parameters for JVM while i launch java.exe .java -Xms512m -Xmx1024m.

So i am not able to understand why i am getting heap space issue....! IS Garbage Collector not running ??

Also one more question , suppose in s system if there are many java based apps running like BEA webogic , ORACLE DB , and xyz java based application , whether all the three wil be sharing same JVM or each will have its own instance of JVM ??

EXPERTS PLEASE DO HELP ME..I DESPERATELY NEED HELP FROM PEOPLE WHO HAVE INDEPTH KNOWEDGE ABOUT JVM AND ITS INTERNAL WORKING.

[masijade]

The problem i am facing is JAVA HEAP SPACE issue. After one or two days after scheduler is started , While invokation of Jobs(ie schedule java programs) starts throwing JVM out of heap space ERROR ...Infact I am relaesing all the resources then and there.

Seemingly not. Or at least, not correctly/completely.

IS Garbage Collector not running ??

Of course, just not enough is available to be garbage collected.

Also one more question , suppose in s system if there are many java based apps running like BEA webogic , ORACLE DB , and xyz java based application , whether all the three wil be sharing same JVM or each will have its own instance of JVM ??

If they are each started separately (i.e. each with their own java command), of course they are all in their own JVM. The "java command" is the JVM.


Seeing the code for this "scheduler" (if you really believe that the problem is there), and any code that uses it, may help.

[charan reddy]

hi thanks for the reply.

So you think that this is the issue with resource not getting freed ?
ok let me explain the logic to you , in scheduler framework there is a controller java class based on some scheduling algorithm , it schedules and invokes the jobs(ie java programs) hosted in the framework.
So at any point of time , it will be executing only one job and no concurency .

For invoking a job , i create a child thread and start invocation of the job , and wait for teh job to complete before i execute the next job. So if you see , the controller is a light weight program i mean relativey less data processinbg involved but i agree jobs that are running in the framework are heavy weight , So bascially what i am trying to get at is , since i a create a new child thread each time for each job invocation , the heap space occupied by jobs will automatically get released after the thread dies?
So where is the memory management problem existing ?

Also as you see , i am giving 1GB to JVM....thats quite a big thing....
Despite of this fact how are we still ending in HEAP SPACE ISSUE.?

Regards
Charan

[masijade]

Not if you still have a reference to something, somewhere. I haven't seen your code, nor the programs involved. Also, if the "program" executed through the thread doesn't close some resource it is completely possible that the thread hangs around. But, something is still being held onto somewhere, I can't tell you where, as I haven't seen your code.

[masijade]

In any case, run the thing with a profiler.

[masijade]

Also, judging by your other thread where you are attemtping to use "stop", I have no confidence in your abilities to do this correctly.

[charan reddy]

heyy,
i guess you are judging my coding capabilities wrongly by looking in to the thread which i posted for stop() method.I must say in the other thread the answer u gave was wrong...please understand why java team makes a method deprecated. Functionality wise the stop() will defintely stop a thread no matter as per the documentation even though its deprecated..Dont think that deprecated method never work properly..u are absolutely wrong ....The notion is they work but its not advisabe to use them. thats it..as simple as that..........

Coming to this topic...let me put question straight.....suppose there is a thread which invokes a job or program which uses lot of resources and dint free them.....! After the job when compltes ,,,the thread will terminate ...and subsequenty the resources allocated for the thread also becomes eligible for garbage collection since scope of those resources is over and hence gc will collect ir...am i right?

[masijade]

I didn't say it wouldn't "work". I said it is extremely "dangerous". I know why it is deprecated, and those reasons are more than good enough not to use it, ever, regardless of how much you may think you have it "under control". You seemingly don't.

[masijade]

And, if you think the dprecation in this case is only "not advisable", you are dead wrong. You do know that memory leaks are one of the possible (and usually almost guaranteed) problems with that "stop", don't you?

[charan reddy]

friend, in my code i dont have any mutex conditions or multipe thread synchronization sort of stuffs......only if those was existing ,,,then it would have been dangerous to use stop().

Since in my case none of the above are present..no harm in using stop()....the notion here is ...rather than leaving the job in a struck state ,,,i just wan tto kill it....! Any concerns?

[masijade]

Do what you wish. You seemingly know more than me.

[charan reddy]

ok if memory leaks are guaranteed with stop(),,then i am ready to drop off using stop method...but tell me what other aternative i have instead of stop().
Understand this usecase.....say if the child thread invokes a job and timout for the job is set to 5mins....the funtionality of this job is to read some data from a third party site,.,,,but unfortunately its taking more than timeout interval ..and i want it to be stopped rather than alowing it to tak some more time..
then the only option i was left with is stop() isnt it??? or there are anyother way?

For this situation..i am damn sure interrupt() or flag based mechainsm to exit run is not possible...

i hope u understood the usecase i am talking about.

[masijade]

Not that I see any need for this "scheduler". That is what Timer and TimerTask are for.

[masijade]

ok if memory leaks are guaranteed with stop(),,then i am ready to drop off using stop method...but tell me what other aternative i have instead of stop().
Understand this usecase.....say if the child thread invokes a job and timout for the job is set to 5mins....the funtionality of this job is to read some data from a third party site,.,,,but unfortunately its taking more than timeout interval ..and i want it to be stopped rather than alowing it to tak some more time..
then the only option i was left with is stop() isnt it??? or there are anyother way?

For this situation..i am damn sure interrupt() or flag based mechainsm to exit run is not possible...

i hope u understood the usecase i am talking about.

Then create the URL resource outside of the thread and pass it to the thread, then at the "timeout" close the connection.

[charan reddy]

ok if memory leaks are guaranteed with stop(),,then i am ready to drop off using stop method...but tell me what other aternative i have instead of stop().
Understand this usecase.....say if the child thread invokes a job and timout for the job is set to 5mins....the funtionality of this job is to read some data from a third party site,.,,,but unfortunately its taking more than timeout interval ..and i want it to be stopped rather than alowing it to tak some more time..
then the only option i was left with is stop() isnt it??? or there are anyother way?

For this situation..i am damn sure interrupt() or flag based mechainsm to exit run is not possible...

i hope u understood the usecase i am talking about.

[charan reddy]

that was just a example i gave u...the operation i invokefrom the thread is out of my control ,..,.. ..if i kill the thread which invoked the operation also be killed?

[charan reddy]

forget about scheduler..there are many reasons why we dint use timer and timer task..for our business logic..there werent appropriate ones..
hence we have our own scheduler...

[masijade]

Connections, and other IO type operations, may hang, thereby possibly preserving the entire thread, or, at the very least, the resources associated with those operations.

That is one of the problems with "stop", "suspend", and "destroy", that "cleanup" was done well, if at all, and that has only gotten worse in the versions since they were deprecated as there is no interest in bettering them since everyone is told not to use them anyway.

This is one case where I believe they should have not payed any attention to "backward compatability" and should have simply removed the methods, but "backward compatability" is very nearly a "holy grail". It is very important, but should have been sacrificed in this case.

[masijade]

forget about scheduler..there are many reasons why we dint use timer and timer task..for our business logic..there werent appropriate ones..
hence we have our own scheduler...

IOW it was either an unifromed decision or a management decision (which is about the same thing). In what way were they "not applicable".

[charan reddy]

jade..the scheduler logic has different business criteria on which it schedule..and there is no harm in it...
wat we need to look in to is the actual topic of heap issue,,