Results 1 to 12 of 12
  1. #1
    christophe is offline Member
    Join Date
    Jun 2010
    Posts
    5
    Rep Power
    0

    Default Memory in java doesn't correspond with eclipse.ini

    Hello everyone,

    I have to make at least 2 really big objects (1 int[13800][13800] and 1 int[13800][9980]).

    Therefore I need a lot of memory (more then 1GB).

    I tried to change /etc/eclipse.ini (I use Ubuntu 10.04) and it looks now like this:

    Java Code:
    -startup
    plugins/org.eclipse.equinox.launcher_1.0.201.R35x_v20090715.jar
    --launcher.library
    plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.0.200.v20090519
    -showsplash
    org.eclipse.platform
    --launcher.XXMaxPermSize
    3072m
    -vmargs
    -Xms2048m
    -Xmx3072m
    -Dorg.eclipse.equinox.p2.reconciler.dropins.directory=/usr/share/eclipse/dropins
    I have 4GB of ram in this and swapping should also be used if needed.
    If I check the following:

    Java Code:
    public static void main(String[] args) {
    		System.out.println(Runtime.getRuntime().maxMemory());
    		System.out.println(Runtime.getRuntime().freeMemory());
    		System.out.println(Runtime.getRuntime().totalMemory());
    }
    I get following return

    920911872
    61738824
    62062592

    These are numbers expressed in bytes, so it is a lot less as I try to get.

    I hope someone can give me a clue.

    Thanks in advance,
    Christophe
    Last edited by christophe; 06-09-2010 at 02:52 AM.

  2. #2
    berkeleybross's Avatar
    berkeleybross is offline Senior Member
    Join Date
    Feb 2010
    Location
    England
    Posts
    187
    Rep Power
    5

    Default

    Have you tried just running eclipse with:
    Java Code:
    eclipse -vmargs -Xms2048m -Xmx3072m
    You may also want to check out Eclipse and memory settings Max's Blog

    (its not my blog, its just a random one i found which helped sort out my problem a while back :P)

    Berkeleybross

  3. #3
    christophe is offline Member
    Join Date
    Jun 2010
    Posts
    5
    Rep Power
    0

    Default

    Thanks berkeleybross for looking into my problem.
    I also used this file to get this far, but it doesn't work for me (even command line).

    Even if I try with less memory maxMemory() etc. keeps the same

    Java Code:
    eclipse -vmargs -Xms128m -Xmx256m
    Last edited by christophe; 06-09-2010 at 03:28 AM.

  4. #4
    Eranga's Avatar
    Eranga is offline Moderator
    Join Date
    Jul 2007
    Location
    Colombo, Sri Lanka
    Posts
    11,372
    Blog Entries
    1
    Rep Power
    20

    Default

    Quote Originally Posted by christophe View Post
    Therefore I need a lot of memory (more then 1GB).
    Do you really believe that it's a good idea, or a good design?

  5. #5
    berkeleybross's Avatar
    berkeleybross is offline Senior Member
    Join Date
    Feb 2010
    Location
    England
    Posts
    187
    Rep Power
    5

    Default

    --launcher.XXMaxPermSize
    3072m
    I dont know much about permsize but what i understand is its seperate from heap space and i think its assigned before heap space.

    The website i linked to shows a permsize of about 128m. Have you tried setting it at about that? I've got a feeling its ninja'ing your memory leaving nothing for the heapspace.

  6. #6
    christophe is offline Member
    Join Date
    Jun 2010
    Posts
    5
    Rep Power
    0

    Default

    @Eranga:
    I need to parse 9980 documents and extract all person names out of these. With this I have to make a term-document matrix in which every cell contains the term-frequency in a specific document. As there are 13840 unique names this is a 13840x9980 matrix.
    In a next step I have to calculate the term-term correlation matrix, so this becomes a 13840x13840 matrix.
    This will be used to try some clustering algorithms.

    As you can see this project for school isn't really about 'good' design but more about good algorithms. If you can give me a good alternative to calculate these matrices I am eager to hear

  7. #7
    Eranga's Avatar
    Eranga is offline Moderator
    Join Date
    Jul 2007
    Location
    Colombo, Sri Lanka
    Posts
    11,372
    Blog Entries
    1
    Rep Power
    20

    Default

    I'm not exactly clear what you mean by "parse 9980 documents". How did you pass all those things? And also once you process documents how did you start to build the matrix? Did you write them into a file or keep in heap? I think it is.

  8. #8
    christophe is offline Member
    Join Date
    Jun 2010
    Posts
    5
    Rep Power
    0

    Default

    setting XXMAxPermSize to 128 or 256 doesn't change anything either.

  9. #9
    christophe is offline Member
    Join Date
    Jun 2010
    Posts
    5
    Rep Power
    0

    Default

    @Eranga
    I save and load them from a data file after creating it (serialized objects) so I only have the objects I need loaded.
    After loading the term-document matrix totalMemory() tells me it uses 594608128 bytes so this works. The problem is that i need to create another big matrix while having the other object still in memory.

  10. #10
    berkeleybross's Avatar
    berkeleybross is offline Senior Member
    Join Date
    Feb 2010
    Location
    England
    Posts
    187
    Rep Power
    5

    Default

    I really cant think of much to help you...

    Really stabbing in the dark, but have you tried running just the java file via command line instead of through eclipse?
    May see if it really is a problem with the ini file.

    Also have you tried looking at the task manager to see if linux thinks the memory is a different size than java?

    And can you tell us the exact error message that occurs? Out of memory errors dont usually tell much but every little helps..

  11. #11
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,014
    Rep Power
    20

    Default

    Just a question, but is this a 64 bit system with a 64 bit Java?
    If not then you actually only have about 1.5G max, some of which is used by the JVM itself and so isn't heap.

    Also, as has been pointed out, you don't want perm size to be anywhere near that amount. Perm space is for storing class information.

    One more thing, you really don't want to be using swapping with Java. It really, really slows down the garbage collection.

    Oh yes, and when running your code is it actually running inside the eclipse JVM or another one? Because I think you should be setting this in the individual run configuration for the project and not for the Eclipse launcher...

  12. #12
    Eranga's Avatar
    Eranga is offline Moderator
    Join Date
    Jul 2007
    Location
    Colombo, Sri Lanka
    Posts
    11,372
    Blog Entries
    1
    Rep Power
    20

    Default

    Quote Originally Posted by christophe View Post
    @Eranga
    I save and load them from a data file after creating it (serialized objects) so I only have the objects I need loaded.
    After loading the term-document matrix totalMemory() tells me it uses 594608128 bytes so this works. The problem is that i need to create another big matrix while having the other object still in memory.
    How about keeping them in a database if you really want to use such a huge memory space.

Similar Threads

  1. Java Memory Issue
    By personal in forum Advanced Java
    Replies: 12
    Last Post: 01-07-2012, 02:05 PM
  2. Replies: 9
    Last Post: 05-21-2010, 11:22 PM
  3. memory game in JAVA
    By lclclc in forum New To Java
    Replies: 19
    Last Post: 10-18-2009, 04:41 PM
  4. Out of memory error in eclipse
    By Java.child in forum New To Java
    Replies: 5
    Last Post: 09-05-2008, 04:50 PM
  5. Replies: 2
    Last Post: 08-21-2008, 07:33 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •