Results 1 to 11 of 11
  1. #1
    sheetalmuskan is offline Member
    Join Date
    Jul 2011
    Posts
    4
    Rep Power
    0

    Default OUt of memery error while using string split operation

    Hi,

    I am beginner in Java.I am stucked in Out of memory error. What I am trying to do is : There are string separated by backslash. I am trying to split the string and concatenating the individual string in an arraylist. The code is :

    ArrayList<String> pathParts = new ArrayList<String>();

    for (String fparts : filePath.split("/")) {
    pathParts.add(fparts);
    }

    Note: where filePath="help/on/memory/leak"

    It works fine for low values...but if the file is too large like 100,000 files then I get Out of Memory Error: Java Heap space.

    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.regex.Pattern.compile(Pattern.java:1451)
    at java.util.regex.Pattern.<init>(Pattern.java:1133)
    at java.util.regex.Pattern.compile(Pattern.java:823)
    at java.lang.String.split(String.java:2292)
    at java.lang.String.split(String.java:2334)

    I have tried to increase the heap size to 2Gb using.. java -Xmx2048m .... but still getting same error..

    Any thoughts on this issue ??

    Thank you in advance....

  2. #2
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    17,611
    Rep Power
    25

    Default

    There are string separated by backslash
    How long is the String?
    How many items do you add to the ArrayList before the Exception?
    The sample String you show has 4 parts?

    if the file is too large like 100,000 files
    Can you explain the relationship between the String separated by backslash
    and a file too large?
    and 100,000 files?

  3. #3
    sheetalmuskan is offline Member
    Join Date
    Jul 2011
    Posts
    4
    Rep Power
    0

    Default

    Hello Thank u for d response.

    The string has like 6 parts.. I made a method "pathPartList" which returns arraylist , the operations inside method is splitting string and returning arraylist..

    public ArrayList<String> pathPartList(String filePath) {
    ArrayList<String> pathParts = new ArrayList<String>();

    for (String fparts : filePath.split("/")) {
    pathParts.add(fparts);
    }
    return pathParts;
    }

    Whenever I am fetching records from Database, I called this method with the filePath.... and at times, the records extracted are too big like 100,000 records. So, the ArrayList pathParts gets too large as I am concatenating split string until all my respective records gets extracted.

    Is there a limit in space in arraylist? i think the problem is my ArrayList pathParts gets too large sometimes according to the extracted records from database. Any thoughts???

    Thank u in advance...
    Last edited by sheetalmuskan; 07-11-2011 at 09:38 PM.

  4. #4
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    17,611
    Rep Power
    25

    Default

    Do you expect you can put all the data you want into the computer's memory?

    You can give the JVM some more space by using one of its -X.... options. Read the doc for the java command to find the one that looks good for you. It could be something like this: -Xmx512M

  5. #5
    sheetalmuskan is offline Member
    Join Date
    Jul 2011
    Posts
    4
    Rep Power
    0

    Default

    I have tried that too. I increased to 2 gb by doing , java -Xmx2048m in the terminal....but still getting same error..I get OOM error on the line :
    for (String fparts : filePath.split("/")) {

    Is there any other way for adding individual strings other than ArrayList? Is there anyway that I can offset(adding some portion at a time) while adding in ArrayList ?

    Thank u so much for helping me....

  6. #6
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    17,611
    Rep Power
    25

    Default

    How many Strings are you trying to add? Count them and print out the count every 1000 or so.

    way for adding individual strings
    Adding where? To what? For what purpose?

    What do you want to save all these bits and pieces of file paths for?

  7. #7
    sheetalmuskan is offline Member
    Join Date
    Jul 2011
    Posts
    4
    Rep Power
    0

    Default

    Hi Norm,

    Thank you so much for helping me....I got the problem solved now......I increased the heap size from Eclipse..Now it worked....I was doing that wrong before....Really appreciate your help...

  8. #8
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    17,611
    Rep Power
    25

    Default

    Where does the IDE fit in this problem?

  9. #9
    Junky's Avatar
    Junky is offline Grand Poobah
    Join Date
    Jan 2011
    Location
    Dystopia
    Posts
    3,798
    Rep Power
    7

    Default

    Quote Originally Posted by sheetalmuskan View Post
    I increased the heap size from Eclipse..Now it worked
    Until you process a larger file!

  10. #10
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    12,120
    Rep Power
    20

    Default

    Quite.
    There are few operations that require you to hold tons of data in memory like this.
    Usually there is a way around it by reading, processing, then discarding the data...outputting the result somewhere so you don't have to hold onto it.

  11. #11
    Dark's Avatar
    Dark is offline Senior Member
    Join Date
    Apr 2011
    Location
    Camp Lejuene, North Carolina
    Posts
    643
    Rep Power
    4

    Default

    Agreed, with large files trying to hold all that information in the RAM is a terrible idea. What happens when you need 4gb of RAM space and your user only have 2gb? What happens when they have 4gb, but they are running a media player in the background along with Window's normal processes?
    • Use [code][/code] tags when posting code. That way people don't want to stab their eyes out when trying to help you.
    • +Rep people for helpful posts.

Similar Threads

  1. Split a String with split()--Help
    By danilson in forum New To Java
    Replies: 7
    Last Post: 11-19-2010, 05:08 PM
  2. ArrayList out of bound, split string error.
    By Kerrai in forum New To Java
    Replies: 4
    Last Post: 11-03-2010, 10:37 AM
  3. String split
    By soccer_kid_6 in forum New To Java
    Replies: 3
    Last Post: 10-29-2010, 08:51 PM
  4. How to split a String using split function
    By Java Tip in forum java.lang
    Replies: 4
    Last Post: 04-17-2009, 09:27 PM
  5. How to split a String using split function
    By JavaBean in forum Java Tip
    Replies: 0
    Last Post: 10-04-2007, 10:32 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •