Results 1 to 8 of 8
  1. #1
    RR_QQ is offline Member
    Join Date
    Sep 2008
    Posts
    25
    Rep Power
    0

    Default int array - any other data structure that can handle large sets?

    Ok, I'm reading audio files with InputStream and I want to save the bytes in an int array:

    Java Code:
    InputStream urlDownload = url.openStream();
    int bytesRead;
    ArrayList<integer> data = new ArrayList<Integer>();
    while((bytesRead = urlDownload.read()) != -1) {
           data.add(bytesRead);
    }
    
    bytes = new int [data.size()];
    for (int i=0; i<data.size(); i++) {
          bytes[i] = data.get(i);
    }
    For an audio file that is just about 3MB (3,000KB) i get a

    Java Code:
    java.lang.OutOfMemoryError: Java heap space
    at this line:

    Java Code:
    bytes = new int [data.size()];
    The data ArrayList at this point has a size of over 3 million. So I'm guessing the bytes array can't handle that size. Now I tried changing the heap memory with no success. Is there any other equivalent to the int array I can use? Or how exactly do I correctly change the heap space. I found several places in a search here on how to do that but the solutions just did not work.

    Thanks.

  2. #2
    Fubarable's Avatar
    Fubarable is offline Moderator
    Join Date
    Jun 2008
    Posts
    19,316
    Blog Entries
    1
    Rep Power
    26

    Default

    bytes array can't handle that size
    I believe that this assumption is incorrect. I'd work further on increasing the heap size here. Do you know that you are increasing it enough?

  3. #3
    RR_QQ is offline Member
    Join Date
    Sep 2008
    Posts
    25
    Rep Power
    0

    Default

    Perhaps I'm not doing it correctly. I am using NetBeans. I found the conf file in C:\Program Files\NetBeans 6.5\etc\netbeans.conf

    I changed:

    netbeans_default_options="-J-Dorg.glassfish.v3.installRoot=\"C:\Program Files\glassfish-v3-prelude\" -J-Dcom.sun.aas.installRoot=\"C:\Program Files\glassfish-v2ur2\" -J-client -J-Xverify:none -J-Xss2m -J-Xms32m -J-XX:PermSize=32m -J-XX:MaxPermSize=200m -J-Dapple.laf.useScreenMenuBar=true -J-Dsun.java2d.noddraw=true"

    to:

    # Options used by NetBeans launcher by default, can be overridden by explicit
    # command line switches:
    netbeans_default_options="-J-Dorg.glassfish.v3.installRoot=\"C:\Program Files\glassfish-v3-prelude\" -J-Dcom.sun.aas.installRoot=\"C:\Program Files\glassfish-v2ur2\" -J-client -J-Xverify:none -J-Xss2m -J-Xms128m -J-XX:PermSize=32m -J-XX:MaxPermSize=400m -J-Dapple.laf.useScreenMenuBar=true -J-Dsun.java2d.noddraw=true"

    I close and restart NetBeans. I still get the same problem. Perhaps I'm updating the wrong config? Perhaps I'm updating the wrong values?

  4. #4
    Fubarable's Avatar
    Fubarable is offline Moderator
    Join Date
    Jun 2008
    Posts
    19,316
    Blog Entries
    1
    Rep Power
    26

    Default

    I use Eclipse not NetBeans, but I imagine that you'd set your heap size in the "VM Options" text box that's present in the Build -- Set Main Project Configuration -- Customize -- Run (category) menu.

  5. #5
    RR_QQ is offline Member
    Join Date
    Sep 2008
    Posts
    25
    Rep Power
    0

    Default

    Ok i found it...and my program runs now. I added this to the VM Options: -Xms64m -XX:MaxPermSize=400m -Xmx256m

    Are all those options needed? what would you add/remove?

  6. #6
    neilcoffey is offline Senior Member
    Join Date
    Nov 2008
    Posts
    286
    Rep Power
    6

    Default

    Drop the -XX:MaxPermSize option unless you're reeeally sure you want it and know what it does.

  7. #7
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    3 million times 4 = 24 million

    You're array just took 3/4 of the 32m of memory you allocated. You left 8m for everything else, which means you left nothing since the VM generally doesn't actually give you everything you ask for.

    What you're doing is very inefficient with the arraylist. It makes more sense to read an InputStream into a byte[]. Worse, if this is encoded audio rather than just a raw byte stream (which it almost certainly is if you're reading it from a URL), then putting the bytes into an int[] is probably not even going to work.

  8. #8
    RR_QQ is offline Member
    Join Date
    Sep 2008
    Posts
    25
    Rep Power
    0

    Default

    OK! thanks sooo much for the advice. I'm just really new to all this and yes I read an InputStream into a byte[]. Thanks for the tips!

Similar Threads

  1. data file structure
    By Nicholas Jordan in forum Advanced Java
    Replies: 2
    Last Post: 01-07-2009, 04:16 AM
  2. Replies: 1
    Last Post: 12-28-2008, 10:25 AM
  3. parsing/storing large text data
    By hkansal in forum New To Java
    Replies: 4
    Last Post: 10-19-2008, 06:34 PM
  4. Queue data structure
    By Java Tip in forum java.lang
    Replies: 0
    Last Post: 04-14-2008, 08:35 PM
  5. data structure code
    By vgvt in forum New To Java
    Replies: 1
    Last Post: 01-17-2008, 02:49 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •