Java Memory Issue

[personal]

I have a website that needs to upload images. My heap size is set to -Xmx352m. I was doing performance testing using jmeter uploading 10 concurrent 5mb files. I'd think everything would be ok since 10 * 5mb < 352mb. I'd notice that I'd get out of memory exceptions. I tried setting -Xmn96m (10 * 5mb < 96mb) for the "young" generation but still get errors.

BTW this is what I'm using for my
$JAVA_OPTS='-Xmx352m -Xmn96m -XX:MaxPermSize=256m'

[Tolls]

Are you turning the input stream into an Image in memory?
If so then that Image will be uncompressed and consequently much larger than 5mb.

If you want to check it out take a heap dump and look after you've read in 8 images.

[personal]

I actually am. I read the image in memory so that I can compress it into something smaller, so that later pages with multiple images load much faster. do you know if there is some ratio of compression for some file types?

[Tolls]

No idea.
But I have seen pretty big increases (orders of magnitude).

Just do a heap dump and analyse it.
Should be pretty clear how big the Image objects are getting.

Are you holding the compressed stuff in memory?
That is, are you reading in the image, creating an Image object and then scaling?

[toadaly]

Decompressed images can easily be 10x or even 100x bigger than the compressed version if it's lossy compression (which is typical).

[personal]

yes I am holding the compressed image in memory. then writing to the fs. I can post the code. hopefully I can get some advice on how to improve things

[Tolls]

You're writing to the filesystem, then discarding the image?

[drez]

Are you using JApplet?

If so, put the following

<applet code=YourApplication.class name=YourApplication width=10% height=10% >
<param name="java_arguments" value="-Xms128m -Xmx256m">


Use BufferedImage and then when you are done processing that image don't forget to flush() and then set to null

If you are using Graphics2D then don't forget to dispose() after you are done painting.

[Tolls]

In almost all instances setting something to null is a pointless exercise.
Just let it go out of scope.

[personal]

to answer peoples questions. its not an applet. i save the file to a tmp dir then read it in and resize.

i was talking to a co worker and he mentioned that i should not do this in java and instead call out to another program to do the resize so i don't have to deal with java memory issues. thoughts on that approach? any suggested command line tools?

            // load image from INFILE
            Image image = ImageIO.read(oldImage);

            // determine image size from WIDTH and HEIGHT
            int newWidth = imageSize.getWidth();
            int newHeight = imageSize.getHeight();

            double ratio = (double) newWidth / (double) newHeight;
            int imageWidth = image.getWidth(null);
            int imageHeight = image.getHeight(null);
            double imageRatio = (double) imageWidth / (double) imageHeight;
            if (ratio < imageRatio) {
                newHeight = (int) (newWidth / imageRatio);
            } else {
                newWidth = (int) (newHeight * imageRatio);
            }

            // draw original image to new image object and
            // scale it to the new size on-the-fly
            BufferedImage newImage = new BufferedImage(newWidth, newHeight, BufferedImage.TYPE_INT_RGB);
            Graphics2D graphics2D = newImage.createGraphics();
            graphics2D.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
                    RenderingHints.VALUE_INTERPOLATION_BILINEAR);
            graphics2D.drawImage(image, 0, 0, newWidth, newHeight, null);

            // save image to storage
            String newFileName = createJPGName(oldImage.getName());
            // creates empty file object with new destination
            File newFile = _fileStorage.createFile(newFileName, destinationDir);
            BufferedOutputStream out = new BufferedOutputStream(new FileOutputStream(newFile));
            JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(out);
            JPEGEncodeParam param = encoder. getDefaultJPEGEncodeParam(newImage);

            int quality = imageSize.getQuality();
            quality = Math.max(0, Math.min(quality, 100));
            param.setQuality((float) quality / 100.0f, false);
            encoder.setJPEGEncodeParam(param);
            encoder.encode(newImage);
            graphics2D.dispose();
            out.close();
            image = null;
            newImage = null;

[Tolls]

Wish I had my resizing code to hand so I could compare what I did end of last year with what you're doing.

It resized an uploaded image and stored that as a file (essentially what you're doing) and there wasn't any obvious memory issues.

I'd seriously consider taking that heap dump and simply looking at what's holding the memory. Something somewhere is being held. Oh, and setting things to null is largely pointless. If this is a self contained method then they'll be eligible for gc when the method exits.

[Ke vin Kelly]

Negative news - Syria's 'mutilation mystery' deepens...

[JosAH]

@Ke vin Kelly: please, no politics here; this is a Java forum.

kind regards,

Jos