Merge jpeg files using one BufferedImage

[carek]

Hi. I've got huge image which I would like to encode as jpeg. Unfortunately I can't read it as one piece into memory (OOM exception) so I'd like to read it piece by piece and iteratively write it to file. Is it possible to make it with ImageIO? I found writeInsert method which is responsible for that, but it doesn't work for me. UnsupportedOperationException is thrown on each attempt of using it. Does anybody know how can I deal with that?

Thanks for any suggestions

[Norm]

Just curious. How large is the image file? What program created it?
To be able to read parts of it an make sense of those parts you would need to know the internal format of the file.
Is the file a concatenation of a number of other image files? If so, if you could detect the boundaries between the images, you could read the sub parts and make images of them.

[carek]

well the image is generated by ILog component embedded in our web UI. It's IBM product for vector graphics etc... The image can be of resolution even like 100k x 100k pixels (it's our limit, in theory it can be bigger I guess) which gives 28GB of data in RGB.
Generated picture can be zoomed, scaled, transformed, rotated, moved and much more so cutting it and reading chunks of it is quite simple through api. The problem appeared when one of our clients started to have big topology and wanted to export it into image of big resolution. (OOM due to allocating BufferedImage for whole picture)...

so the problem here is not getting chunks of image but saving them in one file one by one.

Feel free to ask more questions if you need to know anything more

[Norm]

Still not sure what you expect.
If the file contains ONE image , do you want to read any part of it and create a smaller image file from the part that was read?

What format is the large image file in?

[carek]

not exactly... The thing I want to achieve is to encode in jpeg the image that comes from ILOG (IBM's component mentioned above). ILOG has print method which fills BufferedImage object. Step by step it looks like this:

ILOG stores whole image in vector form so it's very compressed and extremely scalable.
User might want to have a big picture of his topology (ILOG can scale, zoom, move, etc...)
problem is here when picture is about to be written in BufferedImage as RGB if user wants to have 100kx100k resolution out of memory is thrown...
so I want to get part of picture scale and save as jpeg... then get second part scale and append to the jpeg file... get n'th part... well you get the idea.

and I've got everything prepared already, but I am unable to append content to jpeg because writeInsert isn't supported

still unclear? I don't know maybe I shall post some snippets of code?

[Norm]

when picture is about to be written in BufferedImage as RGB if user wants to have 100kx100k resolution

How are you going to satisfy the user with a 100kx100k image?
Are you going to divide that into 4 50kX50K image files for example?

append to the jpeg file

I have no idea if you can "append" to an existing jpg file and end up with one good jpg file.

[carek]

well making 4 or n image files out of one topology is unacceptable. I need to have one file showing whole view... and it seems that it is impossible to append content to already existing jpeg image. That's why I asked. Anyway thanks a lot maybe there will be someone who will know what can I do about it

[Norm]

Let me restate what you're trying to do.
You have a source(file) of data the can be used to create an image but the size of the image is too large to fit into memory. You want a program that will read data little by little from the source, configure it and write it out into the superlarge image and do it in some way that the whole of the image is never in memory at one time.
You might be able to do that to create a bmp file.

What can be done with this superlarge image file once it is created?

[carek]

source(file)

the source of image is web component. I don't have it as a file and what I am going to achieve is to have it as a file (more accurately it's export topology functionality of our product)

size of the image is too large to fit into memory

correct

You want a program

not a program. I'd like to write it in java ;)
I tried to use ImageIO in order to do that, but it seems that jpeg plugin doesn't allow to do that (or I don't know how and that's why I am asking ;))

You might be able to do that to create a bmp file

This part I actually didn't get. BMP in the middle of process to encode the picture as JPEG, so holding all the image in memory is impossible and that's why I need to process it in chunks.

What can be done with this superlarge image file once it is created?

The user that requested the view of his topology can then without turning on and logging in to our product see topology of his let's say company structure.

Ok. I guess some code will be needed. Here's how the picture is acquired:

BufferedImage targetImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
Graphics graphic = targetImage.createGraphics();
graphic.setColor(Color.WHITE);
graphic.fillRect(0, 0, targetImage.getWidth(), targetImage.getHeight());
graphic.setRenderingHint(RenderingHints.KEY_ANTIALIASING, RenderingHints.VALUE_ANTIALIAS_ON);
graphic.setRenderingHint(RenderingHints.KEY_TEXT_ANTIALIASING, RenderingHints.VALUE_TEXT_ANTIALIAS_OFF);
sourceManager.print(graphic, sourceArea, sourceView, transformer, false);

sourceArea is selected area I'd like to draw in graphic
sourceView is the whole topology(it is not hold as a picture so it can be in memory as a whole view)
transformer are geometry transformations like scaling, moving, rotating, etc that will be put on sourceArea before it's written in graphic

After calling print, graphic can be disposed and a picture of topology will be drawn in targetImage (it needs to be alocated with proper size, so if I try to allocate to much memory for pixels OOM is thrown)
And back to the source of the problem - If user wants to have a big resolution picture of his topology I can't select whole topology and do print on it. I need to select it piece by piece and write it to a file as jpeg

[Norm]

user that requested the view of his topology can then without turning on and logging in to our product see topology

That is what I don't understand. How is the user able to look at the image? What software/program does he have that is able to load the supersized image for viewing?

Can you talk about your project in terms of bits and bytes and programs?
You keep using your application's terminology that has no meaning for me.
Some where there is a source of bytes (a web component???). You want to read those bytes with a java program, reformat them and write them out as a jpg file. The problem you are having is that there are too many bytes to fit into your PC's memory. What you need is a java program that will read some bytes, convert them and write them out a few at a time so that the whole never needs to be in memory at one time.

[carek]

You keep using your application's terminology that has no meaning for me.

Good point. Sorry for that.

That is what I don't understand. How is the user able to look at the image? What software/program does he have that is able to load the supersized image for viewing?

Hmm interesting question :) I will ask if there's any reason for holding so big picture in one file.

...and yes, I guess your description fits the problem more or less