Printable View
I receive compressed data in chunks. Instead of saving the whole data and then uncompressing it I want to directly perform the uncompression for each chunk.
Can a ZipInputStream be used to uncompress chunks of data? If the answer is yes, how can this be done given the fact that I have only one ZipEntry received in multiple chunks. If the answer is no, please tell me how to solve this problem.
Thank you
A ZipInputStream wraps an ordinary InputStream; it isn't much trouble to create an InputStream that reads those chunks and passes them as if they were a continuous stream of bytes.
kind regards,
Jos
Can a compressed file be divided anywhere in its stream of bytes or are there boundaries that must be maintained?
You may not be able to divide a compressed file without understanding its internals.
Ok, Thank you, I succeeded uncompressing the data in chunks.
However I have another issue: the uncompression is really slow. The inputStream points to a database Blob. I tried to uncompress 500Kb/chunk in order to save memory.
while ((count = zin.read(data, 0, chunckSize)) != -1)
{
LOGGER.debug("Reading chunck count="+count+", time="+(System.currentTimeMillis()-time));
time = System.currentTimeMillis();
fout.write(data, 0, count);
LOGGER.debug("Writting chunck count="+count+", time="+(System.currentTimeMillis()-time));
time = System.currentTimeMillis();
}
The above logs are displaying count~512 bytes.
I read that ZIpInputStream uses a PushbackBuffer of 512 bytes. How can I improve the performance(time) of the uncompression.
Wrap your chunk reading InputStream around a BufferedInputStream?
kind regards,
Jos
It's already wrapped with a BufferedInputStream.
It's pretty strange. I also tried this with a FileInputStream because I thought it might be the database reading from a BLOB. It worked much faster but the ammount of read data was still around 512.
What is the value of 'chunkSize' in your code?
kind regards,
Jos
The value is 400000