Results 1 to 8 of 8
  1. #1
    alexalex1234 is offline Member
    Join Date
    Mar 2011
    Posts
    6
    Rep Power
    0

    Question Data uncompression question

    I receive compressed data in chunks. Instead of saving the whole data and then uncompressing it I want to directly perform the uncompression for each chunk.

    Can a ZipInputStream be used to uncompress chunks of data? If the answer is yes, how can this be done given the fact that I have only one ZipEntry received in multiple chunks. If the answer is no, please tell me how to solve this problem.

    Thank you

  2. #2
    JosAH's Avatar
    JosAH is online now Moderator
    Join Date
    Sep 2008
    Location
    Voorschoten, the Netherlands
    Posts
    13,368
    Blog Entries
    7
    Rep Power
    20

    Default Re: Data uncompression question

    A ZipInputStream wraps an ordinary InputStream; it isn't much trouble to create an InputStream that reads those chunks and passes them as if they were a continuous stream of bytes.

    kind regards,

    Jos
    cenosillicaphobia: the fear for an empty beer glass

  3. #3
    Norm's Avatar
    Norm is offline Moderator
    Join Date
    Jun 2008
    Location
    SW Missouri
    Posts
    17,306
    Rep Power
    25

    Default Re: Data uncompression question

    Can a compressed file be divided anywhere in its stream of bytes or are there boundaries that must be maintained?
    You may not be able to divide a compressed file without understanding its internals.
    If you don't understand my response, don't ignore it, ask a question.

  4. #4
    alexalex1234 is offline Member
    Join Date
    Mar 2011
    Posts
    6
    Rep Power
    0

    Default Re: Data uncompression question

    Ok, Thank you, I succeeded uncompressing the data in chunks.

    However I have another issue: the uncompression is really slow. The inputStream points to a database Blob. I tried to uncompress 500Kb/chunk in order to save memory.

    while ((count = zin.read(data, 0, chunckSize)) != -1)
    {
    LOGGER.debug("Reading chunck count="+count+", time="+(System.currentTimeMillis()-time));
    time = System.currentTimeMillis();
    fout.write(data, 0, count);
    LOGGER.debug("Writting chunck count="+count+", time="+(System.currentTimeMillis()-time));
    time = System.currentTimeMillis();
    }

    The above logs are displaying count~512 bytes.
    I read that ZIpInputStream uses a PushbackBuffer of 512 bytes. How can I improve the performance(time) of the uncompression.

  5. #5
    JosAH's Avatar
    JosAH is online now Moderator
    Join Date
    Sep 2008
    Location
    Voorschoten, the Netherlands
    Posts
    13,368
    Blog Entries
    7
    Rep Power
    20

    Default Re: Data uncompression question

    Wrap your chunk reading InputStream around a BufferedInputStream?

    kind regards,

    Jos
    cenosillicaphobia: the fear for an empty beer glass

  6. #6
    alexalex1234 is offline Member
    Join Date
    Mar 2011
    Posts
    6
    Rep Power
    0

    Default Re: Data uncompression question

    It's already wrapped with a BufferedInputStream.

    It's pretty strange. I also tried this with a FileInputStream because I thought it might be the database reading from a BLOB. It worked much faster but the ammount of read data was still around 512.

  7. #7
    JosAH's Avatar
    JosAH is online now Moderator
    Join Date
    Sep 2008
    Location
    Voorschoten, the Netherlands
    Posts
    13,368
    Blog Entries
    7
    Rep Power
    20

    Default Re: Data uncompression question

    What is the value of 'chunkSize' in your code?

    kind regards,

    Jos
    cenosillicaphobia: the fear for an empty beer glass

  8. #8
    alexalex1234 is offline Member
    Join Date
    Mar 2011
    Posts
    6
    Rep Power
    0

Similar Threads

  1. Replies: 2
    Last Post: 02-19-2012, 07:31 AM
  2. Getting problem inserting data in data base
    By anupama in forum New To Java
    Replies: 4
    Last Post: 12-15-2010, 10:03 PM
  3. Replies: 0
    Last Post: 10-31-2008, 02:13 PM
  4. Data Sorting in a .data file using java
    By stutiger99 in forum New To Java
    Replies: 2
    Last Post: 10-08-2008, 02:52 AM
  5. simple question about private data
    By littleBean in forum New To Java
    Replies: 12
    Last Post: 07-02-2008, 04:09 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •