Results 1 to 3 of 3
  1. #1
    aneuryzma is offline Member
    Join Date
    Aug 2008
    Posts
    46
    Rep Power
    0

    Default Issues with parsing huge file

    I'm parsing a document and writing to disk pairs such as these ones:
    0 vs 1, true
    0 vs 2, false
    0 vs 3, true
    1 vs 2, true
    1 vs 3, false
    ..
    and so on.

    Successively i'm balancing the trues and falses rows for each instance, by removing random lines (lines with true value if they exceed, and viceversa) and I end up with a file such as this one:
    0 vs 1 true
    0 vs 2 false
    1 vs 2 true
    1 vs 3 true
    1 vs 4 false
    1 vs 5 false

    The falses are usually much much more than trues, so in the previous example, I could keep only 1 false for isntance 0, and only 2 falses for instance 1.

    I'm doing this process in 2 steps, before parsing and then balancing.


    Now, my issue is that the unbalanced file is too big: more than 1GB, and most of its rows are going to be removed by the balancing step.

    My question is: can I balance the rows while parsing ?

    My guess is no, because I don't know which items are arriving and I can't delete any row until when all rows for a specific instance have been discovered.

    I hope it is clear.
    thanks

  2. #2
    Tolls is online now Moderator
    Join Date
    Apr 2009
    Posts
    12,182
    Rep Power
    20

    Default

    I have absolutely no idea what any of that means.
    There is so little useful requirements in there that I can't make head nor tail of it all.

  3. #3
    Skiller is offline Member
    Join Date
    Jan 2011
    Posts
    67
    Rep Power
    0

    Default

    Quote Originally Posted by aneuryzma View Post
    My question is: can I balance the rows while parsing ?
    I have no idea what you mean by "balance" but if a row can be "balanced" without needing to know about another row in the file then you could "balance" the row while parsing.

Similar Threads

  1. HTML FILE parsing.
    By makpandian in forum New To Java
    Replies: 8
    Last Post: 11-02-2010, 04:00 PM
  2. parsing xml file
    By mp.saaravanan in forum New To Java
    Replies: 1
    Last Post: 05-02-2010, 03:55 PM
  3. parsing and updating html file using JSP.
    By tskarthic in forum JavaServer Pages (JSP) and JSTL
    Replies: 1
    Last Post: 04-02-2009, 10:12 PM
  4. display huge size file in jtextarea
    By tskarthic in forum AWT / Swing
    Replies: 1
    Last Post: 03-21-2009, 07:30 PM
  5. Parsing XML File
    By rstar7 in forum XML
    Replies: 1
    Last Post: 10-10-2008, 04:31 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •