Results 1 to 3 of 3
  1. #1
    mapred.learn is offline Member
    Join Date
    Mar 2011
    Posts
    3
    Rep Power
    0

    Default how to run parallel processes ?

    Hi,
    My use case is to split a huge gz file into multiple files and then run another java code in parallel on these multiple files and exit the main program when these all child programs finish.

    My question is
    how can I do it in java ?:
    i) what is best way to split a huge file (20-30 GB) into smaller chunks ?
    ii) what is best way to run lot of parallel processes from a java program, where all processes do the same thing and main process should exit when these processes finish.
    iii) Also, is it possible for main program to get exception if any of the child fails and it can re-submit this child again after correcting error condition .

    Thanks,
    -JJ

  2. #2
    sunde887's Avatar
    sunde887 is offline Moderator
    Join Date
    Jan 2011
    Location
    Richmond, Virginia
    Posts
    3,069
    Blog Entries
    3
    Rep Power
    8

    Default

    This is probably going to be quite challenging if you are not fairly experienced with java. But I would imagine you are looking for threads to run multiple processes, unfortunately, there is not much else I can add.

  3. #3
    doWhile is offline Moderator
    Join Date
    Jul 2010
    Location
    California
    Posts
    1,641
    Rep Power
    7

    Default

    ii) See Lesson: Concurrency (The Java™ Tutorials > Essential Classes) for how to create threads, which will allow you to do things in parallel.
    i)As for 'the best way' of splitting a file, this depends upon what you wish to split on (delim, regex, etc...?)
    iii) Create a listener (or many listeners) that are called upon when something bad happens.

    None of the above is extremely trivial for a newcomer to the language (you don't mention your experience so I'm throwing this out there just in case)

Similar Threads

  1. two parallel arrays
    By Adomini in forum New To Java
    Replies: 12
    Last Post: 09-07-2010, 02:45 AM
  2. Too many java.exe processes due to netbeans
    By abhishek.jain in forum NetBeans
    Replies: 7
    Last Post: 12-29-2009, 02:52 PM
  3. Replies: 22
    Last Post: 09-06-2009, 06:21 PM
  4. Halt applet processes.
    By barusk in forum Java Applets
    Replies: 1
    Last Post: 04-06-2009, 06:11 PM
  5. netbeans processes donŽt die
    By karlen in forum NetBeans
    Replies: 0
    Last Post: 06-11-2007, 10:55 PM

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •