Results 1 to 2 of 2
- 03-01-2011, 11:06 PM #1Member
- Join Date
- Mar 2011
- Rep Power
how to run parallel processes from a java program ?
My use case is to split a huge gz file into multiple files and then run another java code in parallel on these multiple files and exit the main program when these all child programs finish.
My question is
how can I do it in java ?:
i) what is best way to split a huge file (20-30 GB) into smaller chunks ?
ii) what is best way to run lot of parallel processes from a java program, where all processes do the same thing and main process should exit when these processes finish.
iii) Also, is it possible for main program to get exception if any of the child fails and it can re-submit this child again after correcting error condition .
- 03-02-2011, 03:04 AM #2
have a look at the Java Parallel Processing Framework
They have an API to facilitate creating workers and message passing infrastructure.
For reading a large tar.gz file ,that part might be single thread only, as reading a stream generally is sequential. Perhaps what can be done is the large file gets un-tarred into individual files and folders into a temporary or work folder, and as a file is created, an event is fired to say "this is a file that needs working on" and use that File handle as the token or input to the work order to be dispatched to a processing node.
- By mapred.learn in forum New To JavaReplies: 2Last Post: 03-02-2011, 01:45 AM
- By naman.jain14 in forum NetworkingReplies: 0Last Post: 02-22-2011, 06:57 PM
- By abhishek.jain in forum NetBeansReplies: 7Last Post: 12-29-2009, 02:52 PM
- By Java Tip in forum Java SoftwareReplies: 0Last Post: 05-06-2008, 06:44 PM
- By JavaBean in forum Java SoftwareReplies: 0Last Post: 11-17-2007, 03:12 PM