Results 1 to 9 of 9
  1. #1
    gmcouto is offline Member
    Join Date
    May 2010
    Posts
    5
    Rep Power
    0

    Default [SOLVED] Slow Output reading with Runtime.getRuntime().exec()

    Hello, it is my first post here. I really need your help guys.

    I am developing a GUI for a C program. Im using Java to execute a command with Runtime.getRuntime.exec().... and using a different thread to monitor the InputStream.

    The program generetas several lines of output per second.
    I need to read the program output and show the progress on the GUI.

    The problem is that it takes a lot of seconds to start printing something... then it prints everything that it should have printed before... and stops printing for a lot more seconds(like 10+ seconds)... then it prints everything again just once.... and stops.... and so on until the program finishes.

    Java Code:
    public class CommandLineRun {
        Process p;
        Runtime rt;
        BufferedReader stdInput, stdError;
        private boolean running;
    
        public CommandLineRun(File f,String[] args){
            rt = Runtime.getRuntime();
            try {
                p = rt.exec(f.getCanonicalPath(), args, f.getCanonicalFile().getParentFile());
    
                running=true;
            } catch (IOException ex) {
                Logger.getLogger(CommandLineRun.class.getName()).log(Level.SEVERE, null, ex);
            }
            stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
            stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
        }
        public BufferedReader getOutput(){
            return stdInput;
        }
        public BufferedReader getError(){
            return stdError;
        }
        public void halt(){
            p.destroy();
            running=false;
        }
        public boolean procDone() {
            try {
                int v = p.exitValue();
                running=false;
                return true;
            }
            catch(IllegalThreadStateException e) {
                return false;
            }
        }
        public boolean isRunning() {
            return running;
        }
    }
    Java Code:
    public class monExatoModuleOut extends Thread {
    
        CommandLineRun run;
    
        public monExatoModuleOut(CommandLineRun run) {
            this.run = run;
        }
    
        public void run() {
            String s;
            while (!run.procDone() && run.isRunning()) {
                try {
                    while (( s = run.getError().readLine()) != null) {
                        System.out.println(s);
                    }
                } catch (IOException ex) {
                    Logger.getLogger(execExatoModuleThread.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
    
        }
    }
    The fact is the program I need to execute is CPU intensive, so I'm afraid that the thread that puts info on InputStream barely gets CPU.

    Is there anything to do to fix this and print the output something about real time?

    ---
    Sorry for my poor english
    Last edited by gmcouto; 05-14-2010 at 05:49 AM.

  2. #2
    Fubarable's Avatar
    Fubarable is offline Moderator
    Join Date
    Jun 2008
    Posts
    19,315
    Blog Entries
    1
    Rep Power
    26

    Default

    The key may be in your GUI code. Is this a Swing program? If so, are you running your long-running process in a different thread than the GUI's main thread (the EDT)? If I were doing this with Java 1.6, I'd consider using a SwingWorker object to be sure that the C program is run on a background thread, and then would display the text via the SwingWorker's publish/process method pair.

  3. #3
    gmcouto is offline Member
    Join Date
    May 2010
    Posts
    5
    Rep Power
    0

    Default

    Runtime.getRuntime.exec() starts a new process, so it is definitely in a different thread(am I wrong?).

    As you can see, even the class that monitors output runs in a different thread.

    ---

    I'll try to do something with SwingWorker. brb.

    Thanks for your help by the way

  4. #4
    Fubarable's Avatar
    Fubarable is offline Moderator
    Join Date
    Jun 2008
    Posts
    19,315
    Blog Entries
    1
    Rep Power
    26

    Default

    Quote Originally Posted by gmcouto View Post
    Runtime.getRuntime.exec() starts a new process, so it is definitely in a different thread(am I wrong?).
    Sorry, but I think this statement above is wrong. The new process is created in the thread that spawns it, and can tie up that thread.

    As you can see, even the class that monitors output runs in a different thread.
    ---
    I'll try to do something with SwingWorker. brb.
    Thanks for your help by the way
    You're welcome. If it fails, then you may with to try to create a very simple and small program that demonstrates your problem.

    Best of luck.

  5. #5
    gmcouto is offline Member
    Join Date
    May 2010
    Posts
    5
    Rep Power
    0

    Default

    Sorry, but I couldn't figure out a way to do it using SwingWorkers.
    I tought that different processes couldn't share the same thread.

    Ive just made a debug version of my "console reader" and it might be easier to understand what I want

    Java Code:
    public class CmdOutputPrinter {
    
        Process p;
        Runtime rt;
        BufferedReader stdInput;
        private String s;
    
        public CmdOutputPrinter() {
            rt = Runtime.getRuntime();
            try {
                p = rt.exec("./exato x corpus.txt");
            } catch (IOException ex) {
            }
            stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
            while (!procDone()) {
                try {
                    while ((s = stdInput.readLine()) != null) {// it hangs here, waiting to receive readLine()
                        //everything else runs fine
                        System.out.println(s);
                    }
                } catch (IOException ex) {
                    Logger.getLogger(CmdRunWorker.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
        }
    
        public boolean procDone() {
            try {
                int v = p.exitValue();
                return true;
            } catch (IllegalThreadStateException e) {
                return false;
            }
        }
    
        public static void main(String[] args) {
            new CmdOutputPrinter();
        }
    }
    So, when it comes to that point it hang... it stays there until every output is over... or at least, a lot of it(and hang again immediately again).

    So it's like you create a GUI to a program that count seconds, and print'em every second. And the problem is that the GUI just receive the results of the original program output every ten seconds.
    Last edited by gmcouto; 05-14-2010 at 02:15 AM.

  6. #6
    Fubarable's Avatar
    Fubarable is offline Moderator
    Join Date
    Jun 2008
    Posts
    19,315
    Blog Entries
    1
    Rep Power
    26

    Default

    It sounds like, smells like, and looks like a Swing thread issue. The key is how you call this in the GUI, and you may wish to show us this. I still think that a SwingWorker would work best to solve this problem. Consider creating a very small compilable program (a GUI program) that mimics this problem, and posting it here. The smaller the better (less work and easier for us).

    Much luck!

  7. #7
    gmcouto is offline Member
    Join Date
    May 2010
    Posts
    5
    Rep Power
    0

    Default

    If you look at the dummy app I made last post... It only prints the results on the console.
    There is no Swing in that test, and the problem remains...


    Even if I create a new thread to call the runtime.exec() + a thread to monitor the stream the problem persists.... So my conclusion it is not the Swing.... (as in both of cases, with/without Swing, it hangs with BufferedReader.getLine)


    And I tried to reproduce a problem compiling a sample of a C++ program... but then it works normally.
    It only happens with the program I'm trying to create a GUI.(maybe because it really uses 100% of CPU)

    ---

    This is the class that executes the program in my GUI project, and changes the values in the progress bar of my Swing panel.

    Java Code:
    public class execExatoModuleThread extends Thread {
    
        JTextArea taOut;
        JProgressBar pbRead;
        JProgressBar pbWrite;
        int qtdFiles;
        int qtdGrams;
        CommandLineRun run;
    
        public execExatoModuleThread(JTextArea ta, JProgressBar pbRead, JProgressBar pbWrite, int numFiles, int numGrams) {
            super();
            taOut = ta;
            this.pbRead = pbRead;
            this.pbWrite = pbWrite;
            qtdFiles = numFiles;
            qtdGrams = numGrams;
        }
    
        public void halt() {
            run.halt();
        }
    
        public void run() {
            taOut.setText("");
            String line;
            //mod style of bar
            pbRead.setIndeterminate(true);
            pbWrite.setIndeterminate(true);
            //change text
            pbRead.setString("Start process");
            pbWrite.setString("Start process");
            //mod style bar
            pbRead.setIndeterminate(false);
            pbWrite.setIndeterminate(false);
    
            System.out.println("\n\nPreparing to execute ExATO...\n");
            String[] xpar = {"x", "corpus.txt"};
            run = new CommandLineRun(Utils.ExATO, xpar);
    
    
            int iFiles = 0;
            int iGrams = 0;
            String lineout;
    
            while (!run.procDone() && run.isRunning()) {
                try {
                    while ((lineout = run.getOutput().readLine()) != null) {
    
                        taOut.append(lineout + "\n");
                        System.out.println(lineout);//debug
    
                        if (iFiles < qtdFiles && lineout.contains("reading file")) {
                            iFiles++;
                            int valuePBread = (100 * iFiles) / qtdFiles;
                            pbRead.setValue(valuePBread);
                            pbRead.setString(iFiles + "/" + qtdFiles);
                            System.out.println(iFiles + " of " + qtdFiles + " done!");
                        } else if (iFiles == qtdFiles) {
                            pbRead.setString("Complete Read");
                            pbRead.setValue(100);
                            if (iGrams < qtdGrams && lineout.contains("generating")) {
                                iGrams++;
                                int valuePBwrite = (100 * iGrams) / (qtdGrams);
                                pbWrite.setValue(valuePBwrite);
                                pbWrite.setString(iGrams + "/" + qtdGrams);
                            } else if (iFiles == qtdFiles) {
                                pbWrite.setString("Complete Extract");
                                pbWrite.setValue(100);
                            }
                        }
    
                        System.out.println("Working!");
                    }
                } catch (IOException ex) {
                    Logger.getLogger(execExatoModuleThread.class.getName()).log(Level.SEVERE, null, ex);
                }
            }
            System.out.println("Done!");
        }
    }
    It is important to notice that it is a Thread.

    This is the piece of code of the GUI that calls the Thread:
    Java Code:
     private void doProcess() {
            jButton2.setEnabled(false);
            jButton3.setEnabled(true);
    
            String out = parameters.toString();
            File f = new File("corpus.txt");
            if (f.exists()) {
                f.delete();
            }
            try {
                f.createNewFile();
            } catch (IOException ex) {
                Logger.getLogger(ExATO_AnalysisPanel.class.getName()).log(Level.SEVERE, null, ex);
            }
            Utils.writeTxt(out, f);//generates parameters file
            System.out.println(out);//debug
    
            eeModule = new execExatoModuleThread(jTextArea1,jProgressBar1,jProgressBar2,parameters.getFilesLength(),parameters.getGramsQuota());
            eeModule.start(); //now the real thing happens
        }
    I even tried to use a different(than the one that called Runtime.exec) thread to monitor the InputStream, but the result was the same.

    I really don't know what to do.
    Last edited by gmcouto; 05-14-2010 at 04:21 AM.

  8. #8
    gmcouto is offline Member
    Join Date
    May 2010
    Posts
    5
    Rep Power
    0

    Default

    I think I found the solution.

    Java InputStream keeps reading until the buffer is flushed or filled up. That's why it hangs on ReadLine().... because the it is filling the buffer, to then send it to InputStream.

    I could reproduce the problem with this little C++ program:
    Java Code:
    #include <iostream>
    int main(){
    	long a = 0;
    	long b = 0;
    	long c = 1;
    	while(true){
    		std::cout << "Oi " << c << "\n";//it doesnt flush, it fucks up
    		while(a<1000000){ a++; }
    		a=0;
    		while(b<1000000){ b++; }
    		b=0;
    		c++;
    	}
    }
    The only thing I did important is use "\n" for linebreak instead of endl.

    So I've researched a little further and I found that endl flushes a buffer.
    How could I be sure? I tested this code too:
    Java Code:
    #include <iostream>
    int main(){
    	long a = 0;
    	long b = 0;
    	long c = 1;
    	while(true){
    		std::cout<< "Oi " << c << "\n" << std::flush;// it flushes now, its ok
    		while(a<1000000){ a++; }
    		a=0;
    		while(b<1000000){ b++; }
    		b=0;
    		c++;
    	}
    }
    Conclusion, I need to ask the developer to change his app in need to develop a GUI that does the expected behavior.

    ---

    Sorry for the double posting

  9. #9
    JosAH's Avatar
    JosAH is offline Moderator
    Join Date
    Sep 2008
    Location
    Voorschoten, the Netherlands
    Posts
    13,772
    Blog Entries
    7
    Rep Power
    21

    Default

    Have a look at the other BufferedReader constructor; it allows for a user defined buffer size; try a value of 80 or so. If it works you (they?) don't have to fiddle-diddle with the C++ application.

    kind regards,

    Jos

Similar Threads

  1. problem with Runtime.getRuntime().exec when running java in .bat
    By Shayko in forum Threads and Synchronization
    Replies: 2
    Last Post: 01-27-2010, 08:46 PM
  2. help with Runtime.getRuntime().exec
    By collin389 in forum AWT / Swing
    Replies: 3
    Last Post: 11-09-2009, 05:22 AM
  3. Problem with Runtime.getRuntime().exec with Linux Commands
    By swapnilnawale in forum Threads and Synchronization
    Replies: 1
    Last Post: 09-23-2009, 11:23 PM
  4. Replies: 3
    Last Post: 04-02-2009, 09:16 PM
  5. grep on multiple files using Runtime.getRuntime().exec()
    By cprash.aggarwal in forum Advanced Java
    Replies: 3
    Last Post: 02-11-2009, 07:55 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •