Results 1 to 15 of 15
  1. #1
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Question ProcessBulider output

    Hi All,

    I am a newbee to java and this is my first attempt.
    I have written a shell script which connects to oracle database and performs operation and returns some output message. Script works fine from command prompt. My requirement is to make it run from a web page which I have accomplished. I am getting the output of shell script on a different web page defined by HTML form action in my jsp file. Everything is working fine.


    Below is the part of java class I have written:
    =============
    String fileName = "build_"+ request.getParameter("region");
    String arg1 = request.getParameter("release");
    String arg2 = request.getParameter("email");

    ProcessBuilder pb = new ProcessBuilder(fileName,arg1,arg2);
    Process p = pb.start();

    InputStream inputStream = p.getInputStream();
    BufferedReader br = new BufferedReader(new InputStreamReader(inputStream));

    try {
    p.waitFor();
    }
    catch (InterruptedException e)
    {
    System.out.println(e.getMessage());
    }
    while (br.ready()) {
    String out =br.readLine();
    System.out.println(out);
    response.getWriter().write(out);
    response.getWriter().write("<br>");
    }
    ======================

    As shown above, my shell script accepts 2 arguements . User enters arguements in the fields provided which are passed to shell script by processbuilder. Script runs successfully and displays output [ success/error ].

    But my problem is , the shell script takes nearly 1 hour to do its job in the database and untill then my web page keeps showing as busy. Whereas in command prompt, shell script displays messages as it runs and thus keeps the end user informed about its progress. However, when called from java it waits untill the shell script completes/terminates and finally displays all the messages at once. I want to know if there is a way to display the shell script output on web page as it continues to run in the background ( child process ).

    I tried without the waitFor() function but no luck. It still waits for shell script to exit before displaying any message.

    Hope my question is clear. Thanks all.
    Last edited by hamsa; 03-10-2014 at 11:05 AM.

  2. #2
    jashburn is offline Senior Member
    Join Date
    Feb 2014
    Posts
    219
    Rep Power
    1

    Default Re: ProcessBulider output

    Hi,

    Firstly, do wrap your code with [code] tags, e.g.,
    [code]your
    code
    here[/code]

    This preserves formatting, and adds syntax highlighting.

    Approach the problem in 2 steps:
    1. Java code to output script's output in real-time (without waiting for the script process to terminate)
    2. Output written to web page in real-time

    For #1, p.waitFor() is definitely not the right call to use. According to Process (Java Platform SE 7 ), this call causes the thread to block until the subprocess terminates. This should be pretty obvious by looking at the output from System.out.println(out).

    Instead you can write a simple Java program using most of the code that you already have to execute the script, and observe the output to see if it achieves #1. If you can't get this to work as desired, there's no point proceeding to #2. To speed up this part you can replace the script with a simple shell script containing "echo"s and "sleep"s.

    The 2 things that you should do with the Java program are:

    a) remove p.waitFor()

    b) replace
    Java Code:
    while (br.ready()) {
    String out =br.readLine();
    System.out.println(out);
    response.getWriter().write(out);
    response.getWriter().write("<br>");
    }
    with
    Java Code:
    String line = "";
    while ((line = reader.readLine()) != null) {
        System.out.println(line);
    }
    (In fact try with both versions to see what difference do they make.)

    Once you're happy with the Java program, move on to #2.

  3. #3
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Default Re: ProcessBulider output

    Thanks very much for your response.

    As suggested by you I removed waitFor() from code and tried. But still web page shows as busy until the script finishes. Is there any other possible solution?

  4. #4
    jashburn is offline Senior Member
    Join Date
    Feb 2014
    Posts
    219
    Rep Power
    1

    Default Re: ProcessBulider output to console

    Have you completed #1 by writing

    [...] a simple Java program using most of the code that you already have to execute the script, and observe the output to see if it achieves #1
    ? You mentioned you tried (a) (removed p.waitFor()), but have you also done (b) (change the way the script execution output is read)?

    If/once you have, please post your modified code here, especially the code for your simple standalone program that should show on the console (not web page) real-time output of your shell script execution. State whether the real-time output is observed with the standalone program, and if the issue now is only with #2.

  5. #5
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Default Re: ProcessBulider output to console

    Hello,

    Sorry for the late response. Got tied up with other critical work.

    My shell script displays output in real-time when run from command prompt. The output looks something like this. Below message gets displayed as the script performs some operation in the database in the background.

    Java Code:
    ____ -----------------------------------------------------------------------------------------
    ____ Script name : build_XXXXX_pkg
    ____ Date : Mon Mar 20 00:59:43 EDT 2014
    ____ -----------------------------------------------------------------------------------------
    ____ Copying files from FTP site
    ____ Files are unzipped successfully
    ____ Remove carriage returns from the files
    ____ Connection to database successful
    ____ Tables truncated successfully in database
    ____ Loading tables
    ____ Buidling pkg
    ____ Renaming package
    ____ Checking version of package
    ____ FILE IS GOOD
    ____ ***** BUILD SUCCESSFULL *****
    As you suggested , i modified the script to not to have waitFor() as below.

    Java Code:
    ProcessBuilder pb = new ProcessBuilder(fileName,arg1,arg2);
    Process p = pb.start();
    		
    InputStream inputStream = p.getInputStream();
    BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
    		 
     try {
          String line = "";
          while ((line = reader.readLine()) != null) {
          response.getWriter().write(line + "<br>");
    }
    } catch (IOException e) {
          response.getWriter().write(e.getMessage());
    }
    Even after removing waitFor() , the response was same. What other changes i need to make in this class? More help on this would be appreciated. Thank you.

  6. #6
    gimbal2 is online now Just a guy
    Join Date
    Jun 2013
    Location
    Netherlands
    Posts
    3,984
    Rep Power
    5

    Default Re: ProcessBulider output to console

    Probably you have to add a flush() when writing out a line in the web side, or otherwise the server will just buffer all the data until a certain amount has been generated.

    I already find it questionable that the posted code works, I would expect you to get an exception the second time that response.getWriter() is called. The code doesn't even compile in the state that it is posted here, there is an } too many in the try/catch handler.
    "Syntactic sugar causes cancer of the semicolon." -- Alan Perlis

  7. #7
    jashburn is offline Senior Member
    Join Date
    Feb 2014
    Posts
    219
    Rep Power
    1

    Default Re: ProcessBulider output to console

    On top of the flush, I've found that you also need to set the response header's Content-Type to text/html. The default Content-Type, plain/text, still results in the browser waiting for the script execution to complete before displaying anything, even with flush included. (Tested with Tomcat and Firefox running on Linux. Ymmv.)

    Here's what works for me (modify to suit your purposes, including exception handling):
    Java Code:
            String name = request.getParameter("name");
    
            ProcessBuilder pb = new ProcessBuilder("/home/jashburn/tmp/wait.sh", name);
            Process p = pb.start();
    
            InputStream inputStream = p.getInputStream();
            BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
    
            String line = "";
            response.addHeader("Content-Type", "text/html; charset=utf-8");
            
            Writer writer = response.getWriter();
            while ((line = reader.readLine()) != null) {
                System.out.println(line);
                writer.write(line);
                writer.flush();
            }
    I wrote in post #2 to first write a simple standalone Java program to confirm that the Process-related Java code is indeed getting the script output in real-time before proceeding to include the code into the servlet. Did you do this? If not, you're missing a trick in diagnosing similar problems yourself in the future. The diagnostic process sometimes involves pulling code apart to make sure each part works before putting them back together. This is exactly what I did for the earlier p.waitFor() issue.

  8. #8
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Default Re: ProcessBulider output to console

    Thanks all for your response.

    @jashburn
    Thanks for modifying the code to suit my purpose. It helped as I am new to java.

    Your code with writer.flush() does not work as expected. The web page still shows busy and displays output only after shell script exits. But surprisingly your code worked well on my team mate's machine. The only difference is I am using IE11 and she is using IE9. It does not work on Google Chrome either. I thought that java's strength lies in being platform independent

    BTW, I followed your suggestion to first run a simple standalone java program. I executed a simple HelloWorld.java and it printed the output immediately. So I suspect my problem drills down to bufferreader and IE version.

    Now I am looking for a way to make it browser independent. Probably I am missing something to make my code run successfully on all browsers. Any suggestions please ??

  9. #9
    jashburn is offline Senior Member
    Join Date
    Feb 2014
    Posts
    219
    Rep Power
    1

    Default Re: ProcessBulider output to console

    Strange... I'm not able to test it on IE 11 yet, but just tested this on Chrome version 33.0.1750.152 (running on Linux), and the output was displayed line-by-line as expected.

    My test shell script is very basic:
    Java Code:
    #!/bin/sh
    
    echo "one $1<br>"
    sleep 1
    echo "two $1<br>"
    sleep 1
    echo "three $1<br>"
    sleep 1
    echo "four $1<br>"
    sleep 1
    echo "done $1<br>"
    I doubt this has anything to do with Java's platform independence as the protocol that is used between the client (browser) and the server is HTTP. I can even see the timing of the response coming in to Chrome every second via Chome's Developer Tools (accessed in Chrome using Ctrl-Shift-I > Network tab, then refresh web page).

    Can you confirm you've added the following line to your code?
    Java Code:
    response.addHeader("Content-Type", "text/html; charset=utf-8");
    Also, can you try replacing your shell script with my test script to see if it makes a difference? (Remember to adjust ProcessBuilder's parameters.)

  10. #10
    gimbal2 is online now Just a guy
    Join Date
    Jun 2013
    Location
    Netherlands
    Posts
    3,984
    Rep Power
    5

    Default Re: ProcessBulider output to console

    Welcome to the wonderful world of the world wide web. You can flush data to the browser, that still does not guarantee when the browser will decide to start rendering the content it gets. You're at Microsoft's mercy, which it does not have.

    IMO the only "true" solution is to go about this in a whole different way; in stead of pushing the data to the client line by line, periodically pull it. That involves starting an asynchronous job in the server side which will update the progress somewhere (in the session for example) and then with javascript and ajax technology periodically polling that progress and rendering it on screen.

    But that's a pretty hefty leap to make.
    "Syntactic sugar causes cancer of the semicolon." -- Alan Perlis

  11. #11
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Default Re: ProcessBulider output to console

    Hi all,

    @jashburn

    Code worked well for me on Chrome version 33.0.1750.154 with below line. IE11 still has problem :(
    Java Code:
    response.addHeader("Content-Type", "text/html; charset=utf-8");
    I had no problem printing the output line-by-line because I used in java code itself.
    Java Code:
    pw.write( line + "<br>");
    I am happy that now atleast I can show a demo to my client in next week's meeting

    @gimbal2
    I did make an attempt to use Ajax technology before approaching this forum, but it was bit too much for me to understand and implement. I managed to get the output in same page ( without reload ) but control used to jump to 2nd page again. I could not fix it.

  12. #12
    gimbal2 is online now Just a guy
    Join Date
    Jun 2013
    Location
    Netherlands
    Posts
    3,984
    Rep Power
    5

    Default Re: ProcessBulider output to console

    I don't understand most of what you just said, but yes it is a difficult concept to grasp. If you invest time into learning how to use something like JQuery, it becomes easier because it has standardized ways of doing things (such as implementing a polling mechanism).
    "Syntactic sugar causes cancer of the semicolon." -- Alan Perlis

  13. #13
    jashburn is offline Senior Member
    Join Date
    Feb 2014
    Posts
    219
    Rep Power
    1

    Default Re: ProcessBulider output to console

    I agree with gimbal2 that this would be better implemented using AJAX (with jQuery or otherwise). The current method (besides the IE11 issue) has a major weakness - it ties up 1 thread per request in the web server for the duration in which the response is being returned to the browser. Most web servers have a finite number of threads in their connection pool, and so if there are a lot of users running this concurrently, this can result in web server user-scalability problems.

    Not only that, if there's a firewall between the browser and the server, the long-running connection with which the script execution response is being returned may be terminated prematurely by the firewall.

    This does not occur with AJAX because the connection from the browser to the web server does not need maintained over the duration of the script execution. See Periodic Refresh - Ajax Patterns on this approach. It would be quite a big change from what you already have at the moment though. Do weigh the pros and cons against your project schedule to decide which way you'd like to proceed before we invest further to try to resolve the IE11 issue.

  14. #14
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Default Re: ProcessBulider output to console

    Thanks all.

    Got busy with other critical project work. I have communicated this issue to client and obtained some time to research.

    I added below AJAX code to my jsp page. I want to know if this is good enough.

    Java Code:
     		function display ()
            {
                    var SJAX;
                    if (window.XMLHttpRequest)
                    { SJAX=new XMLHttpRequest(); }
                     else
                    { SJAX=new ActiveXObject("Microsoft.XMLHTTP"); }
    
                                    SJAX.onreadystatechange=function() {
                    if (SJAX.readyState==4 && SJAX.status==200)
                    {   document.getElementById("msg3").innerHTML=SJAX.responseText;
                    }
                    }
                    var saveContentUrl = "http://XXXX";
                    params = "&region=" + document.getElementById('region').value;
                    params += "&release=" + document.getElementById('release').value;
                    params += "&email=" + document.getElementById('email').value;
                    SJAX.open("POST",saveContentUrl,true);
                    SJAX.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
                    SJAX.setRequestHeader("Accept", "application/html");
                    SJAX.send(params);
            }
    This gets invoked from the html form.

    Java Code:
     <input type="submit" value="Build" onclick="display(); this.form.submit() ; this.disabled=true; " />

    This setup is working fine but I could not make out how different it is from "not using ajax".

    It still does not work in IE11 [ I have stopped worrying about it as of now ]

    Now I am facing an unexpected problem with my code. The shell script which I call from java, internally calls another script which has restricted execute permission. By default the script starts running under the account of "tomcat" ( my web server starter ). And hence my script fails after some point with permission issue. This has forced me to use sudo option. Is it possible to use sudo while calling the shell script in java ??? I tried something like shown below but failed.

    Here xxx stands for the user who owns shell script.

    Java Code:
    String fileName = "sudo -u xxx build_"+ request.getParameter("region");
    String arg1 = request.getParameter("release");
    String arg2 = request.getParameter("email");
    
    ProcessBuilder pb = new ProcessBuilder(fileName,arg1,arg2);
    Process p = pb.start();

    Regards,
    Hamsa
    Last edited by hamsa; 04-07-2014 at 12:47 PM.

  15. #15
    hamsa is offline Member
    Join Date
    Mar 2014
    Location
    Bangalore , India
    Posts
    7
    Rep Power
    0

    Default Re: ProcessBulider output to console

    Hi All,

    Happy to let everyone know that I resolved the above sudo issue by using List Code is as shown

    Java Code:
    String fileName = "build_"+ request.getParameter("region");
    		String arg1 = request.getParameter("release");
    		String arg2 = request.getParameter("email");
    		
    		List<String> command = new LinkedList<String>();
    		      command.add("sudo");
    		      command.add("-u");
    		      command.add("xxx");
    		      command.add(fileName);
    		      command.add(arg1);
    		      command.add(arg2);
    
    		ProcessBuilder pb = new ProcessBuilder(command);  
    		Process p = pb.start();
    Regards,
    Hamsa

Similar Threads

  1. Reading output from console
    By Maygs in forum New To Java
    Replies: 2
    Last Post: 06-14-2013, 06:33 AM
  2. Need help with printing console output to JTextArea
    By ShinTec in forum AWT / Swing
    Replies: 4
    Last Post: 06-04-2010, 10:10 AM
  3. output to terminal instead of console on mac
    By firen in forum Advanced Java
    Replies: 0
    Last Post: 06-01-2010, 12:42 PM
  4. How Do You?? Get the Console Output as a GUI??
    By Lyricid in forum AWT / Swing
    Replies: 10
    Last Post: 11-20-2009, 11:35 PM
  5. How to align the output on console?
    By sfe23 in forum New To Java
    Replies: 5
    Last Post: 03-30-2009, 03:28 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •