Page 1 of 2 12 LastLast
Results 1 to 20 of 25
Like Tree1Likes

Thread: Tail end of BufferedWriter output file missing.

  1. #1
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Tail end of BufferedWriter output file missing.

    Hello

    The purpose of the code below is to read in a file line by line, identify lines with Chinese characters and translate them.
    All lines are then save to an output file.

    I am reading in a file, then writing it out to another directory.
    I can tell from the console output that the entire input file is being read and processed.
    The output file is identical except that about 40 lines are missing from the end.

    I have included a flush command but that doesn't seem to have any effect.
    I don't know why the lines are missing from the output file.
    What am I doing wrong??

    below is the code.

    Java Code:
    package translate;
    
    // Imports the Google Cloud client library
    import com.google.cloud.translate.Translate;
    import com.google.cloud.translate.Translate.TranslateOption;
    import com.google.cloud.translate.TranslateOptions;
    import com.google.cloud.translate.Translation; 
    import java.io.IOException;
    import java.io.BufferedReader;
    import java.io.BufferedWriter;
    import java.io.FileReader;
    import java.io.FileWriter;
    //import java.io.PrintWriter;
    
    public class Translate_Chinese_to_English {
      public static void main(String[] args) throws Exception {
      String fpI = "D:/temp/";  //source directory
      String fpO = "D:/temp/translated/";  //destination directory
      Boolean isChinese;
        // Instantiates a client
        Translate translate = TranslateOptions.getDefaultInstance().getService();
    
        for (String file : args) {  // for each of the files entered
            System.out.println("Translating: ... " + file);
             // the source file
            FileWriter fileOut = new FileWriter(fpO+file, true); //Output file set to append mode
            BufferedWriter bw = new BufferedWriter(fileOut);  //use the same filename as input.
            try(FileReader fileIn = new FileReader (fpI+file)) {
                BufferedReader br = new BufferedReader(fileIn);
                for(String line; (line = br.readLine()) != null; ) {
                    // line holds the string to translate    
                    // if there is a Mandarin character in the line, translate the whole line
                    isChinese = false;
                    for (int i = 0; i < line.length() & !isChinese ; ) {  //scan each char in the line
                        int codepoint = line.codePointAt(i);
                        i += Character.charCount(codepoint);
                        if (Character.UnicodeScript.of(codepoint) == Character.UnicodeScript.HAN) {
                            isChinese = true;  //mark for translation
                        } 
                    }    
                    if (isChinese){
                        Translation translation =
                        translate.translate(
                            line,
                            TranslateOption.sourceLanguage("zh-CN"),
                            TranslateOption.targetLanguage("en"));    
                         line = translation.getTranslatedText();                          
                         //  pw.println(translation.getTranslatedText());
                         //Google translate adds a space after "</"
                         //search and replace "</ " with "</"
                        line = line.replaceAll("<\\/ ", "</");
                        bw.write(line);
                        System.out.println("Chinese " + line);
                    } else { 
                        bw.write(line);
                        System.out.println("English " + line);
                    }   // just write the line to the output file and continue;
                }
            } catch (IOException e) { } //do nothing
            //fileIn.close();
            fileOut.flush();
            fileOut.close();       
        }  
      }
    }

  2. #2
    Norm's Avatar
    Norm is online now Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    20,001
    Rep Power
    33

    Default Re: Tail end of BufferedWriter output file missing.

    Try the code with a short simple text file. What is written out?

    Try closing bw instead.
    Last edited by Norm; 04-12-2017 at 01:50 PM.
    If you don't understand my response, don't ignore it, ask a question.

  3. #3
    SurfMan's Avatar
    SurfMan is offline Godlike
    Join Date
    Nov 2012
    Location
    The Netherlands
    Posts
    1,989
    Rep Power
    8

    Default Re: Tail end of BufferedWriter output file missing.

    While it works, it's still wrong most of the time: i < line.length() & !isChinese. Boolean "and" is written as &&.

    And +1 for flushing the BufferedWriter. Always flush the writers that you work with, not the underlying ones.

    And I like your Character.UnicodeScript.of(codepoint) solution. Where did you dig that up?
    "It's not fixed until you stop calling the problem weird and you understand what was wrong." - gimbal2 2013

  4. #4
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    13,541
    Rep Power
    26

    Default Re: Tail end of BufferedWriter output file missing.

    You never close the BufferedWriter.
    In the finally block you're flushing then closing the FileWriter, so the buffer (which lives in the BufferedWriter) is still likely to have stuff in it.

    You only need to close the BufferedWriter (and no point flushing it if you're about to close it, as close flushes it first).
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  5. #5
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Hi
    Thanks for all the advice. I have added the bw.close. The example I found on Google didn't close the BufferedWriter.
    When testing the code, I commented out all of the google translation lines, as below. I don't yet have the Google translation API installed.

    Without translation enabled, the input file should = output file.


    The problem I now have is that only the first line appears in the output file.
    The console output shows that the entire input file is being processed, and correctly detecting Mandarin/English lines.
    I always delete the output file from the previous run so the code has an empty folder to write the output file.
    Both bw.write(line); are not doing what I expect. I don't know why.

    I have included a sample of the text to be translated.

    Java Code:
    package translate;
    
    // Imports the Google Cloud client library
    /*import com.google.cloud.translate.Translate;
    import com.google.cloud.translate.Translate.TranslateOption;
    import com.google.cloud.translate.TranslateOptions;
    import com.google.cloud.translate.Translation; */
    import java.io.IOException;
    import java.io.BufferedReader;
    import java.io.BufferedWriter;
    import java.io.FileReader;
    import java.io.FileWriter;
    
    
    public class Translate_Chinese_to_English {
      public static void main(String[] args) throws Exception {
      String fpI = "D:/temp/";  //source directory
      String fpO = "D:/temp/translated/";  //destination directory
      Boolean isChinese;
        // Instantiates a translation client
        //Translate translate = TranslateOptions.getDefaultInstance().getService();
    
        for (String file : args) {  // for each of the files entered
            System.out.println("Translating: ... " + file);
             // the source file
            FileWriter fileOut = new FileWriter(fpO+file, true); //Output file set to append mode
            BufferedWriter bw = new BufferedWriter(fileOut);  //use the same filename as input.
            try(FileReader fileIn = new FileReader (fpI+file)) {
                BufferedReader br = new BufferedReader(fileIn);
                for(String line; (line = br.readLine()) != null; ) {
                    // line holds the string to translate    
                    // if there is a Mandarin character in the line, translate the whole line
                    isChinese = false;
                    for (int i = 0; i < line.length() && !isChinese ; ) {  //scan each char in the line
                        int codepoint = line.codePointAt(i);
                        i += Character.charCount(codepoint);
                        if (Character.UnicodeScript.of(codepoint) == Character.UnicodeScript.HAN) {
                            isChinese = true;  //mark for translation
                        } 
                    }    
                    if (isChinese){
                        /*Translation translation =
                        translate.translate(
                            line,
                            TranslateOption.sourceLanguage("zh-CN"),
                            TranslateOption.targetLanguage("en"));    
                         line = translation.getTranslatedText();    */                      
                         //  pw.println(translation.getTranslatedText());
                         //Google translate adds a space after "</"
                         //search and replace "</ " with "</"
                        //line = line.replaceAll("<\\/ ", "</");
                        bw.write(line);
                        System.out.println("Chinese " + line);
                    } else { 
                        bw.write(line);
                        System.out.println("English " + line);
                    }   // just write the line to the output file and continue;
                }
            } catch (IOException e) { } //do nothing
            bw.close(); 
       }  
     }
    }
    Java Code:
    <tr class="altColor">
    <td class="colFirst"><code><a href="../../../onbon/bx06/file/BxFile.Binary.html" title="class in onbon.bx06.file">BxFile.Binary</a></code></td>
    <td class="colLast"><code><strong><a href="../../../onbon/bx06/file/BxFile.html#generate()">generate</a></strong>()</code>
    <div class="block">产生文件的位元组资讯。</div>
    </td>
    </tr>
    <tr class="rowColor">
    <td class="colFirst"><code>java.lang.String</code></td>
    <td class="colLast"><code><strong><a href="../../../onbon/bx06/file/BxFile.html#getFileName()">getFileName</a></strong>()</code>
    <div class="block">取得文件名称。</div>
    </td>
    </tr>
    <tr class="altColor">
    <td class="colFirst"><code>onbon.bx06.message.common.FileType</code></td>
    <td class="colLast"><code><strong><a href="../../../onbon/bx06/file/BxFile.html#getFileType()">getFileType</a></strong>()</code>
    <div class="block">取得文件类型。</div>
    </td>

  6. #6
    Norm's Avatar
    Norm is online now Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    20,001
    Rep Power
    33

    Default Re: Tail end of BufferedWriter output file missing.

    The problem I now have is that only the first line appears in the output file.
    Strange that only the first line is written out.

    What is shown on the console when the program is executed?

    Did you try using a simple English text file for input?
    If you don't understand my response, don't ignore it, ask a question.

  7. #7
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Hi
    I am mystified.
    The console output indicates that the input file is processed right to the end, just like yesterday.
    The output file is created, just like yesterday.
    Unlike yesterday, it has just one line.

    I haven't changed any of the code related to creating and writing to the output file.
    Very frustrating.

  8. #8
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    On closer inspection, the output file has lost line breaks.
    The input file has been converted to a single line.
    I don't know why.

  9. #9
    Norm's Avatar
    Norm is online now Moderator
    Join Date
    Jun 2008
    Location
    Eastern Florida
    Posts
    20,001
    Rep Power
    33

    Default Re: Tail end of BufferedWriter output file missing.

    The input file has been converted to a single line.
    I don't know why.
    The readLine method removes the line breaks.
    If you want the output lines to have line breaks, you will need to add them in your code.
    If you don't understand my response, don't ignore it, ask a question.

  10. #10
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Hi
    I appended \r\n to the end of each line and it now all works.
    Notepad++ shows a perfect match between in and out files.

    Java Code:
                        line = line.replaceAll("<\\/ ", "</");
                        bw.write(line+"\r\n");// restore line break
                        System.out.println("Chinese " + line);
                    } else { 
                        bw.write(line+"\r\n");// restore line break
                        System.out.println("English " + line);
                    }   // just write the line to the output file and continue;
    Yesterday it worked without adding \r\n.

    Thanks for your help.

    Now I just need to get the Google API working.
    The Google documentation assumes a lot higher level of knowledge than I have.

  11. #11
    Tolls is offline Moderator
    Join Date
    Apr 2009
    Posts
    13,541
    Rep Power
    26

    Default Re: Tail end of BufferedWriter output file missing.

    Just a couple of points, don't eat exceptions (that empty catch block for the IOException).
    If something goes wrong you'll never know about it.

    Second, the close for the BufferedWriter should be in a finally block, or make it part of the try-with-resources.
    You can have multiple resources defined in there.
    Please do not ask for code as refusal often offends.

    ** This space for rent **

  12. #12
    gozzy is offline Member
    Join Date
    Jan 2017
    Posts
    46
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    I appended \r\n to the end of each line and it now all works.
    Don't do that. That will mean your code is only correct for Windows. Use PrintWriter.println and will produce the correct line breaks for the OS on which it's running

    In the finally block you're flushing then closing the FileWriter, so the buffer (which lives in the BufferedWriter) is still likely to have stuff in it.
    Of course, that problem can be avoided by the correct chaining of the IO classes. In the above case, the end of the chain and the only bit you would need to ensure closed is the PrintWriter. Do the same for the reading classes

  13. #13
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Hi
    Thanks for the additional feedback. I understand my coding is rough but I am the only one to see or use this on my machine. I just need this to work once on a bunch of files and throw it away.

    My current problem is that Netbeans is telling me the Google Translate API imports don't exist. I have the API lib jar files, a Google account and an API key but I don't know what is missing, mis-placed or mis-configured.
    Tail end of BufferedWriter output file missing.-1.png

  14. #14
    SurfMan's Avatar
    SurfMan is offline Godlike
    Join Date
    Nov 2012
    Location
    The Netherlands
    Posts
    1,989
    Rep Power
    8

    Default Re: Tail end of BufferedWriter output file missing.

    Quote Originally Posted by gozzy View Post
    Don't do that. That will mean your code is only correct for Windows. Use PrintWriter.println and will produce the correct line breaks for the OS on which it's running


    Of course, that problem can be avoided by the correct chaining of the IO classes. In the above case, the end of the chain and the only bit you would need to ensure closed is the PrintWriter. Do the same for the reading classes
    Be wary of using PrintWriter. It will encode the characters using the platform default. Since this not standard ASCII output, the default Windows encoding (ISO-8859-15 most likely) will probably fuck this up real bad. You probably would like to enforce UTF-8 for the output. I don't have an example at the ready, but just be aware that this might become an issue one day.

    If you need the platform line ending use System.lineSeparator().
    "It's not fixed until you stop calling the problem weird and you understand what was wrong." - gimbal2 2013

  15. #15
    gozzy is offline Member
    Join Date
    Jan 2017
    Posts
    46
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Be wary of using PrintWriter. It will encode the characters using the platform default.
    That's true. But it's true of any Writer, apart from the version of OutputStreamWriter that takes a charset in its ctor. If you want to set the output encoding, chain the PrintWriter on that. If you want to keep using BufferedWriter, then use BufferedWriter.newLine()

    I have included a flush command but that doesn't seem to have any effect.
    In fact, the only time you really need to call flush() is when you're consuming the result during the write. The buffers will be flushed anyway when the stream is closed.
    Last edited by gozzy; 04-13-2017 at 07:32 PM.

  16. #16
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Hi
    In this case, the source files have \r\n and I want to keep the output that way because it creates less noise doing an in/out comparison. In this case, forcing \r\n is appropriate.
    Leaving flush in did nothing so I took it out. It all works nicely now.

  17. #17
    jim829 is offline Senior Member
    Join Date
    Jan 2013
    Location
    Northern Virginia, United States
    Posts
    6,226
    Rep Power
    13

    Default Re: Tail end of BufferedWriter output file missing.

    Another suggestion. In the future, use methods for specific processing. Instead of doing it like you did above, do something like this.

    Java Code:
    for(String line; (line = br.readLine()) != null; ) {
         process(line);  // this does the translation 
     }
    Breaking up large pieces of code into smaller methods can be a big help when debugging different portions of your code. In some cases, different methods might do the same thing except one has debugging statements in it. Then, instead of adding and deleting (or commenting out) statements, you just switch to the proper method. It is cleaner, faster, and your are less prone to making mistakes.

    Regards,
    Jim
    The JavaTM Tutorials | SSCCE | Java Naming Conventions
    Poor planning on your part does not constitute an emergency on my part

  18. #18
    SurfMan's Avatar
    SurfMan is offline Godlike
    Join Date
    Nov 2012
    Location
    The Netherlands
    Posts
    1,989
    Rep Power
    8

    Default Re: Tail end of BufferedWriter output file missing.

    Quote Originally Posted by jim829 View Post
    Another suggestion. In the future, use methods for specific processing. Instead of doing it like you did above, do something like this.

    Java Code:
    for(String line; (line = br.readLine()) != null; ) {
         process(line);  // this does the translation 
     }
    Breaking up large pieces of code into smaller methods can be a big help when debugging different portions of your code. In some cases, different methods might do the same thing except one has debugging statements in it. Then, instead of adding and deleting (or commenting out) statements, you just switch to the proper method. It is cleaner, faster, and your are less prone to making mistakes.

    Regards,
    Jim
    You might as well go the Java 8 way:
    Java Code:
    br.lines().forEach(this::process);
    jim829 likes this.
    "It's not fixed until you stop calling the problem weird and you understand what was wrong." - gimbal2 2013

  19. #19
    dazz is offline Member
    Join Date
    Apr 2017
    Posts
    27
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Quote Originally Posted by SurfMan View Post
    You might as well go the Java 8 way:
    Java Code:
    br.lines().forEach(this::process);
    Hi

    I like that. I know I will be using that in the future.

    I decided to look up the instructions and found this: bufferedwriter newline()
    Whoever wrote BufferedWriter experienced or foresaw the need to restore new line characters.

    I did briefly look at implementing stream processing but the learning curve was high to justify the return on effort.

    Dazz
    Last edited by dazz; 04-15-2017 at 12:50 AM.

  20. #20
    gozzy is offline Member
    Join Date
    Jan 2017
    Posts
    46
    Rep Power
    0

    Default Re: Tail end of BufferedWriter output file missing.

    Whoever wrote BufferedWriter experienced or foresaw the need to restore new line characters.
    It's more that they know that there's a need to produce a linefeed. Actually some (older) platforms don't use the newline characters per se in linefeeds

Page 1 of 2 12 LastLast

Similar Threads

  1. Replies: 2
    Last Post: 02-19-2016, 07:08 PM
  2. Replies: 7
    Last Post: 07-02-2013, 01:59 AM
  3. Missing DLL-file
    By sotwnoz in forum New To Java
    Replies: 4
    Last Post: 02-29-2012, 04:37 PM
  4. Replies: 3
    Last Post: 04-19-2011, 06:21 PM
  5. Tail without polling
    By taotree in forum Advanced Java
    Replies: 2
    Last Post: 11-12-2008, 08:57 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •