Results 1 to 10 of 10
Thread: Cleaning txt file
- 10-08-2008, 04:55 PM #1
Member
- Join Date
- Oct 2008
- Posts
- 4
- Rep Power
- 0
Cleaning txt file
I need some advise on a direction. I have a file (log file from application) that I need to "scrub". Basically I need to evaluate each line in the file against a list of strings and remove the entire line then write it to another file.
I am not asking for anyone to write the code for me. I am only asking for a direction.
Any thoughts or ideas?
ONSLast edited by ONS; 10-08-2008 at 05:49 PM.
- 10-08-2008, 05:56 PM #2
To remove text from a file, the file has to be rewritten with the desired content.
One way is to read the original file and copy to another work file what you want to save. At EOF, rename/delete the original file and rename the work file to the original name.
- 10-09-2008, 12:20 PM #3
- Join Date
- Jul 2007
- Location
- Colombo, Sri Lanka
- Posts
- 11,370
- Blog Entries
- 1
- Rep Power
- 26
- 10-09-2008, 02:01 PM #4
Member
- Join Date
- Oct 2008
- Posts
- 4
- Rep Power
- 0
Thank you both for the response. I know my post was vague and apologize. The direction I am looking for is more about evaluating the line of data. I have a list of 80 words or phrases that I need to remove from the file. Not only remove from the file but remove the entire line that the word or phrase resides. I would prefer not to hard code each of these as there will come a day when I need to add or delete one.
ONS
- 10-09-2008, 02:28 PM #5
To see if a line contains a string, see the indexOf method.
Put the list of words in a file and read the file.
- 10-09-2008, 10:19 PM #6
Member
- Join Date
- Oct 2008
- Posts
- 4
- Rep Power
- 0
Cleaning txt file
Thank you!
I have most of my code written. I am using the indexOf to find the string and I am reading the pattern matching strings into an array. I am assuming that there is no easy way to pattern match these. If I have an log file that is 40,000 lines and a pattern matching file that has 100 lines, I will end up reading up to 4,000,000 times.
In short, I will have to read the 100 line pattern file for each of the 40,000 lines.
Here is my unfinished code. I have code in there just to test parts.
Again, I am not looking for someone to write the code for me just get me going in the right direction.
Java Code:import java.io.*; class copyfile_1 { public static void main(String[] args) { String [] aPattern = new String[10]; int count = 0; try { File inputFile = new File(args[0]); File outputFile = new File(args[1]); File patternFile = new File(args[2]); BufferedReader fis = new BufferedReader(new FileReader(inputFile)); BufferedWriter fos = new BufferedWriter(new FileWriter(outputFile)); BufferedReader pis = new BufferedReader(new FileReader(patternFile)); String line = null; String pline = null; String Pattern = "SELECT "; while ((pline = pis.readLine ()) != null) // Read in pattern matching file { aPattern[count] = pline; System.out.println(pline); } while ((line = fis.readLine()) != null) // Read in source file { if (line.indexOf(Pattern) > 0 ) // Pattern matching { fos.write(line); fos.newLine(); } else { fos.write(""); } } fis.close(); fos.close(); } catch (FileNotFoundException e) { System.err.println("FileStreamsTest: " + e); } catch (IOException e) { System.err.println("FileStreamsTest: " + e); } } }
- 10-09-2008, 10:31 PM #7
Senior Member
- Join Date
- Aug 2008
- Posts
- 384
- Rep Power
- 13
Because I'm in a good mood:
Java Code:public final boolean removeFromFile(final String file, final String toRemove, boolean start) { if (isinFile(file, toRemove)) { try { File temp = new File(file+".tmp"); File file1 = new File(file); BufferedReader reader = new BufferedReader(new FileReader(file1)); BufferedWriter writer = new BufferedWriter(new FileWriter(temp)); String line; while ((line = reader.readLine()) != null && line.trim().length() > 0) if ((!start && !line.equalsIgnoreCase(toRemove)) && !isinString(line,toRemove) || (start && !line.startsWith(toRemove))) writer.write(line+"\n"); else if (!start && isinString(line,toRemove)) writer.write(removeFromString(line,toRemove)+"\n"); reader.close(); writer.flush(); writer.close(); if (!file1.delete()) { echo("Error deleting file "+file1); return false; } if (!temp.renameTo(file1)) { echo("Error renaming file "+temp+" to file "+file1); return false; } } catch (Exception e) { echo("Error in boolean removeFromFile(String, String, boolean): "+e); e.printStackTrace(); return false; } return true; } return false; }
I die a little on the inside...
Every time I get shot.
- 10-09-2008, 11:24 PM #8have to read the 100 line pattern file for each of the 40,000 lines.
Why not read the 100 patterns one time and save them in a collection before trying to match them against the records in the log file.
- 10-10-2008, 08:26 PM #9
Member
- Join Date
- Oct 2008
- Posts
- 4
- Rep Power
- 0
I started reading about collection and it seems that it will work better than my original plan. This is the first I have worked with collection and will take me a little time to digest.
ONS
- 10-10-2008, 10:50 PM #10
Similar Threads
-
File fp = new File(filePath);fp.exists() does not yeild proper result
By ganeshp in forum Advanced JavaReplies: 2Last Post: 04-07-2009, 06:25 AM -
Cleaning out the closet
By Norm in forum Java SoftwareReplies: 6Last Post: 10-31-2008, 12:18 PM -
code for cleaning variables?
By Ak-Emm in forum New To JavaReplies: 13Last Post: 08-22-2008, 02:40 AM -
To open an image file such as Jpeg file using JAva Program
By itmani2020 in forum Advanced JavaReplies: 10Last Post: 07-11-2008, 09:57 AM -
How to parse the CSV(Comma separation values)file and validate the file using java
By padmajap13 in forum Advanced JavaReplies: 7Last Post: 05-23-2008, 03:46 AM
Bookmarks