Some help with regex and loop

[moaxjlou]

Hi All

I have a bit of a problem with my program.
The program that I create lets you search for a file and choose a specific .csv file.
Then using generateResult() method that I have defined, it should be able to recognize any occurence of Pattern "[a-zA-Z]\\d{5}" and return as many match as there is in the input file.

My code is as follows:

import java.io.*;
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
import java.util.*;
import java.util.regex.*;

public class Exam extends JFrame implements ActionListener{

	public JFileChooser jfc = new JFileChooser(".");
	
	JFrame frame = new JFrame();
	public JButton findButton = new JButton("Find");
    	public JButton resultButton = new JButton("Generate result");
	public JButton generateFileButton = new JButton("Generate new result file");
	public JTextField filePathField = new JTextField();
    	public File input = null;
    	public File output = null;
	public String result;

	
	public Exam(){
	
		JPanel panelTop = new JPanel();
		panelTop.add(findButton);
		panelTop.add(filePathField);
		panelTop.setLayout(new FlowLayout());
	
		JPanel panelBottom = new JPanel();
		panelBottom.add(resultButton);
		panelBottom.add(generateFileButton);
		panelTop.setLayout(new GridLayout());
	
		this.getContentPane().add(panelTop,BorderLayout.CENTER);
		this.getContentPane().add(panelBottom,BorderLayout.SOUTH);
	
		findButton.setToolTipText("Click to find the result sheet input");
		resultButton.setToolTipText("Generate result from the input");
	
		findButton.addActionListener(this);
		resultButton.addActionListener(this);
	
		this.setLocationRelativeTo(null);
		this.setDefaultCloseOperation(EXIT_ON_CLOSE);
	
		this.setTitle("Examination Decision Support Tool");
		this.setResizable(false);
	
		this.setVisible(true);
		this.pack();
	}

	
	public void getFile(){
	
		jfc.setFileSelectionMode(JFileChooser.FILES_AND_DIRECTORIES);
        	int returnVal = jfc.showOpenDialog(this);
        
		if(returnVal == JFileChooser.APPROVE_OPTION){
            		input = jfc.getSelectedFile();
			filePathField.setText(input.getPath());
        	}
	}
	
	
	public void generateResult(){
	
		if (input == null) //may need to change to if (input == null || output null) later on
			return;
		
		try{
		
			BufferedReader reader = new BufferedReader (new FileReader(input));
			//Scanner in = new Scanner(reader);
			
			Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
			Matcher matcher = candidateNumberPattern.matcher(reader.readLine());
			while(matcher.find()){
				for (int i = 1; i <= matcher.groupCount(); i++ ){
					result = matcher.group();
					
				}
					
			}
			
		
			
			
			reader.close();
		}
		
		
		catch(Exception e){
			
		}
		
	 System.out.println("The candidate number found is" + " " + result);
	
	}
	
	public void actionPerformed(ActionEvent e){
	
		if (e.getSource() == findButton)
			getFile();
	
		else if (e.getSource() == resultButton)
			generateResult();
	}
	
	public static void main(String[] args) throws IOException{

		new Exam();
		
	}
}

When tested against this test input file:
test.txt

it returns
"The candidate number found is null"

when it's supposed to return:
"The candidate number found is Z11236"
"The candidate number found is Z11243"
"The candidate number found is Z54362"

using one click of the "Generate Result" button.

1. I think the problem is with my loop, but I already ran out of idea how to do it.

2. Also, I'm not sure if it's also caused by me using the FileReader (since the input file contains a space character). Any method to read the next line after you've finished with the previous ones?

3. Can anyone help me with this, preferably giving a working example. I have spent 5 hours on this to no avail.

Really appreciate any help on this.

[Fubarable]

I recommend testing unclear concepts with a simple example, not a complex one. for instance, this is really what you are testing, isn't it?

import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class SimpleExam
{
  public String result;

  public void generateResult()
  {
    String testString = "2007/8, #Z11236  #Z11243 #Z54362";

    Pattern myPattern = Pattern.compile("[a-zA-Z]\\d{5}");
    Matcher matcher = myPattern.matcher(testString);
    while (matcher.find())
    {
      for (int i = 1; i <= matcher.groupCount(); i++)
      {
        result = matcher.group();
      }
    }

    System.out.println("The candidate number found is" + " " + result);
  }

  public static void main(String[] args)
  {
    SimpleExam simple = new SimpleExam();
    simple.generateResult();
  }
}

I'm no expert in regex, but I get better results by doing this:

import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class SimpleExam
{
  public void generateResult()
  {
    String testString = "2007/8, #Z11236  #Z11243 #Z54362";

    Pattern myPattern = Pattern.compile("[a-zA-Z]\\d{5}");
    Matcher matcher = myPattern.matcher(testString);
    while (matcher.find())
    {
      String result = matcher.group();
      System.out.println("The candidate number found is " + result);
    }
  }

  public static void main(String[] args)
  {
    SimpleExam simple = new SimpleExam();
    simple.generateResult();
  }
}

[moaxjlou]

Hi Fubarable,

The code that you suggested can now generate the desired answer (all 3 of them), but ONLY IF THEY ARE IN THE SAME LINE.

I think this is the reason that I test it against the text file, as I want the Matcher object to find matches in the next line as well.

Here is my enhanced version of code based on your sugesstions:

import java.io.*;
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import javax.swing.event.*;
import java.util.*;
import java.util.regex.*;

public class Exam extends JFrame implements ActionListener{

	public JFileChooser jfc = new JFileChooser(".");
	
	JFrame frame = new JFrame();
	public JButton findButton = new JButton("Find");
    	public JButton resultButton = new JButton("Generate result");
	public JButton generateFileButton = new JButton("Generate new result file");
    	public JTextField filePathField = new JTextField();
    	public File input = null;
    	public File output = null;
	public String result;
	public int resultPos;
	
	
	public Exam(){
	
	JPanel panelTop = new JPanel();
	panelTop.add(findButton);
	panelTop.add(filePathField);
	panelTop.setLayout(new FlowLayout());
	
	JPanel panelBottom = new JPanel();
	panelBottom.add(resultButton);
	panelBottom.add(generateFileButton);
	panelTop.setLayout(new GridLayout());
	
	this.getContentPane().add(panelTop,BorderLayout.CENTER);
	this.getContentPane().add(panelBottom,BorderLayout.SOUTH);
	
	findButton.setToolTipText("Click to find the result sheet input");
	resultButton.setToolTipText("Generate result from the input");
	
	findButton.addActionListener(this);
	resultButton.addActionListener(this);
	
	this.setLocationRelativeTo(null);
	this.setDefaultCloseOperation(EXIT_ON_CLOSE);
	
	this.setTitle("Examination Decision Support Tool");
	this.setResizable(false);
	this.setVisible(true);
	this.pack();
	
	}

	public void getFile(){
	
		jfc.setFileSelectionMode(JFileChooser.FILES_AND_DIRECTORIES);
        int returnVal = jfc.showOpenDialog(this);
        
		if(returnVal == JFileChooser.APPROVE_OPTION){
            		input = jfc.getSelectedFile();
			filePathField.setText(input.getPath());
        	}
	}
	
	public void generateResult(){
	
		if (input == null) 
		return;
		
		try{
		
			BufferedReader reader = new BufferedReader (new FileReader(input));
			//Scanner in = new Scanner(reader);
			
			Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
			Matcher matcher = candidateNumberPattern.matcher(reader.readLine());
			while(matcher.find()){
				
				result = matcher.group();
				
				System.out.println("The candidate number found is" + " " + result);
				
					
			}
			
			
			reader.close();
		}
		
		
		catch(Exception e){
			
		}
		
	 
	}
	
	public void actionPerformed(ActionEvent e){
	
		if (e.getSource() == findButton)
			getFile();
	
		else if (e.getSource() == resultButton)
			generateResult();
	}
	
	public static void main(String[] args) throws IOException{

		new Exam();
		
		
	}
}

When tested against this file (which contains only 1 line):
testOneLine.txt

The output is as follows:
"The candidate number is Z11236 "
"The candidate number is Z11243 "
"The candidate number is Z54362 "

However, when tested against multiple lines of text, as below:
testMultipleLine.txt

The result is only this:
"The candidate number is Z11236 "

From my code above, do you, or anyone else knows how the reader.readLine() can advance to the next line, and loop until all lines in the file has been visited? (So that Matcher object can find all the result of all lines)


This time, the question is not about the regex anymore, but I think the loop of BufferedReader. I can't seem to find a method that does this (advance to the next line after having read the first line)

I would appreciate any help on this.

[Norm]

You need to put the read of the next line inside a loop so that it continues reading lines until EOF.

[moaxjlou]

i modified the generateResult() method to be like this:

public void generateResult(){
	
		if (input == null) 
		return;
		
		else try{
		
			BufferedReader reader = new BufferedReader (new FileReader(input));
			
			
			Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
			Matcher matcher = candidateNumberPattern.matcher(reader.readLine());
				
					
					while(!(reader.readLine().isEmpty())){
					do{
						if(matcher.find()){
					
						result = matcher.group();
				
						System.out.println("The candidate number found is" + " " + result);
						reader.readLine();
						
						}
						}
					}
		
			
			reader.close();
		}
		
		
		catch(Exception e){
			
		}
		
	}

so this method should first visit any line (regardless if it contains the match), then if it has, then return the match.

but it returns error saying that it "reached end of file while parsing".

Anyone any idea what's happening?

[Norm]

Please post full text of error message.

What program gives you the message?

[moaxjlou]

Hi

OK I modified again the generateResult method().
This time the previous error I mentioned already gone, but a new error occured.

The new generateResult() method is below:

public void generateResult(){
	
		if (input == null) 
		return;
		
		else try{
		
			BufferedReader reader = new BufferedReader (new FileReader(input));
			Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
			
			String line = null;
			while((line = reader.readLine()) != null){
				Matcher matcher = candidateNumberPattern.matcher(reader.readLine());
				while(matcher.find()){
					result = matcher.group();
					System.out.println("The candidate number found is" + " " + result);
					
				}
						
			}
			
	
		}
		
		catch (FileNotFoundException ex) {
            ex.printStackTrace();
        }
		catch (IOException ex) {
            ex.printStackTrace();
        }
		
		finally {
            
            try {
                if (reader != null)
                    reader.close();
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }

		
		
	}

the input of my test file is (one text file with multiple lines as input):

2007/8,,,
#Z11231,, 
#Z11236,,,,
#Z11243,,,
#Z54361,, 
#Z54362,,
#Z54366

when I clicked generateResult button, it gives me the following error:

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
        at java.util.regex.Matcher.getTextLength(Matcher.java:1140)
        at java.util.regex.Matcher.reset(Matcher.java:291)
        at java.util.regex.Matcher.<init>(Matcher.java:211)
        at java.util.regex.Pattern.matcher(Pattern.java:888)
        at Exam.generateResult(Exam.java:82)
        at Exam.actionPerformed(Exam.java:141)
        at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:19
95)
        at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.jav
a:2318)
        at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel
.java:387)
        at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242
)
        at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonL
istener.java:236)
        at java.awt.AWTEventMulticaster.mouseReleased(AWTEventMulticaster.java:2
72)
        at java.awt.Component.processMouseEvent(Component.java:6041)
        at javax.swing.JComponent.processMouseEvent(JComponent.java:3265)
        at java.awt.Component.processEvent(Component.java:5806)
        at java.awt.Container.processEvent(Container.java:2058)
        at java.awt.Component.dispatchEventImpl(Component.java:4413)
        at java.awt.Container.dispatchEventImpl(Container.java:2116)
        at java.awt.Component.dispatchEvent(Component.java:4243)
        at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4322
)
        at java.awt.LightweightDispatcher.processMouseEvent(Container.java:3986)

        at java.awt.LightweightDispatcher.dispatchEvent(Container.java:3916)
        at java.awt.Container.dispatchEventImpl(Container.java:2102)
        at java.awt.Window.dispatchEventImpl(Window.java:2440)
        at java.awt.Component.dispatchEvent(Component.java:4243)
        at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
        at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThre
ad.java:273)
        at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.
java:183)
        at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThre
ad.java:173)
        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:168)

        at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:160)

        at java.awt.EventDispatchThread.run(EventDispatchThread.java:121)

Any idea how to fix this?

[moaxjlou]

I was wondering whether I should use threads for this?

[Fubarable]

I was wondering whether I should use threads for this?

Of course that won't solve your current error, but could be useful if the file that you need to parse is large.

[Fubarable]

The code that you suggested can now generate the desired answer (all 3 of them), but ONLY IF THEY ARE IN THE SAME LINE.

Fine, but you still don't need to have any Swing code here (or any file reading code, shoot use an array of Strings if you want to test multiple lines). Again, test your problem in isolation if you want to solve it.

[Norm]

at Exam.generateResult(Exam.java:82)

What is at line 82 of Exam.java?

candidateNumberPattern.matcher(reader.readLine());

It'd be better to do this in separate statements so you could check for a null return.

[moaxjlou]

What is at line 82 of Exam.java?

It is statement

Matcher matcher = candidateNumberPattern.matcher(reader.readLine());

which I put inside the while() loop.

It'd be better to do this in separate statements so you could check for a null return.

I'm not sure if you mean this:

Matcher matcher;
matcher = candidateNumberPattern.matcher(reader.readLine());

I also tried this:

Matcher matcher = null;
matcher = candidateNumberPattern.matcher(reader.readLine());

but it still gives error when I clicked the "Generate Result " button

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
        at java.util.regex.Matcher.getTextLength(Matcher.java:1140)
        at java.util.regex.Matcher.reset(Matcher.java:291)
        at java.util.regex.Matcher.<init>(Matcher.java:211)
        at java.util.regex.Pattern.matcher(Pattern.java:888)
        at Exam.generateResult(Exam.java:85)
        at Exam.actionPerformed(Exam.java:123)

I would appreciate if you could help me with this

[DarrylBurke]

I'm not sure if you mean this:

Matcher matcher;
matcher = candidateNumberPattern.matcher(reader.readLine());

I'm pretty sure that's not what he meant.

String line = reader.readLine();
if (line != null) {
   Matcher matcher = candidateNumberPattern.matcher(line);
} else {
   System.out.println("line is null");
}

db

Of course, not exactly like that or the matcher won't be visible outside the if block. The general idea is to test null before invoking a method that requires a non-null parameter.

I haven't read the whole thread, so I'll add this clarification: if it is always expeced that readLine() will return a non-null value, use this construct only for debugging and change it when you've found the problem.

[moaxjlou]

Hi

Db,

Of course, not exactly like that or the matcher won't be visible outside the if block.

I'm not sure what you mean by this. Can you clarify further?


Also, I added a try{} and catch{} clause inside the while loop of the generateResult() method as follows:

while((line = reader.readLine()) != null){
			
				try{
				Matcher matcher;
				matcher = candidateNumberPattern.matcher(reader.readLine());
					if(matcher.find()){
						result = matcher.group();
						System.out.println(result);
						
					}
				}
				catch (java.lang.NullPointerException e){}
				
			}

for reference, the new generateResult() method is now like this:

public void generateResult(){
		BufferedReader reader = null;
		String result = null;
		if (input == null) 
		return;
		
		else try{
		
			reader = new BufferedReader (new FileReader(input));
			Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
			
			String line = null;
			while((line = reader.readLine()) != null){
			
				try{
				Matcher matcher;
				matcher = candidateNumberPattern.matcher(reader.readLine());
					if(matcher.find()){
						result = matcher.group();
						System.out.println(result);
						
					}
				}
				catch (java.lang.NullPointerException e){}
				
			}
			
		
		}
		
		catch (FileNotFoundException ex) {
            ex.printStackTrace();
        }
		catch (IOException ex) {
            ex.printStackTrace();
        }
		
		finally {
     
            try {
                if (reader != null)
                    reader.close();
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }

	}

When tested against this input (a single file with multiple lines input)

2007/8, 
#Z11360,,
#Z11361,,
#Z54362,,
#Z54363,,
#Z54364,,
#Z54365,,
#Z54366,,
#Z54367,,
#Z54368,,
#Z54369,,
#Z54370,,

the output is as follows:

Z11360
Z54362
Z54364
Z54366
Z54368
Z54370

it seems to skip a line (I'm not sure the actual behaviour). What's wrong with my loop in the generateResult() method that I described above?

[Norm]

You have more than one call to readLine() and only use the value returned by one of them. The value returned by the other one is ignored.

[DarrylBurke]

Also you probably shouldn't be reasing a file on the EDT (thread "AWT-EventQueue-0"). I/O is better performed in a background thread.

db

[moaxjlou]

Hi

I have inserted a line to to the try{} clause, as well as 2 lines to the catch{} clause as follows (of the generateResult() method):

try{
	Matcher matcher;
	matcher = candidateNumberPattern.matcher(reader.readLine());
	while(matcher.find()){
		result = matcher.group();
		System.out.println(result);
		matcher = candidateNumberPattern.matcher(reader.readLine());
	}
					
}

catch (NullPointerException e){
             System.err.println("Exception occured:");
	System.out.println(e.getMessage());
}

The new generateMethod() looks like below (apologies for the uneven indentation):

public void generateResult(){
	BufferedReader reader = null;
	String result;
		
	if (input == null) 
		return;
		
		else try{
		
			reader = new BufferedReader (new FileReader(input));
			Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
			
			String line;
			
			while((line = reader.readLine()) != null){
				
				try{
				Matcher matcher;
				matcher = candidateNumberPattern.matcher(reader.readLine());
					while(matcher.find()){
						result = matcher.group();
						System.out.println(result);
						matcher = candidateNumberPattern.matcher(reader.readLine());
					}
					
				}
				catch (NullPointerException e){
				System.err.println("Exception occured:");
				System.out.println(e.getMessage());
				}
				
			}
		
		}
		
		catch (FileNotFoundException ex) {
            		ex.printStackTrace();
        	             }
		catch (IOException ex) {
            		ex.printStackTrace();
        	             }
		
	              finally {
             	  //Close the BufferedReader
            		  try {
                		if (reader != null)
                    			reader.close();
            		  } catch (IOException ex) {
                		ex.printStackTrace();
            		  }
                           }

}

When tested against the file below(one file with multiple lines):
2007/8,

#Z11360,,
#Z11361,,
#Z54362,,
#Z54363,,
#Z54364,,
#Z54365,,
#Z54366,,
#Z54367,,
#Z54368,,
#Z54369,,
#Z54370,,
#Z54371,,
#Z54372,,
#Z54373,,
#Z54374,,

it gives the following output:

Z11360
Z11361
Z54362
Z54363
Z54364
Z54365
Z54366
Z54367
Z54368
Z54369
Z54370
Z54371
Z54372
Z54373
Z54374
Exception occured:
null

It seems it gives all the occurences of the candidate number, but why is the exception still appearing?
Anyone any idea how to fix this? Please I ran out of idea already.

[Norm]

System.err.println("Exception occured:");

In addition to your message about the exception, you need to print the call stack to show where the error is occuring. When you know the source line, you can see what pointer is null and correct your code.

You still are NOT testing the return from readLine() before using it in a method call!!! Read into a variable, test it then use it.

[moaxjlou]

Hi Norm
Thanks for your suggestions. Now the output can be printed correctly.

However, I have some enhancement that need to be made. I now want to get rid of the repeated candidate numbers, how can I do that in the code?

The current while loop in my method called generateCandidateNumber() will list out ALL the occurences of the matched inputs against the specified pattern

while((candidateLine = candidateReader.readLine()) != null){
				
				try{
				Matcher candidateNumberMatcher = candidateNumberPattern.matcher(candidateLine);
				
					while(candidateNumberMatcher.find()){
						candidateNumberResult = candidateNumberMatcher.group();
						studentsData.add(candidateNumberResult);
						
					}
					
				
				}
				catch (NullPointerException e){
				System.err.println("Exception occured:");
				System.out.println(e.getMessage());
				e.printStackTrace();
				}
				
			}

The complete method is below for reference:

public ArrayList<String> generateCandidateNumber(){
		BufferedReader candidateReader = null;
		Pattern candidateNumberPattern = Pattern.compile("[a-zA-Z]\\d{5}");
		String candidateLine = null;
		String candidateNumberResult;
		ArrayList<String> studentsData = new ArrayList<String>();
		
		if (input == null) 
		return studentsData;
		
		else
		try{
		
			candidateReader = new BufferedReader (new FileReader(input));
			
			while((candidateLine = candidateReader.readLine()) != null){
				
				try{
				Matcher candidateNumberMatcher = candidateNumberPattern.matcher(candidateLine);
				
					while(candidateNumberMatcher.find()){
						candidateNumberResult = candidateNumberMatcher.group();
						studentsData.add(candidateNumberResult);
						
					}
					
				
				}
				catch (NullPointerException e){
				System.err.println("Exception occured:");
				System.out.println(e.getMessage());
				e.printStackTrace();
				}
				
			}
		
		}
		
		catch (FileNotFoundException ex) {
            ex.printStackTrace();
        }
		catch (IOException ex) {
            ex.printStackTrace();
        }
		 
		finally {
            //Close the BufferedReader
            try {
                if (candidateReader != null)
                    candidateReader.close();
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }
	
	System.out.println(studentsData);
	
	return studentsData;
	}

How do I say in the while loop to ignore all the repeated occurences?

So for instance when tested against the input

Z12345
Z12345
Z23456
Z23456

it will only print

Z12345
Z23456

I would really appreciate help on this matter.

[Norm]

I now want to get rid of the repeated candidate numbers, how can I do that in the code?

Several ways to keep track of repeated numbers.
If the max range of numbers is small enough, use an array of boolean with the number as an index to test/set number has been used.
Or if the range of numbers is too big, You could use a Collection object has unique keys like a Hashtable to keep track of unique numbers.
Or some containers have a contains() method. Not sure but that may require that you have your own class and override the equals and hashcode methods.