Results 1 to 4 of 4
  1. #1
    sireesha is offline Member
    Join Date
    Nov 2007
    Posts
    50
    Rep Power
    0

    Question Tokenizing with Scanner

    Hi all,
    Please check this code..
    Java Code:
    import java.util.Scanner;
    public class Scanner_Tokenizing
    {
    public static void main(String[] args)
    {
     
        System.out.println("Enter some string:");
        Scanner s=new Scanner(System.in);
        String st;
        int count=0;
        boolean b=false;
        
       while(b=s.hasNext())
        {
            st=s.next();
            count++;
            System.out.println(st);
        }    
       System.out.println("Total number of tokens : "+count);
    }        
    }
    While running this code..it just prints all the tokens and continuously running..
    It never came to the output statement after while loop..
    What's wrong with my code ?
    Please tell me.
    Thanq very much for reading this.

  2. #2
    tim's Avatar
    tim
    tim is offline Senior Member
    Join Date
    Dec 2007
    Posts
    435
    Rep Power
    8

    Default Very challenging

    Hello sireesha

    This was very challenging for me (no IDE used ;)), but I have a solution!
    Java Code:
    import java.util.Scanner;
    import java.io.*;
    public class Scanner_Tokenizing
    {
    	public static void main(String[] args)	{
    		try{
    			System.out.println("Enter some string:");
    			[COLOR="RoyalBlue"]DataInputStream input = new DataInputStream(System.in);
    			String text = input.readLine();[/COLOR]
    			Scanner scanner = new Scanner(text);
    			int count = 0;
    			while (scanner.hasNext() [COLOR="RoyalBlue"]==[/COLOR] true){
    				String st = scanner.next();
    				count++;
    				System.out.println(st);
    			}    
    			System.out.println("Total number of tokens : " + count);
    		} catch (Exception e){
    			System.out.println(e.getMessage());
    		}
    	}
    }
    The fragments in blue are the important ones. Remember that the System.in input stream can be infinitively long. So, the scanner does not know that a new line symbol means stop. That's why I read the input stream with a DataInputStream object before hand, to obtain a finite String object to work with. Also remember that when you compare stuff, you must use "==" :D

    I hope this helps! ;)
    Last edited by tim; 02-05-2008 at 08:04 PM.
    Eyes dwelling into the past are blind to what lies in the future. Step carefully.

  3. #3
    hardwired's Avatar
    hardwired is offline Senior Member
    Join Date
    Jul 2007
    Posts
    1,576
    Rep Power
    9

    Default

    Java Code:
    import java.util.*;
    import java.util.regex.Pattern;
    
    public class Test {
        public static void main(String[] args) {
            System.out.println("Enter some string:");
            Scanner s=new Scanner(System.in);
            int count=0;
    
            while(s.hasNext()) {
                String st = s.nextLine();
                if(st.equalsIgnoreCase("x"))
                    break;
                String[] tokens = getTokens(st);
                System.out.printf("tokens = %s%n", Arrays.toString(tokens));
                count++;
            }    
            System.out.println("Total number of tokens : "+count);
            s.close();
        }
    
        private static String[] getTokens(String str) {
            System.out.println("str = " + str);
            Scanner scanner = new Scanner(str);
            //System.out.println("delimiter = " + scanner.delimiter());
            String[] s = new String[str.length()];
            // Capture the next word character, space or puntuation mark.
            String regex = "(?=[\\w\\s\\p{Punct}])";
            Pattern pattern = Pattern.compile(regex);
            scanner.useDelimiter(pattern);
            //System.out.println("delimiter = " + scanner.delimiter());
            int count = 0;
            while(scanner.hasNext("[\\w\\s\\p{Punct}]")) {
                s[count++] = scanner.next("[\\w\\s\\p{Punct}]");
            }
            scanner.close();
            return s;
        }
    }

  4. #4
    tim's Avatar
    tim
    tim is offline Senior Member
    Join Date
    Dec 2007
    Posts
    435
    Rep Power
    8

    Default Excellent

    Thanks hardwired. I did not think of using regular expressions. :D
    Eyes dwelling into the past are blind to what lies in the future. Step carefully.

Similar Threads

  1. Scanner class
    By ajaymenon.k in forum Advanced Java
    Replies: 1
    Last Post: 11-26-2007, 08:01 AM
  2. Using ava.util.Scanner
    By Java Tip in forum Java Tip
    Replies: 0
    Last Post: 11-20-2007, 05:47 PM
  3. Using the scanner method
    By silvia in forum Advanced Java
    Replies: 1
    Last Post: 08-07-2007, 06:50 AM
  4. help with IP scanner
    By tommy in forum New To Java
    Replies: 1
    Last Post: 08-06-2007, 09:00 PM
  5. JDK 5.0 Scanner Class
    By Sircedric88 in forum New To Java
    Replies: 3
    Last Post: 07-27-2007, 07:55 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •