Problem with delim in StringTokenizer

[acp26b]

I am having a problem with StringTokenizer, i would like to delim on a string "->" the problem i am having is the strings i am tokenizing sometimes have a '-' or a '>' in them, and when ever it finds either it counts it as a token, is there a way to force it to use a string to delim or am i going to have to write my own? I have written an example below, any help would be much appreciated.


Code:

import java.util.*;
class testtoken{
	public static void main (String [] args){
		int i;
		String [] testtoken = new String[4];
		testtoken[0] = "Test->Token->Test"; 
		testtoken[1] = "Test->To-ken->Test";
		testtoken[2] = "Test->To>ken->Test";
		testtoken[3] = "Test->To->ken->Test";
						
		for(i = 0; i < testtoken.length; i++){
			System.out.println("String: " + testtoken[i]);
			StringTokenizer st = new StringTokenizer(testtoken[i],"->");
			
			while(st.hasMoreTokens()){
				System.out.println("\t" + st.nextToken());
			}
		}
	}
}

Output:

String: Test->Token->Test
        Test
        Token
        Test
String: Test->To-ken->Test
        Test
        To
        ken
        Test
String: Test->To>ken->Test
        Test
        To
        ken
        Test
String: Test->To->ken->Test
        Test
        To
        ken
        Test

Desired Output:

String: Test->Token->Test
        Test
        Token
        Test
String: Test->To-ken->Test
        Test
        Token
        Test
String: Test->To>ken->Test
        Test
        Token
        Test
String: Test->To->ken->Test
        Test
        To
        ken
        Test

[neilcoffey]

Dealing with different delimiters with StringTokenizer is a bit fiddly. StringTokenizer has largely been superseded by the String.split() method, and the regular expressions API in general. Try the following:

String[] toks = str.split("\\b(-|>|->)\\b");

In the pattern you pass into split():
\\b means a "word boundary"
| means an "alternative"

To find out more, read about Java regular expressions (link is to one of my tutorials; see also the documentation of the Pattern class).

[acp26b]

Thanks Neil, it is now working perfectly!

import java.util.*;
class token{
	public static void main (String [] args){
		int i;
		String [] testtoken = new String[4];
		testtoken[0] = "Test->Token->Test"; 
		testtoken[1] = "Test->To-ken->Test";
		testtoken[2] = "Test->To>ken->Test";
		testtoken[3] = "Test->To->ken->Test";
						
		for(i = 0; i < testtoken.length; i++){
			System.out.println("String: " + testtoken[i]);
					
			String[] toks = testtoken[i].split("\\b(->)\\b");

			for(int t = 0; t < toks.length; t++){
				System.out.println("\t" + toks[t]);
				
			}
		}
	}
}

Output:

String: Test->Token->Test
        Test
        Token
        Test
String: Test->To-ken->Test
        Test
        To-ken
        Test
String: Test->To>ken->Test
        Test
        To>ken
        Test
String: Test->To->ken->Test
        Test
        To
        ken
        Test