[Gilgamesh]

Ive used StringTokenizer to take the words from a text. I used the delimeters ",", "." etc.

questions

1) I tried to define a final string DELIMETERS="!@#" (etc) but when i type

Code:

StringTokenizer (line, DELIMITERS);

though it recognizes the delimeters, it creates one token made by every line of the text document (without including the delimeters).
if you cant figure out the problem, can you please tell me if there is a different way to set the delimeters? except from this:

Code:

StringTokenizer (line, ".", ",", "?");

?

2)some texts, at the end of the line use a hiven to continue the word to the next line. what can I 'unify' these two tokens that consist one word?