String Tokenizer help

[Arigo]

Hello, I'm having some trouble with a program I am trying to make - This program is supposed to go through a phrase, or sentence entered by the user, and replace any four letter word with four asterisks (e.g four gets replaced with ****)

I need this to be pretty basic - What I have so far is a while phrase.hasMoreTokens loop.. Is it possible to individually select each token and replace it with the asterisks? I was thinking something along this lines of.. "If this token has 4 characters then replace with 4 asterisks". I am a bit confused as how I can do this.. Or if there is a better way. Thanks

[pbrockway2]

Is it possible to individually select each token and replace it with the asterisks?

What StringTokenizer gives you is a method, nextToken(), which returns small strings (one per token) one at a time.

The tokens "as a whole" don't exist anywhere so there's no real question of "replacing" one token with another. Rather you get the tokens one at a time and you have to do something with each and every one of them. For instance if you want to end up with another string which is like the first but sanitised you might do something like:

create an output string
while there are more tokens
    append a space to the output
    get the next token
    if it has length 4
        append "****" to the output
    else
        append the token itself to the output

A StringBuilder is often used to append (concatenate) strings.

------------------

The StringTokenizer docs linked to above state "StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead", although it has to be admitted that using regexes requires some work.

There is also BreakIterator which locates words in text taking locale and other linguistic considerations into account (hyphenation, punctuation etc).

[Arigo]

What StringTokenizer gives you is a method, nextToken(), which returns small strings (one per token) one at a time.

The tokens "as a whole" don't exist anywhere so there's no real question of "replacing" one token with another. Rather you get the tokens one at a time and you have to do something with each and every one of them. For instance if you want to end up with another string which is like the first but sanitised you might do something like:

create an output string
while there are more tokens
    append a space to the output
    get the next token
    if it has length 4
        append "****" to the output
    else
        append the token itself to the output

A StringBuilder is often used to append (concatenate) strings.

------------------

The StringTokenizer docs linked to above state "StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead", although it has to be admitted that using regexes requires some work.

There is also BreakIterator which locates words in text taking locale and other linguistic considerations into account (hyphenation, punctuation etc).

That is kinda what I have been trying to do, but how can I get the length of the individual tokens, and what do you mean by "append a space to the output"

Edit: Heres what I have so far.. Im getting a NoSuchElement exception..

This is what I'm trying to get it to do..
While there are more tokens in the phrase, it should then go to the first token, count the amount of characters(which is index), if this token has an index of 3, then make it ****, and if not, output it as it is? I am still a bit confused heh.

StringTokenizer phrase = new StringTokenizer (input, " ", true);
        while (phrase.hasMoreTokens ())
        {
            for (index = 0 ; index < input.length () ; index++)
            {
                currentChar = input.charAt (index);
                currentStr = String.valueOf (currentChar);
                test = phrase.nextToken ();
                if (index == 3)
                {
                    test = "****";
                    c.print (test);
                }
                else
                {
                test  = test;
                }
            }
        }

[sunde887]

You don't need the nested for loop. In the while loop extract the next token and then test it. You will need a string or stringbuilder declared before the loop so you can concatenate the items as you go through the loop.

[Junky]

but how can I get the length of the individual tokens

The StringTokenizer.nextToken method returns a String. What methods does the String class have? Is there a method that can help?

and what do you mean by "append a space to the output"

You have to add a space between each token/word. Otherwiseyouroutputwilllooklikethis.

[Arigo]

The StringTokenizer.nextToken method returns a String. What methods does the String class have? Is there a method that can help?


You have to add a space between each token/word. Otherwiseyouroutputwilllooklikethis.

Thanks for not directly giving me the answer. My code ended up looking like this.

    {

        int index = 0;
        String word;
        String censor = "";

        StringTokenizer phrase = new StringTokenizer (input, " ", true);
        while (phrase.hasMoreTokens ())
        {
            word = phrase.nextToken ();
            if (word.length() == 4)
            {
                word = "****";
                c.print (word);
            }
            else
            {
                c.print(word);
            }
        }

Which seems to be working okay. Although probably not as efficient as it could be.

[pbrockway2]

I'm glad you've got it doing what you want.

Just to explain the business about appending a space: you have used the StringTokenizer constructor that tells it to return the delimiter ie the space. In this case you just print the space (because its length is 1). Had you used the some other constructor which does not return the space as if it were a word you would have to insert a space. Only in that case would you have to say something like

else
{
    c.print(" ");
    c.print(word);
}

[Arigo]

I'm glad you've got it doing what you want.

Just to explain the business about appending a space: you have used the StringTokenizer constructor that tells it to return the delimiter ie the space. In this case you just print the space (because its length is 1). Had you used the some other constructor which does not return the space as if it were a word you would have to insert a space. Only in that case would you have to say something like

else
{
    c.print(" ");
    c.print(word);
}

Yeah, okay - That makes sense. Thanks I appreciate it.. Now onto the next one.. hehe