Results 1 to 8 of 8
  1. #1
    Arigo is offline Member
    Join Date
    Apr 2011
    Posts
    11
    Rep Power
    0

    Default String Tokenizer help

    Hello, I'm having some trouble with a program I am trying to make - This program is supposed to go through a phrase, or sentence entered by the user, and replace any four letter word with four asterisks (e.g four gets replaced with ****)

    I need this to be pretty basic - What I have so far is a while phrase.hasMoreTokens loop.. Is it possible to individually select each token and replace it with the asterisks? I was thinking something along this lines of.. "If this token has 4 characters then replace with 4 asterisks". I am a bit confused as how I can do this.. Or if there is a better way. Thanks

  2. #2
    pbrockway2 is offline Moderator
    Join Date
    Feb 2009
    Location
    New Zealand
    Posts
    4,565
    Rep Power
    12

    Default

    Is it possible to individually select each token and replace it with the asterisks?

    What StringTokenizer gives you is a method, nextToken(), which returns small strings (one per token) one at a time.

    The tokens "as a whole" don't exist anywhere so there's no real question of "replacing" one token with another. Rather you get the tokens one at a time and you have to do something with each and every one of them. For instance if you want to end up with another string which is like the first but sanitised you might do something like:

    Java Code:
    create an output string
    while there are more tokens
        append a space to the output
        get the next token
        if it has length 4
            append "****" to the output
        else
            append the token itself to the output

    A StringBuilder is often used to append (concatenate) strings.

    ------------------

    The StringTokenizer docs linked to above state "StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead", although it has to be admitted that using regexes requires some work.

    There is also BreakIterator which locates words in text taking locale and other linguistic considerations into account (hyphenation, punctuation etc).

  3. #3
    Arigo is offline Member
    Join Date
    Apr 2011
    Posts
    11
    Rep Power
    0

    Default

    Quote Originally Posted by pbrockway2 View Post
    What StringTokenizer gives you is a method, nextToken(), which returns small strings (one per token) one at a time.

    The tokens "as a whole" don't exist anywhere so there's no real question of "replacing" one token with another. Rather you get the tokens one at a time and you have to do something with each and every one of them. For instance if you want to end up with another string which is like the first but sanitised you might do something like:

    Java Code:
    create an output string
    while there are more tokens
        append a space to the output
        get the next token
        if it has length 4
            append "****" to the output
        else
            append the token itself to the output

    A StringBuilder is often used to append (concatenate) strings.

    ------------------

    The StringTokenizer docs linked to above state "StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead", although it has to be admitted that using regexes requires some work.

    There is also BreakIterator which locates words in text taking locale and other linguistic considerations into account (hyphenation, punctuation etc).
    That is kinda what I have been trying to do, but how can I get the length of the individual tokens, and what do you mean by "append a space to the output"

    Edit: Heres what I have so far.. Im getting a NoSuchElement exception..

    This is what I'm trying to get it to do..
    While there are more tokens in the phrase, it should then go to the first token, count the amount of characters(which is index), if this token has an index of 3, then make it ****, and if not, output it as it is? I am still a bit confused heh.

    Java Code:
    StringTokenizer phrase = new StringTokenizer (input, " ", true);
            while (phrase.hasMoreTokens ())
            {
                for (index = 0 ; index < input.length () ; index++)
                {
                    currentChar = input.charAt (index);
                    currentStr = String.valueOf (currentChar);
                    test = phrase.nextToken ();
                    if (index == 3)
                    {
                        test = "****";
                        c.print (test);
                    }
                    else
                    {
                    test  = test;
                    }
                }
            }
    Last edited by Arigo; 04-17-2011 at 11:38 PM.

  4. #4
    sunde887's Avatar
    sunde887 is offline Moderator
    Join Date
    Jan 2011
    Location
    Richmond, Virginia
    Posts
    3,069
    Blog Entries
    3
    Rep Power
    8

    Default

    You don't need the nested for loop. In the while loop extract the next token and then test it. You will need a string or stringbuilder declared before the loop so you can concatenate the items as you go through the loop.

  5. #5
    Junky's Avatar
    Junky is offline Grand Poobah
    Join Date
    Jan 2011
    Location
    Dystopia
    Posts
    3,782
    Rep Power
    7

    Default

    Quote Originally Posted by Arigo View Post
    but how can I get the length of the individual tokens
    The StringTokenizer.nextToken method returns a String. What methods does the String class have? Is there a method that can help?

    and what do you mean by "append a space to the output"
    You have to add a space between each token/word. Otherwiseyouroutputwilllooklikethis.

  6. #6
    Arigo is offline Member
    Join Date
    Apr 2011
    Posts
    11
    Rep Power
    0

    Default

    Quote Originally Posted by Junky View Post
    The StringTokenizer.nextToken method returns a String. What methods does the String class have? Is there a method that can help?


    You have to add a space between each token/word. Otherwiseyouroutputwilllooklikethis.
    Thanks for not directly giving me the answer. My code ended up looking like this.

    Java Code:
        {
    
            int index = 0;
            String word;
            String censor = "";
    
            StringTokenizer phrase = new StringTokenizer (input, " ", true);
            while (phrase.hasMoreTokens ())
            {
                word = phrase.nextToken ();
                if (word.length() == 4)
                {
                    word = "****";
                    c.print (word);
                }
                else
                {
                    c.print(word);
                }
            }
    Which seems to be working okay. Although probably not as efficient as it could be.

  7. #7
    pbrockway2 is offline Moderator
    Join Date
    Feb 2009
    Location
    New Zealand
    Posts
    4,565
    Rep Power
    12

    Default

    I'm glad you've got it doing what you want.

    Just to explain the business about appending a space: you have used the StringTokenizer constructor that tells it to return the delimiter ie the space. In this case you just print the space (because its length is 1). Had you used the some other constructor which does not return the space as if it were a word you would have to insert a space. Only in that case would you have to say something like

    Java Code:
    else
    {
        c.print(" ");
        c.print(word);
    }

  8. #8
    Arigo is offline Member
    Join Date
    Apr 2011
    Posts
    11
    Rep Power
    0

    Default

    Quote Originally Posted by pbrockway2 View Post
    I'm glad you've got it doing what you want.

    Just to explain the business about appending a space: you have used the StringTokenizer constructor that tells it to return the delimiter ie the space. In this case you just print the space (because its length is 1). Had you used the some other constructor which does not return the space as if it were a word you would have to insert a space. Only in that case would you have to say something like

    Java Code:
    else
    {
        c.print(" ");
        c.print(word);
    }
    Yeah, okay - That makes sense. Thanks I appreciate it.. Now onto the next one.. hehe

Similar Threads

  1. String Tokenizer
    By udit ajmera in forum Java Software
    Replies: 0
    Last Post: 03-05-2011, 06:35 PM
  2. Need help with string tokenizer
    By ShortIt in forum New To Java
    Replies: 1
    Last Post: 02-18-2011, 07:04 PM
  3. String Tokenizer help
    By GreenTea in forum New To Java
    Replies: 4
    Last Post: 10-30-2010, 02:44 AM
  4. String Tokenizer
    By viperlasson in forum New To Java
    Replies: 1
    Last Post: 03-09-2010, 01:14 PM
  5. string tokenizer
    By twinytwo in forum New To Java
    Replies: 2
    Last Post: 03-26-2009, 02:10 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •