Results 1 to 11 of 11
  1. #1
    strat1 is offline Member
    Join Date
    Feb 2009
    Posts
    2
    Rep Power
    0

    Default Peeking or pushing back into a StringTokenizer?

    Solved. No need for a flame fest.
    Last edited by strat1; 03-10-2009 at 10:44 PM.

  2. #2
    Steve11235's Avatar
    Steve11235 is offline Senior Member
    Join Date
    Dec 2008
    Posts
    1,046
    Rep Power
    8

    Default

    Read all the tokens into an ArrayList. Then, work through the ArrayList. I assume you don't process an entire source member at once. If you are, break it down by end of line markers.

  3. #3
    OrangeDog's Avatar
    OrangeDog is offline Senior Member
    Join Date
    Jan 2009
    Location
    Cambridge, UK
    Posts
    838
    Rep Power
    6

    Default

    Use a StreamTokenizer instead, it's much more powerful and already supports pushback.

  4. #4
    OrangeDog's Avatar
    OrangeDog is offline Senior Member
    Join Date
    Jan 2009
    Location
    Cambridge, UK
    Posts
    838
    Rep Power
    6

    Default

    Alternatively, just using regular expressions should be able to lex for you.

  5. #5
    Steve11235's Avatar
    Steve11235 is offline Senior Member
    Join Date
    Dec 2008
    Posts
    1,046
    Rep Power
    8

    Default

    Props to OrangeDog...

    StringTokenizer is "legacy" in 1.6, and the API suggests using String.split(), which takes advantage of regex.

    StreamTokenizer seems to be designed for parsing, and it does support push back. You can create a Reader from a String as well. It's parsing seems to be centered around C and C++, it recognizes comments, and it does allow for a certain amount of configuration.
    Last edited by Steve11235; 03-03-2009 at 04:57 PM. Reason: not to mention

  6. #6
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    This sounds like a textbook case for using regular expressions (easily accessed through String.split() ).

  7. #7
    angryboy's Avatar
    angryboy is offline Senior Member
    Join Date
    Jan 2009
    Posts
    742
    Rep Power
    6

    Default

    why reinvent the wheel?
    Generated Documentation (Untitled)

    edit:
    oops, that the doc.
    this the main site: Syntax Highlighting
    Last edited by angryboy; 03-04-2009 at 07:57 AM.
    USE CODE TAGS--> [CODE]...[/CODE]
    Get NotePad++ (free)

  8. #8
    strat1 is offline Member
    Join Date
    Feb 2009
    Posts
    2
    Rep Power
    0

    Default

    Heh, i wasn't looking for alternative ways to tackle this solution. It seems like whenever someone asks for help, people always jump to suggesting some other, off-the-wall way of solving the task, which usually never works in personal implementations.

    My goal was to use StringTokenizer and StringTokenizer only. If you really want to know why, it is because i need to split the token stream by specific delimiters, and in some instances return those delimiters. StringTokenizer is the only class to my knowledge with a constructor that can do that (StreamTokenizer certainly cant). And i dont think you understand what I'm trying to do if you suggest using regex.

    But anyways, I figured out the solution already on my own. A simple buffer method to hold the nextToken was all that was needed.

  9. #9
    toadaly is offline Senior Member
    Join Date
    Jan 2009
    Posts
    671
    Rep Power
    6

    Default

    From the Java API:

    StringTokenizer is a legacy class that is retained for compatibility reasons although its use is discouraged in new code. It is recommended that anyone seeking this functionality use the split method of String or the java.util.regex package instead.
    But if you want to do things the hard way, you're welcome to.

  10. #10
    Steve11235's Avatar
    Steve11235 is offline Senior Member
    Join Date
    Dec 2008
    Posts
    1,046
    Rep Power
    8

    Default

    Toadaly, I think someone already pointed that out to him. Let's no waste the poor man's time any further... Apparently, he knows better than anyone trying to help.

  11. #11
    angryboy's Avatar
    angryboy is offline Senior Member
    Join Date
    Jan 2009
    Posts
    742
    Rep Power
    6

Similar Threads

  1. StringTokenizer Question
    By coder09 in forum New To Java
    Replies: 5
    Last Post: 02-26-2009, 01:46 AM
  2. Help with StringTokenizer!
    By ookie833 in forum New To Java
    Replies: 13
    Last Post: 12-14-2008, 05:09 PM
  3. StringTokenizer
    By carderne in forum New To Java
    Replies: 1
    Last Post: 01-26-2008, 09:19 PM
  4. StringTokenizer
    By Java Tip in forum Java Tip
    Replies: 0
    Last Post: 11-08-2007, 09:48 AM
  5. StringTokenizer
    By Java Tip in forum Java Tip
    Replies: 0
    Last Post: 11-03-2007, 10:24 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •