Results 1 to 6 of 6

Thread: Java Tokenizer

  1. #1
    mike_ledis is offline Member
    Join Date
    Mar 2010
    Posts
    24
    Rep Power
    0

    Default Java Tokenizer

    hello is how can i use kind of chars to tokenize an sentence?

    suppose that i have the following sentence:
    15A-45B+25C-56H+91V

    And i want to take the following tokens
    15A
    45B
    25C
    56H
    91V


    How should i use the tokenizer class?

  2. #2
    sunde887's Avatar
    sunde887 is offline Moderator
    Join Date
    Jan 2011
    Location
    Richmond, Virginia
    Posts
    3,069
    Blog Entries
    3
    Rep Power
    8

    Default

    using String.split() is usually better, look it up.

    It's pretty simple thought, what items do you want to remove from the string?

  3. #3
    eRaaaa is offline Senior Member
    Join Date
    Oct 2010
    Location
    Germany
    Posts
    786
    Rep Power
    5

    Default

    Java Code:
    StringTokenizer tokenizer = new StringTokenizer(s, "-+");
    But the Tokenizer is a legacy class, see : StringTokenizer (Java Platform SE 6)

    Java Code:
    		String s = "15A-45B+25C-56H+91V";
    		for (String string : s.split("[-+]")) {
    			System.out.println(string);
    		}

  4. #4
    mike_ledis is offline Member
    Join Date
    Mar 2010
    Posts
    24
    Rep Power
    0

    Default

    Imagine the sentence that i gave you as an equation! I want to tokenize the equation to its vars as in the example...

  5. #5
    sunde887's Avatar
    sunde887 is offline Moderator
    Join Date
    Jan 2011
    Location
    Richmond, Virginia
    Posts
    3,069
    Blog Entries
    3
    Rep Power
    8

    Default

    I understood how to do it, a poster gave you the code which is frowned upon, however, he did show you how to do it. I was intending to get you to explain in more detail what you wanted to do, which would make doing this much easier. If for example you responded and said "I want to split the input into tokens from between +'s and -'s" You would be halfway there. From there you would simply have to look up what kind of arguments the constructor to the tokenizer class takes.

  6. #6
    Bagzli is offline Member
    Join Date
    Feb 2011
    Posts
    63
    Rep Power
    0

    Default

    Scroll up and read post by eRaaaa the line of code he gave you will work, +- are treated as two sepparate and not just 1 so it will work for your code.

Similar Threads

  1. String Tokenizer help
    By GreenTea in forum New To Java
    Replies: 4
    Last Post: 10-30-2010, 02:44 AM
  2. String Tokenizer
    By sumaih in forum Java Gaming
    Replies: 2
    Last Post: 08-21-2010, 03:23 PM
  3. Java String Tokenizer
    By zaz_rin in forum Forum Lobby
    Replies: 2
    Last Post: 03-17-2010, 04:17 AM
  4. java tokenizer
    By mia69 in forum New To Java
    Replies: 3
    Last Post: 02-06-2010, 06:41 PM
  5. string tokenizer
    By twinytwo in forum New To Java
    Replies: 2
    Last Post: 03-26-2009, 02:10 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •