Multi Delimeter String Parsing

Hello,

I have a string that contains |% as a delimiter. The StringTokenizer class doesn't seem to be working for this particular case. Is there something else I can use that would parse out this multi character delimiter and also return the delimiter as well?

So for example

col1|%col2|%|%

The output should be
col1
|%
col2
|%
|%

Thanks in advance for any assistance.

I usually use String.split(...) rather than StringTokenizer. You can't use "|%" as a delimiter without escaping the '|' character since it has meaning in regular expressions. Thus "\\|%" can be used as a delimiter, but even so, the delimiter will be swallowed when you split your String, so the output will be
col1
col2


For instance:

Code:

---

public class Fu1 {
  public static void main(String[] args) {
    String test = "col1|%col2|%|%";
   
    String[] tokens = test.split("\\|%", -1);
    for (String token : tokens) {
      System.out.println(token);
    }
  }
}

---

Ok thank you, the thing is that the file that I will be parsing will have some empty values which I still need to capture. If String.split() will swallow the delimiter then I will lose that information. Is this the only option?

Have you tried calling split with the second parameter, the limit < 0? It should then capture all empty values. Give it a try.

Another thing you can try is a delimiter made up of the more advanced components of regular expressions -- look ahead and look behind. This may allow you to split on your "|%" String and not swallow this String. I'm still learning this, so this code may be wrong, but if you do this, it may work:

Code:

---

public class Fu1b {
  public static void main(String[] args) {
    String test = "col1|%col2|%|%";
   
    String[] tokens = test.split("(?=\\|%)|(?<=\\|%)");
    for (String token : tokens) {
      System.out.println(token);
    }
  }
}

---

Ok appreciate it, I will play around with the split function. You can close out this thread.