What is StringTokenizer Class in Java?

In this article, we will learn StringTokenizer Class in Java,This free Java tutorial for complete beginners will help you learn Java from scratch.

StringTokenizer Class in Java

Q: How to use StringTokenizer Class in Java in Java?

StringTokenizer class in Java is used to break a string into tokens

String vs StringBuilder vs StringBuffer in Java

StringTokenizer Methods in Java with Examples | Set 2

StringTokenizer class in Java is used to break a string into tokens. A StringTokenizer object internally maintains a current position within the string to be tokenized. Some operations advance this current position past the characters processed.

A token is returned by taking a substring of the string that was used to create the StringTokenizer object. It provides the first step in the parsing process often called lexer or scanner.

Java String Tokenization

The String Tokenizer class allows an application to break strings into tokens. It implements the Enumeration interface. This class is used for parsing data. To use the String Tokenizer class we have to specify an input string and a string that contains delimiters. Delimiters are the characters that separate tokens. Each character in the delimiter string is considered a valid delimiter. Default delimiters are whitespaces, new lines, spaces, and tabs.

Illustration:

Constructors of StringToken

Let us consider ‘str’ as the string to be tokenized. In that case we have three constructor cases as mentioned below:

Constructor	Description
StringTokenizer(String str)	Default delimiters like newline, space, tab, carriage return, and form feed.
StringTokenizer(String str, String delim)	delim is a set of delimiters that are used to tokenize the given string.
StringTokenizer(String str, String delim, boolean flag)	The first two parameters have the same meaning wherein The flag serves the following purpose.

Cases of StringToken Constructors

1. If the flag is false, delimiter characters serve to separate tokens

Example:

Input : if string --> "hello Beginner" and Delimiter is " ", then 
Output:  tokens are "hello" and "Beginner".

2. If the flag is true, delimiter characters are considered to be tokens.

Example:

Input : String --> is "hello Beginner"and Delimiter is " ", then 
Output: Tokens --> "hello", " " and "Beginner".

3. Multiple delimiters can be chosen for a single string.

Example:

Syntax: StringTokenizer st1 = new StringTokenizer( "2+3-1*8/4", "+*-/");

Input : String --> is "2+3-1*8/4" and Delimiters are +,*,-,/ 
Output: Tokens --> "2","3","1","8","4".

Examples of Java String Tokenizer Constructors

Java

// Java Program to implement
// Java String Tokenizer Constructors
import java.util.*;
 
// Driver Class
class GFG {
    // main function
    public static void main(String[] args)
    {
        // Constructor 1
        System.out.println("Using Constructor 1 - ");
 
        // Creating object of class inside main() method
        StringTokenizer st1 = new StringTokenizer(
            "Hello Beginner How are you", " ");
 
        // Condition holds true till there is single token
        // remaining using hasMoreTokens() method
        while (st1.hasMoreTokens())
            // Getting next tokens
            System.out.println(st1.nextToken());
 
        // Constructor 2
        System.out.println("Using Constructor 2 - ");
 
        // Again creating object of class inside main()
        // method
        StringTokenizer st2 = new StringTokenizer(
            "JAVA : Code : String", " :");
 
        // If tokens are present
        while (st2.hasMoreTokens())
 
            // Print all tokens
            System.out.println(st2.nextToken());
 
        // Constructor 3
        System.out.println("Using Constructor 3 - ");
 
        // Again creating object of class inside main()
        // method
        StringTokenizer st3 = new StringTokenizer(
            "JAVA : Code : String", " :", true);
 
        while (st3.hasMoreTokens())
            System.out.println(st3.nextToken());
    }
}

Output

Using Constructor 1 - 
Hello
Beginner
How
are
you
Using Constructor 2 - 
JAVA
Code
String
Using Constructor 3 - 
JAVA
 
:
 
Code
 
:
 
String

Methods Of StringTokenizer Class

Method	Action Performed
countTokens()	Returns the total number of tokens present
hasMoreToken()	Tests if tokens are present for the StringTokenizer’s string
nextElement()	Returns an Object rather than String
hasMoreElements()	Returns the same value as hasMoreToken
nextToken()	Returns the next token from the given StringTokenizer.

Example of Methods of StringTokenizer Class

Below is the implementation of Methods:

Java

// Java Program to implement
//
import java.util.*;
 
// Driver Class
class GFG {
    // main function
    public static void main(String[] args)
    {
        // Creating a StringTokenizer
        StringTokenizer str = new StringTokenizer(
            "Welcome to w3wiki");
 
        StringTokenizer temp = new StringTokenizer("");
 
          // countTokens Method
        int count = str.countTokens();
        System.out.println(count);
       
          // hasMoreTokens Methods
          System.out.println("Welcome to w3wiki: "+str.hasMoreTokens());
          System.out.println("(Empty String) : "+temp.hasMoreTokens());
       
          // nextElement() Method
          System.out.println("\nTraversing the String:");
       
          while(str.hasMoreTokens()){
              System.out.println(str.nextElement());
        }
           
    }
}

Output

3
Welcome to w3wiki: true
(Empty String) : false

Traversing the String:
Welcome
to
w3wiki

Tags:

#Java-Library #Java-Strings #Java #Java #Java-Strings

String vs StringBuilder vs StringBuffer in Java

StringTokenizer Methods in Java with Examples | Set 2