Class Tokenizer


  • public final class Tokenizer
    extends Object
    String tokenizer for parsing headers.
    • Constructor Detail

      • Tokenizer

        public Tokenizer​(String input)
        Create a new instance.
        Parameters:
        input - string to parse
    • Method Detail

      • consumeTokenIfPresent

        public String consumeTokenIfPresent​(CharMatcher matcher)
        Get the token represented by the specified matcher and advance the position the to next character if matched.
        Parameters:
        matcher - matcher to use
        Returns:
        token matched, or null if not matched
        Throws:
        IllegalStateException - if hasMore() returns false
      • consumeToken

        public String consumeToken​(CharMatcher matcher)
        Get the token represented by the specified matcher and advance the position the to next character.
        Parameters:
        matcher - matcher to use
        Returns:
        IllegalStateException if hasMore() returns false or if the matcher wasn't matched.
      • consumeCharacter

        public char consumeCharacter​(CharMatcher matcher)
        Get the one character at the current position and matches it with the specified matcher, then update the position to the next character.
        Parameters:
        matcher - matcher to use
        Returns:
        consumed character
        Throws:
        IllegalStateException - if hasMore() returns false or if the specified matcher does not match the character at the current position
      • consumeCharacter

        public char consumeCharacter​(char c)
        Get the one character at the current position and matches it with the specified character and update the position to the next character.
        Parameters:
        c - character to match
        Returns:
        matched character
        Throws:
        IllegalStateException - if hasMore() returns false or if the specified character does not match the character at the current position
      • previewChar

        public char previewChar()
        Get the character at the current position.
        Returns:
        char
        Throws:
        IllegalStateException - if hasMore() returns false
      • hasMore

        public boolean hasMore()
        Test if there are more characters to process.
        Returns:
        true if there are more characters to process, false otherwise
      • normalize

        public static String normalize​(CharMatcher matcher,
                                       String token)
        Verify that the given token matches the specified matcher and return a lower case only token string.
        Parameters:
        matcher - matcher to use
        token - input token
        Returns:
        normalized token string (lower case only)