Class RegexTokenizer

java.lang.Object
edu.cmu.tetrad.data.RegexTokenizer

public final class RegexTokenizer extends Object
Tokenizes the given input character sequence using the type of delimiter specified bythe given CPDAG. Meant to function just like StringTokenizer, with more control over what counts as a tokenization delimiter.
Author:
josephramsey
  • Constructor Details

    • RegexTokenizer

      public RegexTokenizer(CharSequence line, Pattern delimiterPattern, char quoteChar)
      Constructs a tokenizer for the given input line, using the given Pattern as delimiter.
  • Method Details

    • hasMoreTokens

      public boolean hasMoreTokens()
      Returns:
      true iff more tokens exist in the line.
    • nextToken

      public String nextToken()
      Returns:
      the next token in the line.
    • setQuoteSensitive

      public void setQuoteSensitive(boolean quoteSensitive)
      True iff the parser should be aware of quotation marks and remove them from returned strings.