Package edu.cmu.tetrad.data

Class RegexTokenizer

java.lang.Object

edu.cmu.tetrad.data.RegexTokenizer

public final class RegexTokenizer
extends Object

Tokenizes the given input character sequence using the type of delimiter specified bythe given CPDAG. Meant to function just like StringTokenizer, with more control over what counts as a tokenization delimiter.

Author:

Joseph Ramsey

Constructor Summary

Constructors

Constructor	Description
RegexTokenizer(CharSequence line, Pattern delimiterPattern, char quoteChar)	Constructs a tokenizer for the given input line, using the given Pattern as delimiter.

Method Summary

Modifier and Type	Method	Description
boolean	hasMoreTokens()
String	nextToken()
void	setQuoteSensitive(boolean quoteSensitive)	True iff the parser should be aware of quotation marks and remove them from returned strings.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

RegexTokenizer

public RegexTokenizer(CharSequence line,
 Pattern delimiterPattern,
 char quoteChar)

Constructs a tokenizer for the given input line, using the given Pattern as delimiter.

Method Details

hasMoreTokens

public boolean hasMoreTokens()

Returns:

true iff more tokens exist in the line.

nextToken

public String nextToken()

Returns:

the next token in the line.

setQuoteSensitive

public void setQuoteSensitive(boolean quoteSensitive)

True iff the parser should be aware of quotation marks and remove them from returned strings.