stream-tokenizer.H File Reference

Provides the StreamTokenizer class. More...

#include <iostream>
#include <set>
#include <sstream>
#include <stdexcept>
#include <string>
#include <string.h>
#include <vector>

Go to the source code of this file.

Classes
class	reranker::StreamTokenizer
	A simple class for tokenizing a stream of tokens for the formally specified language used to construct objects for the Reranker framework. More...
struct	reranker::StreamTokenizer::Token
	Information about a token read from the underlying stream. More...

Namespaces
	reranker
	Provides reranking models for discriminative modeling, with some special handling for discriminative language models.

Macros
#define	DEFAULT_RESERVED_CHARS   "(){},=;/"
	Default set of reserved characters for the StreamTokenizer class. More...

Detailed Description

Provides the StreamTokenizer class.

Author

dbike.nosp@m.l@go.nosp@m.ogle..nosp@m.com (Dan Bikel)

Definition in file stream-tokenizer.H.

Macro Definition Documentation

#define DEFAULT_RESERVED_CHARS   "(){},=;/"

Default set of reserved characters for the StreamTokenizer class.

Definition at line 80 of file stream-tokenizer.H.