There's no documentation for this item.
| Classes |
| Default |
A default tokenizer for handling syntaxes that are not explicitly handled
elsewhere. It simply yields the given text as a single token.
|
| Ruby |
A tokenizer for the Ruby language. It recognizes all common syntax (and
some less common syntax) but because it is not a true lexer, it will make
mistakes on some ambiguous cases.
|
| Token |
A single token extracted by a tokenizer. It is simply the lexeme itself,
decorated with a ‘group’ attribute to identify the type of the
lexeme.
|
| Tokenizer |
The base class of all tokenizers. It sets up the scanner and manages the
looping until all tokens have been extracted. It also provides convenience
methods to make sure adjacent tokens of identical groups are returned as a
single token.
|
| XML |
A simple implementation of an XML lexer. It handles most cases. It is not a
validating lexer, meaning it will happily process invalid XML without
complaining.
|
| YAML |
A simple implementation of an YAML lexer. It handles most cases. It is not
a validating lexer.
|
| Constants |
| SYNTAX |
A hash for registering syntax implementations.
|
| Public Methods |
| all |
Return an array of the names of supported syntaxes.
|
| load |
Load the implementation of the requested syntax. If the syntax cannot be
found, or if it cannot be loaded for whatever reason, the Default syntax
handler will be returned.
|
<code/>and<pre/>for code samples.