Overview | Package | Class | Use | Tree | Deprecated | Index | Help
| Qizx/open 4.1 API
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## com.qizx.api.util.text
Class LexicalTokenizer

    java.lang.Object
      com.qizx.xquery.impl.Lexer
          com.qizx.api.util.text.LexicalTokenizer

    public class LexicalTokenizer

extends com.qizx.xquery.impl.Lexer

Iterator on tokens of a XQuery script. Used for implementing syntax coloring in Qizx Studio.

Returns the lexical group and the position and size of each token.

Field Summary
`static int` | `LC_COMMENT` Lexical category of XQuery comments.
`static int` | `LC_FUNCTION` Lexical category of XQuery function calls.
`static int` | `LC_KEYWORD` Lexical category of reserved keywords.
`static int` | `LC_MISC` Lexical category of others tokens.
`static int` | `LC_NAME` Lexical category of identifiers.
`static int` | `LC_NUMBER` Lexical category of numeric literals.
`static int` | `LC_PRAGMA` Lexical category of XQuery pragmas.
`static int` | `LC_SPACE` Lexical category of whitespace.
`static int` | `LC_STRING` Lexical category of string literals.
`static int` | `LC_TAG` Lexical category of element constructors.

Fields inherited from class com.qizx.xquery.impl.Lexer
`debug, ERR_SYNTAX, T_END`

Constructor Summary
`LexicalTokenizer(String script)` Constructs a tokenizer of a XQuery script.

Method Summary
`String` | `getSpace()` Gets the space that is before the current token.
`int` | `getSpaceLength()` Size of leading space before the current token.
`int` | `getTokenLength()` Returns the length of the current token.
`int` | `getTokenStart()` Returns the start position of the current token.
`String` | `getTokenValue()` Returns the text of the current token: null if the end is reached.
`int` | `nextToken()` Moves to next token and returns its category.

Methods inherited from class com.qizx.xquery.impl.Lexer
`match, startLexer`

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### LC_TAG

    public static final int LC_TAG

  Lexical category of element constructors.

  See Also:
    Constant Field Values

### LC_SPACE

    public static final int LC_SPACE

  Lexical category of whitespace.

  See Also:
    Constant Field Values

### LC_NUMBER

    public static final int LC_NUMBER

  Lexical category of numeric literals.

  See Also:
    Constant Field Values

### LC_STRING

    public static final int LC_STRING

  Lexical category of string literals.

  See Also:
    Constant Field Values

### LC_MISC

    public static final int LC_MISC

  Lexical category of others tokens.

  See Also:
    Constant Field Values

### LC_NAME

    public static final int LC_NAME

  Lexical category of identifiers.

  See Also:
    Constant Field Values

### LC_KEYWORD

    public static final int LC_KEYWORD

  Lexical category of reserved keywords.

  See Also:
    Constant Field Values

### LC_COMMENT

    public static final int LC_COMMENT

  Lexical category of XQuery comments.

  See Also:
    Constant Field Values

### LC_PRAGMA

    public static final int LC_PRAGMA

  Lexical category of XQuery pragmas.

  See Also:
    Constant Field Values

### LC_FUNCTION

    public static final int LC_FUNCTION

  Lexical category of XQuery function calls.

  See Also:
    Constant Field Values

Constructor Detail

### LexicalTokenizer

    public LexicalTokenizer(String script)

  Constructs a tokenizer of a XQuery script.

  Parameters:
    `script` - source XQuery script to tokenize

Method Detail

### nextToken

    public int nextToken()
                  throws CompilationException

  Moves to next token and returns its category.

  Overrides:
    `nextToken` in class `com.qizx.xquery.impl.Lexer`
  Returns:
    token code (LC_xxx)
  Throws:
    `CompilationException` - if parsing error

### getTokenValue

    public String getTokenValue()

  Returns the text of the current token: null if the end is reached.

### getTokenStart

    public int getTokenStart()

  Returns the start position of the current token.

### getTokenLength

    public int getTokenLength()

  Returns the length of the current token.

### getSpaceLength

    public int getSpaceLength()

  Size of leading space before the current token.

### getSpace

    public String getSpace()

  Gets the space that is before the current token. Returns null if no space.

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
| © 2010 Axyana Software
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD