Lines Matching defs:tokens

21     * List of tokens from scanner
31     public function __construct(array &$tokens) {
32       $this->_tokens = &$tokens;
36      * Return parser tokens list
65      * provided tokens. If a match is found it is removed from the token list
71      * The $expectedTokens parameter may be an array of tokens or a scalar
95       // None of the given tokens matched
104      * lookahead position against all of the provided tokens. If a match is
110      * The $expectedTokens parameter may be an array of tokens or a scalar
148       // None of the given tokens matched
168      * provided tokens. Matching tokens are removed from the list until a non
171      * The $expectedTokens parameter may be an array of tokens or a scalar
177      * The method return TRUE if tokens were removed, otherwise FALSE.
205       // remove the tokens from the stream
255      * Handle the case if none of the expected tokens could be found.