Lines Matching full:token

23     * @var array(Scanner\Token)
28      * Construct a parser object taking the token list to operate on as
43      * Execute the parsing process on the provided token stream
46      * current subsegment of the token stream. It is supposed to return a valid
53      * operate on the token stream. They will throw PhpCssParserExceptions
61      * Try to read any of the $expectedTokens from the token list and return
64      * This method tries to match the current token list against all of the
65      * provided tokens. If a match is found it is removed from the token list
75      * The special Token Scanner\Token::ANY may be used to indicate
77      * token may be specified, which does not make any sense, anyway.
81      * @return Scanner\Token
83     protected function read($expectedTokens): Scanner\Token {
84       // Allow scalar token values for better readability
89       foreach($expectedTokens as $token) {
90         if ($this->matchToken(0, $token)) {
100      * Try to match any of the $expectedTokens against the given token stream
103      * This method tries to match the current token stream at the provided
105      * found it simply returned. The token stream remains unchanged.
114      * The special Token Scanner\Token::ANY may be used to indicate
116      * token may be specified, which does not make any sense, anyway.
119      * arbitrary token stream position. Therefore unlimited lookahead is
126      * @return Scanner\Token|NULL
130     ): ?Scanner\Token {
131       // Allow scalar token values for better readability
136       // If the the requested characters is not available on the token stream
137       // and this state is allowed return a special ANY token
139         return new Scanner\Token(Scanner\Token::ANY, '', 0);
142       foreach($expectedTokens as $token) {
143         if ($this->matchToken($position, $token)) {
153      * Validate if the of the token stream is reached. The position parameter
164      * Try to read any of the $expectedTokens from the token stream and remove them
167      * This method tries to match the current token list against all of the
169      * matching token is found or the token list ends.
175      * The special Token Scanner\Token::ANY is not valid here.
184       // Allow scalar token values for better readability
189       // increase position until the end of the token stream is reached or
190       // a non matching token is found
194         foreach ($expectedTokens as $token) {
195           if ($found = $this->matchToken($position, $token)) {
219      * of providing the current token stream as well as instantiating the
232      * Match a token on the token stream against a token type.
234      * Returns true if the token at the given position exists and the provided
235      * token type matches type of the token at this position, false otherwise.
246       if ($type === Scanner\Token::ANY) {
247         // A token has been found. We do not care which one it was
262       // If the token stream ended unexpectedly throw an appropriate exception
267       // We found a token but none of the expected ones.