Searched refs:Tokenizer (Results 1 – 9 of 9) sorted by relevance
6 use dokuwiki\Search\Tokenizer; alias53 $this->length = Tokenizer::tokenLength($this->base);
15 use dokuwiki\Search\Tokenizer; alias370 $group = $this->splitByLength ? Tokenizer::tokenLength($token) : 0;
18 class Tokenizer class
179 $words = Tokenizer::getWords($data['body']);516 if (!Tokenizer::isValidSearchTerm($token)) continue;
100 if (!Tokenizer::isValidSearchTerm($word)) continue;
203 ==== Tokenizer ====205 The ''Tokenizer'' class (in ''\dokuwiki\Search'') is responsible for splitting text into indexable …207 ''Tokenizer::getWords($text, $wc)'' splits the given text into an array of lowercase tokens. Tokens…236 …ere indexed values preserve case (the fulltext token index is already lowercased by the Tokenizer).246 … callers should filter short words before calling ''addTerm()'' using ''Tokenizer::isValidSearchTe…
5 use dokuwiki\Search\Tokenizer; alias38 $words = Tokenizer::getWords($term_noparen, true);
7 use dokuwiki\Search\Tokenizer; alias82 $tokens = Tokenizer::getWords($text);
6 use dokuwiki\Search\Tokenizer; alias