en Copyright 2025 Java http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#9369b4a991666bc911474806b106d8958e79f4c1 SearchIndex: rector, phpcs, type hint fixes List of files: /dokuwiki/inc/Search/Tokenizer.php Wed, 08 Apr 2026 18:30:08 +0000 Andreas Gohr <andi@splitbrain.org> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#1148921de6af6909f19cb5b30b698d0f27d7751e SearchIndex: unify CollectionSearch API and optimize search pipeline- Remove separate lookup() API from CollectionSearch. All searches now use addTerm()/execute() with a single unified pipeline.- Add matches() predicate to Term using efficient string functions (===, str_starts_with, str_ends_with, str_contains) instead of regex.- Add caseInsensitive() support on CollectionSearch and Term for metadata/title searches where indexed values preserve case.- Remove callback support from MetadataSearch::lookupKey() — the only real usage (case-insensitive substring) is replaced by caseInsensitive() + wildcards.- Remove min-length validation from Term. Add Tokenizer::isValidSearchTerm() for callers that need it (FulltextSearch, Indexer::lookup).- Optimize execute() from 4 group passes to 2: scan tokens + resolve frequencies in one pass per group, batch entity name resolution, then populate Terms.- Store full match detail in Term: entity → token → frequency. New accessors getMatches(), getEntityTokens(), getEntityFrequencies() derive different views from this single data structure.- Term no longer used as scratch pad by CollectionSearch. Index-internal data (token IDs, entity IDs) stays local to execute(). Terms receive only final resolved results.- Use title from search results in MetadataSearch::pageLookupCallBack() instead of re-fetching via p_get_first_heading().- Update concept.txt documentation. List of files: /dokuwiki/inc/Search/Tokenizer.php Wed, 08 Apr 2026 16:51:16 +0000 Andreas Gohr <andi@splitbrain.org> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#596d5287d7a816d606ef4153ef9e0f4704bf8f73 Working fulltext collection and searchThis finalizes the FulltextCollection and FulltextCollectionSearchclasses. Proper locking is implemented, tests have been enhanced.It should be possible to reimplement the page full text search on top ofit. List of files: /dokuwiki/inc/Search/Tokenizer.php Thu, 11 May 2023 10:55:00 +0000 Andreas Gohr <andi@splitbrain.org> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#1755450b32c7398d40e13297b3285732f4296005 change Tokenizer static utilityfrequently used in ajax call, singleton is not effective to reduce multiple instantiations. List of files: /dokuwiki/inc/Search/Tokenizer.php Sat, 26 Sep 2020 00:10:51 +0000 Satoshi Sahara <sahara.satoshi@gmail.com> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#15f699ac4db38c7098b4ae4cd0782dff13d46637 replace user errors with exceptionsExceptions are better to handle than errors. What I don't like is thatwe now have an unfortunate mix of return code and exception signallingfor errors. Some methods still return false for errors while othersnow throw exceptions (always returning true otherwise). List of files: /dokuwiki/inc/Search/Tokenizer.php Thu, 10 Sep 2020 15:33:48 +0000 Andreas Gohr <andi@splitbrain.org> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#f2e1d0bfac09ce9b54863873a7d6960d34519d2c fix set Stopwords property List of files: /dokuwiki/inc/Search/Tokenizer.php Sun, 02 Feb 2020 08:56:27 +0000 Satoshi Sahara <sahara.satoshi@gmail.com> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#743c9a28dd4c99f9336a634d173da73f2c7f1e59 rename PagewordIndex to FulltextIndex List of files: /dokuwiki/inc/Search/Tokenizer.php Fri, 31 Jan 2020 15:14:06 +0000 Satoshi Sahara <sahara.satoshi@gmail.com> http://127.0.0.1:8080/history/dokuwiki/inc/Search/Tokenizer.php#094ebf29c8846c56a4fe657a639ccf9ded62d429 separate Tokenizer class List of files: /dokuwiki/inc/Search/Tokenizer.php Mon, 20 Jan 2020 03:54:33 +0000 Satoshi Sahara <sahara.satoshi@gmail.com>