fix(search): restore histogram() as a Collection method with a BC shim

The indexer rework dropped Doku_Indexer::histogram(), which plugins reach
through idx_get_indexer()->histogram() to build tag c

fix(search): restore histogram() as a Collection method with a BC shim

The indexer rework dropped Doku_Indexer::histogram(), which plugins reach
through idx_get_indexer()->histogram() to build tag clouds (e.g. tagfilter
via the 'subject' metadata index). Calling it fatalled through
LegacyIndexer::__call as an undefined method.

Add histogram() to AbstractCollection: it sums each token's frequency across
all entities and returns them ordered by frequency, filtered by min/max/minlen,
handling both length-split (fulltext) and single-group (metadata) collections.
DirectCollection overrides it for the 1:1 title case (count shared values).
LegacyIndexer::histogram() is a deprecated shim dispatching by $key to the
matching collection.

show more ...

fix(search): don't let collections release a caller-owned index lock

Indexer::addPage() locks the page index once and hands the same instance to
several collections, each of which lock() and unlock(

fix(search): don't let collections release a caller-owned index lock

Indexer::addPage() locks the page index once and hands the same instance to
several collections, each of which lock() and unlock() it around their work.

AbstractCollection::lock() locked and tracked every index for later release.
When a shared AbstractIndex was already locked by the caller, locking it again
was a no-op but the collection still tracked it, so the collection's unlock()
released the caller's lock and flipped the shared index read-only. Each later
collection then re-acquired and re-released it, leaving windows in which another
process could grab the page lock mid-operation; contention could abort addPage()
with the title index updated but the fulltext index stale.

Treat a shared index that is already writable as owned by its creator: leave its
lock untouched and only manage the locks the collection acquires itself.

Add a regression test asserting a caller-held index lock survives repeated
collection lock()/unlock() cycles.

show more ...

fix(search): read split index suffixes correctly

AbstractIndex::max() used /(\d)+\.idx$/ to determine the highest split
index suffix. For filenames like w10.idx or w12.idx that regex captured
only t

fix(search): read split index suffixes correctly

AbstractIndex::max() used /(\d)+\.idx$/ to determine the highest split
index suffix. For filenames like w10.idx or w12.idx that regex captured
only the last digit, so the maximum token-length group was truncated to 9.
Wildcard searches therefore skipped all fulltext index shards for words
with 10 or more characters.

Tighten the match to the basename and require the current index name
followed immediately by digits: ^<idx>(\d+)\.idx$. This captures the full
numeric suffix and also avoids counting unrelated index families that
share the same prefix, such as treating wiki2.idx as a numbered shard of
the w index.

Add regressions for both behaviors: multi-digit suffixes are parsed
correctly, and same-prefix indexes are ignored when determining the
maximum shard number.

show more ...

SearchIndex: remove write side effect from retrieveRow()

retrieveRow() padded the index file when the requested RID was beyond
the current length. This was an optimization for subsequent changeRow()

SearchIndex: remove write side effect from retrieveRow()

retrieveRow() padded the index file when the requested RID was beyond
the current length. This was an optimization for subsequent changeRow()
calls, but changeRow() already handles padding on its own. The side
effect was also inconsistent with retrieveRows() which is a pure read.

Added a cross-index integration test verifying RID consistency across
entity, token, frequency and reverse indexes when multiple entities
share tokens.

show more ...

SearchIndex: unify CollectionSearch API and optimize search pipeline

- Remove separate lookup() API from CollectionSearch. All searches now
use addTerm()/execute() with a single unified pipeline.

SearchIndex: unify CollectionSearch API and optimize search pipeline

- Remove separate lookup() API from CollectionSearch. All searches now
use addTerm()/execute() with a single unified pipeline.
- Add matches() predicate to Term using efficient string functions
(===, str_starts_with, str_ends_with, str_contains) instead of regex.
- Add caseInsensitive() support on CollectionSearch and Term for
metadata/title searches where indexed values preserve case.
- Remove callback support from MetadataSearch::lookupKey() — the only
real usage (case-insensitive substring) is replaced by
caseInsensitive() + wildcards.
- Remove min-length validation from Term. Add Tokenizer::isValidSearchTerm()
for callers that need it (FulltextSearch, Indexer::lookup).
- Optimize execute() from 4 group passes to 2: scan tokens + resolve
frequencies in one pass per group, batch entity name resolution, then
populate Terms.
- Store full match detail in Term: entity → token → frequency. New
accessors getMatches(), getEntityTokens(), getEntityFrequencies()
derive different views from this single data structure.
- Term no longer used as scratch pad by CollectionSearch. Index-internal
data (token IDs, entity IDs) stays local to execute(). Terms receive
only final resolved results.
- Use title from search results in MetadataSearch::pageLookupCallBack()
instead of re-fetching via p_get_first_heading().
- Update concept.txt documentation.

show more ...

SearchIndex: add integrity checking to Collection architecture

Add checkIntegrity() to AbstractCollection and DirectCollection that
verifies paired indexes have matching line counts (token==frequenc

SearchIndex: add integrity checking to Collection architecture

Add checkIntegrity() to AbstractCollection and DirectCollection that
verifies paired indexes have matching line counts (token==frequency,
entity==reverse, entity==token for direct collections). Throws
IndexIntegrityException on the first inconsistency found.

Add Countable interface to AbstractIndex with count() implementations
in MemoryIndex and FileIndex. Add Indexer::checkIntegrity() and
Indexer::isIndexEmpty() to orchestrate checks across all collections.

Update infoutils.php to use the new Indexer API instead of the old
FulltextIndex/MetadataIndex classes.

Fix range(1, 0) bug in three places that produced [1, 0] instead of
an empty array when split-by-length indexes were empty.

show more ...

SearchIndex: rewrite MetadataSearch to use Collection classes

Replace MetadataIndex usage in MetadataSearch with the new Collection/Index
architecture. This completes the read-path migration so data

SearchIndex: rewrite MetadataSearch to use Collection classes

Replace MetadataIndex usage in MetadataSearch with the new Collection/Index
architecture. This completes the read-path migration so data written by the
Collection-based Indexer is read back correctly using TupleOps tuple format.

Generalize FrequencyCollectionSearch into CollectionSearch that works with any
AbstractCollection type (Frequency, Lookup, Direct) and handles both
split-by-length and non-split index layouts transparently. DirectCollection
participates via resolveTokenFrequencies() which maps token RID = entity RID.

Key changes:
- AbstractCollection gains isSplitByLength(), resolveTokenFrequencies(),
getEntitiesWithData(), and groupToSuffix() with validation
- Index groups are now int (0 = non-split, positive = token length)
- CollectionSearch provides both addTerm()/execute() for fulltext and
lookup() for metadata-style search (exact/wildcard/callback)
- MetadataSearch delegates entirely to collection APIs
- Shared filterPages() replaces duplicated page filtering logic
- All callers updated from MetadataIndex to MetadataSearch
- Tests moved to Search namespace with full coverage for new APIs

show more ...

SearchIndex: reorganize and expand test suite

Move all Search tests from _test/tests/inc/Search/ to _test/tests/Search/
to match the dokuwiki\test autoloader convention. Fix namespaces from
tests\*

SearchIndex: reorganize and expand test suite

Move all Search tests from _test/tests/inc/Search/ to _test/tests/Search/
to match the dokuwiki\test autoloader convention. Fix namespaces from
tests\* to dokuwiki\test\* so all tests work in isolation.

Extract inline test helpers into separate autoloadable mock files:
TestDirectCollection → MockDirectCollection, TestLookupCollection →
MockLookupCollection, TestFrequencyCollection → MockFrequencyCollection.

Rename AbstractIndexTest → AbstractIndexTestCase to fix PHPUnit warning
about abstract classes with Test suffix.

Replace dead xxxRealWord() with proper testWildcardSearch() verifying
exact token matches and frequencies for all three wildcard types.
Add testTokenizedPageSearch() using a dedicated test data file. Add
testNoMatchReturnsEmptyFrequencies() which exposed a bug in Term where
uninitialized $tokens/$frequencies caused crashes on zero-match terms.

Replace fulltext_query.test.php with modern QueryParserTest in the
Search\Query namespace.

Add new test files:
- LockTest: acquire/release, reference counting, stale lock override,
foreign lock rejection, releaseAll, independent locks
- NamespacePredicateTest: filter/exclude, sub-namespaces, partial prefix
safety, empty sets, score preservation
- PageSetTest: intersect, unite, subtract, isEmpty
- QueryEvaluatorTest: word lookups, AND/OR/NOT, namespace filtering,
combined queries, partial namespace prefix safety

Fix Term.php: initialize $tokens and $frequencies to [] instead of null.

show more ...