864d6c6daac408fa4fc25801adcaf0f744288b30 - fix Lexer so exit-pattern lookbehinds see chars consumed by prior tokens

Tue, 21 Apr 2026 15:28:18 +0000

fix Lexer so exit-pattern lookbehinds see chars consumed by prior tokensLexer::reduce used to hand PCRE a shrinking tail of the subject — eachmatched token was chopped off the front of $raw and the next preg_matchran on what remained. Once a token was consumed, the bytes before thecursor were gone, and any lookbehind assertion in a subsequent patternsilently failed.The bug was latent for DokuWiki's entire history because literal exitpatterns like `\*\*`, ``, or `%%` don't care what's behind them.It surfaced with c3755410a ("require non-whitespace adjacency forinline formatting delimiters"), which added `(?<=[^\s])` to Strong,Emphasis, Underline, Monospace, Subscript, Superscript and Deleted atonce. After that commit, `**[[link]]**` stopped closing — the `]` thatwould satisfy the lookbehind had just been consumed by the link match,so Strong stayed open until end-of-section and swallowed everythingafter it (list items, headings, the lot).Fix:* Lexer::parse and Lexer::reduce track a byte offset into $raw instead of mutating $raw. $initialLength and the shrinking-length arithmetic for absolute match positions are replaced by straight offset increments; the no-progress guard and the trailing-unmatched dispatch both shift to the same cursor.* ParallelRegex::split takes an optional $offset and passes it to preg_match together with PREG_OFFSET_CAPTURE. PCRE scans from the offset forward but still sees the whole subject, so lookbehinds work across already-consumed tokens. The secondary preg_split call used to carve out pre/post is no longer needed — PREG_OFFSET_CAPTURE gives the match start for free, saving one regex operation per reduce() step.Regression tests at all three layers:* ParallelRegexTest — offset plumbing and pre/match accounting.* LexerTest::testIndexLookbehindAcrossConsumedToken — exit-pattern lookbehind targeting the `/>` of a self-closing `` that was consumed as a SPECIAL token on the previous step. Fails under the old Lexer.* FormattingTest — `**[[link]]**` and `**foo//bar//**` round-trip with correct open/close instructions through the full pipeline.Also updates ListsTest::testUnorderedListStrong, whose expectationsdocumented the pre-fix buggy behaviour ("formatting able to spreadacross list items"). With the fix, bold correctly stays within asingle list item; the expected call sequence and the comment areupdated to match. List of files: /dokuwiki/_test/tests/Parsing/Lexer/ParallelRegexTest.php

Changes in ParallelRegexTest.php

864d6c6daac408fa4fc25801adcaf0f744288b30 - fix Lexer so exit-pattern lookbehinds see chars consumed by prior tokens

504c13e8df88563c11b3720b317991bc38835a35 - migrate parser tests to modern namespaced classes