| #
1c00c021
|
| 09-Jul-2026 |
Andreas Gohr <gohr@cosmocode.de> |
fix(parser): validate inline formatting closers with a single memoized scan
The inline formatting modes only open a span when a valid closer exists ahead. That check was a lookahead built on CONTENT
fix(parser): validate inline formatting closers with a single memoized scan
The inline formatting modes only open a span when a valid closer exists ahead. That check was a lookahead built on CONTENT_UNTIL_PARA, tested character by character up to the next paragraph break and re-evaluated from scratch for every opener candidate — openers times paragraph length. With pcre.jit=0 a crafted 32KB page took 16s and an ordinary 34KB page with long paragraphs 37s; with the JIT on (the PHP default) the per-character lookahead exhausted the JIT stack, the match silently failed, and the formatting — or everything after it — rendered as plain text.
The check also decided the wrong thing. It scanned raw text, so a closer lookalike inside content the lexer consumes atomically — a nowiki or %% span, a backtick code span, a link, a URL — counted as a real closer even though the mode's exit pattern can never fire there. And it ignored the enclosing span: an inner delimiter whose only closer lay past the closer of the mode it sits in was entered anyway, so a stray delimiter paired with one in a following sibling span and dragged the boundary along — the `*` in ''glob/*.conf'' joined the `*` of the next ''...'' span and corrupted the paragraph; the same held for //, ** and __ inside monospace, and an emphasis opened inside ((...)) ran past the footnote's )) and the enclosing bold's **.
Each formatting mode now declares its closer through Lexer::addCloserPattern(), mirroring addExitPattern(), and the lexer answers "does a valid closer exist ahead" with one anchored possessive scan per range instead of a lookahead per opener:
- The scan runs left to right from the opener, hopping over opaque spans derived from already-registered patterns — a plain or special match is consumed in one step, an entry into a verbatim mode (nowiki, the backtick code spans) extends to that mode's first exit — so a closer lookalike inside consumed content is never mistaken for a closer. Each hop finds the earliest of boundary, closer, or opaque span in a single leftmost search, keeping the check linear. - An opener is rejected when the nearest enclosing mode that has a closer of its own would close before the opener's own closer, so a delimiter that can never close within its span stays literal. That ancestor is found by walking the mode stack past modes that declare no closer (plugins, footnotes); the nearest guarded ancestor suffices, as it was itself validated against its own when it opened. - Both verdicts are memoized and reset per parse() run: a proven closer validates every earlier candidate, and a proven closer-free range rejects every later candidate before the next boundary. With the lexer consuming each opened span, the whole parse is linear in document size.
Closer patterns match the closing delimiter itself with flanking context in lookarounds — the convention exit patterns already follow — so closer positions compare exactly across modes and a closer directly after an inner opener is seen. AbstractFormatting derives the closer from the exit pattern and registers it with the paragraph break as the boundary, preserving the rule that formatting never spans paragraphs; a mode with other needs can pass a different boundary or none.
Footnote declares its )) as a closer rather than guarding its (( entry with a (?=.*)) lookahead, so the footnote becomes a boundary the scan sees and formatting inside it no longer pairs across the )); its closer takes no paragraph boundary, as footnotes are block-level. GfmEmphasis gains a closer pattern so single * emphasis is validated the same way, while its entry lookahead still enforces CommonMark nearest-delimiter pairing. GfmEmphasis and GfmStrong span bodies cannot contain their delimiter, so their in-pattern lookaheads stay linear on their own; the GFM backtick span bodies get deterministic alternatives with possessive quantifiers, removing their per-character backtracking.
CONTENT_UNTIL_PARA is removed: any entry pattern built with it recreates the quadratic scan. ParallelRegex gains escapePattern() so embedded closer fragments follow the lexer's bare-parenthesis convention, reports PREG_JIT_STACKLIMIT_ERROR so a future JIT exhaustion surfaces instead of silently truncating, and no longer rewrites its registered patterns in place while compiling the compound regex.
The adversarial 32KB page drops to 0.1s, the 37-second benign page to 0.1s, and a 128KB variant stays under 0.6s.
show more ...
|
| #
9a38b8db
|
| 05-Jul-2026 |
Andreas Gohr <andi@splitbrain.org> |
fix(parser): keep a mode's directly-assigned allowedModes when it also declares categories
setModeRegistry() replaced $this->allowedModes with the category-derived list whenever a mode declared cate
fix(parser): keep a mode's directly-assigned allowedModes when it also declares categories
setModeRegistry() replaced $this->allowedModes with the category-derived list whenever a mode declared categories, discarding any mode names the subclass had assigned to the property directly. The old SyntaxPlugin::accepts() merged the category modes into the existing list instead, so a mode using the sibling-component pattern - a non-empty getAllowedTypes() plus $this->allowedModes[] = 'plugin_foo_bar' in its constructor - kept both and its sibling syntax nested as intended.
Merge the category-derived modes into the directly-assigned list rather than replacing it, then deduplicate. The no-categories path is unchanged: the directly-assigned list is used as-is.
show more ...
|
| #
47a02a10
|
| 04-Jun-2026 |
Andreas Gohr <gohr@cosmocode.de> |
Parsing: make parse syntax a per-parse value, drop ModeInterface
The active parse's syntax flavour is a per-parse question, not process- global state: within a single request a plugin can render bun
Parsing: make parse syntax a per-parse value, drop ModeInterface
The active parse's syntax flavour is a per-parse question, not process- global state: within a single request a plugin can render bundled DokuWiki-syntax text inside an otherwise-Markdown page. Yet ModeRegistry was a singleton that read $conf['syntax'] and the $PARSER_MODES global, and every mode reached it through ModeRegistry::getInstance() — so the flavour lived in shared mutable state that two parses in one request would fight over.
Make the registry a short-lived value instead:
- ModeRegistry is constructed once per parse with an explicit $syntax and injected into Parser, Handler and every mode. getSyntax() / isDwPreferred() / isMdPreferred() consult $this->syntax; the DOKU_UNITTEST-gated mode-list cache hack is gone (each registry is fresh, nothing to invalidate). - p_get_instructions() is now the single place in the pipeline where $conf['syntax'] is read; from there the flavour travels as a parameter. No code under inc/Parsing/ reads $conf['syntax'] directly anymore — the five syntax-reading modes (Preformatted, GfmHr, GfmEscape, Externallink, GfmQuote) route through $this->registry.
Keep the two concepts apart, as documented in the ModeRegistry and AbstractMode docblocks: the user's configured *preference* stays in $conf['syntax'] for UI code (toolbar, settings), while the active parse's syntax is a parameter carried by the registry.
$PARSER_MODES is demoted to a deprecated, read-only mirror, published during loadPluginModes() — third-party syntax plugins (columnlist, alphalist2, phpwikify, skipentity) and the bundled info plugin read the global directly, often from their constructors, so the taxonomy must stay visible there. No core code reads the mirror.
Fold ModeInterface into AbstractMode while here: getSort()/handle() are abstract, the connect callbacks carry defaults, and the public $Lexer "FIXME should be done by setter" becomes setLexer()/getLexer() injected by Parser::addMode() alongside the registry. Nested-content resolution moves to the allowedCategories()/filterAllowedModes() hooks, resolved once when the registry is attached.
Tests build their own parser/registry through ParserTestBase::setSyntax() instead of mutating $conf and calling the removed ModeRegistry::reset().
show more ...
|
| #
6b33ca93
|
| 20-Apr-2026 |
Andreas Gohr <gohr@cosmocode.de> |
add regex-primitive constants and getInstructionName() hook
Preparatory refactor for the upcoming GFM parser modes. No behaviour change for any existing mode: CONTENT_UNTIL_PARA still evaluates to t
add regex-primitive constants and getInstructionName() hook
Preparatory refactor for the upcoming GFM parser modes. No behaviour change for any existing mode: CONTENT_UNTIL_PARA still evaluates to the same regex (now factored through NOT_AT_PARA_BREAK), and getInstructionName() defaults to getModeName() so all current AbstractFormatting subclasses emit the same handler instructions as before.
AbstractMode gains four new shared regex constants:
NOT_AT_PARA_BREAK — zero-width assertion: current position is not the start of a paragraph break (blank line). Extracted from CONTENT_UNTIL_PARA for reuse in patterns that need a custom body char class.
NON_WORD_CHAR — char class: ASCII whitespace or ASCII punctuation except `_`. Multibyte-safe by construction: UTF-8 continuation bytes are >= 0x80 and thus fall outside every ASCII class, so checking positively that the surrounding context IS a non-word char correctly treats multibyte letters as word-like. No `u` flag required.
NO_WORD_BEFORE — zero-width: preceded by NON_WORD_CHAR or at start-of-input/line. For intraword-aware openers.
NO_WORD_AFTER — zero-width: followed by NON_WORD_CHAR or at end-of-input. Complement of NO_WORD_BEFORE.
AbstractFormatting gains a getInstructionName() hook that defaults to getModeName(). Subclasses that want to emit handler instructions under a different name than their lexer mode name (so a Gfm mode can share DW's `emphasis_open`/`strong_open` instructions while registering its own lexer state) override this method.
show more ...
|
| #
10fb3d65
|
| 20-Apr-2026 |
Andreas Gohr <gohr@cosmocode.de> |
prevent inline formatting from matching across paragraph boundaries
The Lexer compiles all patterns with the `s` (DOTALL) flag via ParallelRegex::getPerlMatchingFlags(), which makes `.` match newlin
prevent inline formatting from matching across paragraph boundaries
The Lexer compiles all patterns with the `s` (DOTALL) flag via ParallelRegex::getPerlMatchingFlags(), which makes `.` match newlines. Inline formatting modes use lookaheads like `\*\*(?=.*\*\*)` to verify a closing delimiter exists, so with DOTALL a lone `**` happily matched its "closer" many paragraphs later, swallowing blank lines into a single <strong> run.
Add CONTENT_UNTIL_PARA on AbstractMode — a regex snippet matching any character unless it would start a paragraph break (blank line, possibly with horizontal whitespace). Update all inline formatting entry patterns (Strong, Emphasis, Underline, Monospace, Subscript, Superscript, Deleted) to use it in their closing-delimiter lookaheads.
Emphasis also gets a real closing-`//` check; its previous lookahead just verified "content exists with a non-colon char" without requiring the closing delimiter at all.
Single newlines inside a delimiter pair still match (multi-line formatting); only blank lines end it.
BREAKING: This means you no longer can mark multiple paragraphs as bold or strike them out. On the other hand it prevents accidentally breaking the page layout by missing a closing delimiter (as reported many many times over the years) eg. #1025 #3588 #1056
show more ...
|