| #
3a56ec9b
|
| 08-Jul-2026 |
Andreas Gohr <gohr@cosmocode.de> |
fix(parser): stop unclosed GFM fenced code and emphasis from exhausting memory
Both modes repeat a group over their body, and on the non-JIT PCRE engine each iteration of a group repetition keeps it
fix(parser): stop unclosed GFM fenced code and emphasis from exhausting memory
Both modes repeat a group over their body, and on the non-JIT PCRE engine each iteration of a group repetition keeps its own backtracking frame. On unclosed input the body runs to end of input (or a paragraph break), the required closer fails, and the engine unwinds frame by frame: a linear, unbounded memory spike that can fatal a request once the body grows past the memory limit (and, with JIT on, trips pcre.jit_stacklimit so the lexer dumps the rest of the document as literal text).
gfm_code/gfm_file: the fence body (?:(?!CLOSE).)* already stops at the first valid closer, so no backtracking into it is ever legitimate. Make it possessive so the no-closer case fails immediately.
gfm_emphasis: its closer lookahead ended with a character the body had to backtrack to reach (the [^\s*] before the closing *). Verify that flanking-closer rule with a lookbehind instead, so the [^*] body — which cannot match the * it stops at — can be possessive too.
Matching is unchanged for well-formed input. The fenced-code patterns were introduced in b1c59bed2.
show more ...
|
| #
1c00c021
|
| 09-Jul-2026 |
Andreas Gohr <gohr@cosmocode.de> |
fix(parser): validate inline formatting closers with a single memoized scan
The inline formatting modes only open a span when a valid closer exists ahead. That check was a lookahead built on CONTENT
fix(parser): validate inline formatting closers with a single memoized scan
The inline formatting modes only open a span when a valid closer exists ahead. That check was a lookahead built on CONTENT_UNTIL_PARA, tested character by character up to the next paragraph break and re-evaluated from scratch for every opener candidate — openers times paragraph length. With pcre.jit=0 a crafted 32KB page took 16s and an ordinary 34KB page with long paragraphs 37s; with the JIT on (the PHP default) the per-character lookahead exhausted the JIT stack, the match silently failed, and the formatting — or everything after it — rendered as plain text.
The check also decided the wrong thing. It scanned raw text, so a closer lookalike inside content the lexer consumes atomically — a nowiki or %% span, a backtick code span, a link, a URL — counted as a real closer even though the mode's exit pattern can never fire there. And it ignored the enclosing span: an inner delimiter whose only closer lay past the closer of the mode it sits in was entered anyway, so a stray delimiter paired with one in a following sibling span and dragged the boundary along — the `*` in ''glob/*.conf'' joined the `*` of the next ''...'' span and corrupted the paragraph; the same held for //, ** and __ inside monospace, and an emphasis opened inside ((...)) ran past the footnote's )) and the enclosing bold's **.
Each formatting mode now declares its closer through Lexer::addCloserPattern(), mirroring addExitPattern(), and the lexer answers "does a valid closer exist ahead" with one anchored possessive scan per range instead of a lookahead per opener:
- The scan runs left to right from the opener, hopping over opaque spans derived from already-registered patterns — a plain or special match is consumed in one step, an entry into a verbatim mode (nowiki, the backtick code spans) extends to that mode's first exit — so a closer lookalike inside consumed content is never mistaken for a closer. Each hop finds the earliest of boundary, closer, or opaque span in a single leftmost search, keeping the check linear. - An opener is rejected when the nearest enclosing mode that has a closer of its own would close before the opener's own closer, so a delimiter that can never close within its span stays literal. That ancestor is found by walking the mode stack past modes that declare no closer (plugins, footnotes); the nearest guarded ancestor suffices, as it was itself validated against its own when it opened. - Both verdicts are memoized and reset per parse() run: a proven closer validates every earlier candidate, and a proven closer-free range rejects every later candidate before the next boundary. With the lexer consuming each opened span, the whole parse is linear in document size.
Closer patterns match the closing delimiter itself with flanking context in lookarounds — the convention exit patterns already follow — so closer positions compare exactly across modes and a closer directly after an inner opener is seen. AbstractFormatting derives the closer from the exit pattern and registers it with the paragraph break as the boundary, preserving the rule that formatting never spans paragraphs; a mode with other needs can pass a different boundary or none.
Footnote declares its )) as a closer rather than guarding its (( entry with a (?=.*)) lookahead, so the footnote becomes a boundary the scan sees and formatting inside it no longer pairs across the )); its closer takes no paragraph boundary, as footnotes are block-level. GfmEmphasis gains a closer pattern so single * emphasis is validated the same way, while its entry lookahead still enforces CommonMark nearest-delimiter pairing. GfmEmphasis and GfmStrong span bodies cannot contain their delimiter, so their in-pattern lookaheads stay linear on their own; the GFM backtick span bodies get deterministic alternatives with possessive quantifiers, removing their per-character backtracking.
CONTENT_UNTIL_PARA is removed: any entry pattern built with it recreates the quadratic scan. ParallelRegex gains escapePattern() so embedded closer fragments follow the lexer's bare-parenthesis convention, reports PREG_JIT_STACKLIMIT_ERROR so a future JIT exhaustion surfaces instead of silently truncating, and no longer rewrites its registered patterns in place while compiling the compound regex.
The adversarial 32KB page drops to 0.1s, the 37-second benign page to 0.1s, and a 128KB variant stays under 0.6s.
show more ...
|
| #
bcefb8ae
|
| 20-Apr-2026 |
Andreas Gohr <gohr@cosmocode.de> |
add GFM emphasis and underscore-delimited strong modes
Three new inline formatting modes for GitHub Flavored Markdown:
GfmEmphasis `*text*` → <em> GfmEmphasisUnderscore `_text_`
add GFM emphasis and underscore-delimited strong modes
Three new inline formatting modes for GitHub Flavored Markdown:
GfmEmphasis `*text*` → <em> GfmEmphasisUnderscore `_text_` → <em> (MD-preferred only) GfmStrongUnderscore `__text__` → <strong> (MD-preferred only)
All three emit the same handler instructions as DokuWiki's Emphasis / Strong, so existing renderers need no changes.
Design notes:
* Lexer mode names use snake_case (gfm_emphasis, gfm_emphasis_underscore, gfm_strong_underscore) to keep PascalCase readable at the class level. The asterisk variant emits `emphasis_open`/`emphasis_close` via the getInstructionName() hook, so DW's Emphasis (`//...//`) and GfmEmphasis (`*...*`) can coexist in mixed modes without a lexer state collision while still producing the same <em> output.
* Underscore variants gate on Markdown-preferred syntax (`markdown`, `md+dw`) because `__` otherwise means DW underline. GfmStrongUnderscore sorts at 70 (matching Strong) — below Underline at 90 — so when loaded it wins the lexer race for `__` runs. Underline is already gated out of MD-preferred modes in the previous commit.
* Entry patterns enforce the simplified CommonMark flanking rules already shared across DW inline modes (non-whitespace adjacency, no paragraph-boundary crossing) plus the word-boundary check for underscore variants using NO_WORD_BEFORE / NO_WORD_AFTER. The positive non-word-char enumeration makes them multibyte-safe without requiring the `u` flag: `für_etwas` and `пристаням_стремятся_` correctly stay literal.
Per-mode unit tests cover basic matching, single-char bodies, leading/trailing-whitespace rejection, empty-delimiter rejection, paragraph-boundary rejection, multibyte intraword protection, and sort values. ModeRegistryTest's gating data provider picks up the three new rules.
show more ...
|