| #
945681fc
|
| 05-Jul-2026 |
Andreas Gohr <andi@splitbrain.org> |
fix(parser): keep an indented list from being swallowed by a table
The parser rework dropped the list-awareness lookahead from preformatted's entry patterns ('\n (?![\*\-])' became '\n '). In base
fix(parser): keep an indented list from being swallowed by a table
The parser rework dropped the list-awareness lookahead from preformatted's entry patterns ('\n (?![\*\-])' became '\n '). In base mode this is masked by sort order - listblock sorts before preformatted and wins the tie - but table mode connects preformatted as a protected sub-mode without connecting any list mode. Inside a table an indented list line therefore entered preformatted instead of ending the table, and the table rewriter discarded the resulting call: the list vanished from the output and the table's section-edit range grew to cover it, so saving via the inline table editor overwrote the list text.
Restore the lookahead. For DokuWiki syntax it is the original '(?![\*\-])' (Listblock treats a bare * or - as a marker). For Markdown-preferred syntax the guard follows GfmListblock, rejecting -, *, + and ordered markers only when followed by whitespace or end of line, so mixed dw+md / md+dw wikis are fixed too while an indented code block that merely starts with such a character (e.g. a *** line) still parses as code.
show more ...
|
| #
e248eb6f
|
| 05-Jul-2026 |
Andreas Gohr <andi@splitbrain.org> |
fix(parser): don't drop document content after a blank indented line
Preformatted's exit pattern (?=\n[^ \t\n]) is a lookahead-only exit: it consumes no byte so the boundary \n stays in the stream f
fix(parser): don't drop document content after a blank indented line
Preformatted's exit pattern (?=\n[^ \t\n]) is a lookahead-only exit: it consumes no byte so the boundary \n stays in the stream for a following block mode (an <hr> or header after an indented code block) to anchor on.
When that exit fired with nothing else consumed - a "blank" line that is empty after its indent, followed by a column-0 line - the match was zero-width at the current offset and the lexer's no-advance guard aborted the whole parse, silently discarding the rest of the document. Reachable on ordinary trailing whitespace ("abc\n \nmore") and inside footnotes.
Exempt zero-width MODE_EXIT matches from the guard: popping the mode stack is real progress and leaves the boundary byte for the parent mode to consume on the next iteration. The stack strictly shrinks on each such exit, so this cannot loop; the infinite-loop protection is unchanged for every other zero-width match. Preformatted's pattern is left as-is, so <hr>/header after an indented code block still works, including after a blank indented line.
Add regressions for the blank-line, blank-line-after-code, and boundary-preservation (hr after a blank indented line) cases.
show more ...
|
| #
47a02a10
|
| 04-Jun-2026 |
Andreas Gohr <gohr@cosmocode.de> |
Parsing: make parse syntax a per-parse value, drop ModeInterface
The active parse's syntax flavour is a per-parse question, not process- global state: within a single request a plugin can render bun
Parsing: make parse syntax a per-parse value, drop ModeInterface
The active parse's syntax flavour is a per-parse question, not process- global state: within a single request a plugin can render bundled DokuWiki-syntax text inside an otherwise-Markdown page. Yet ModeRegistry was a singleton that read $conf['syntax'] and the $PARSER_MODES global, and every mode reached it through ModeRegistry::getInstance() — so the flavour lived in shared mutable state that two parses in one request would fight over.
Make the registry a short-lived value instead:
- ModeRegistry is constructed once per parse with an explicit $syntax and injected into Parser, Handler and every mode. getSyntax() / isDwPreferred() / isMdPreferred() consult $this->syntax; the DOKU_UNITTEST-gated mode-list cache hack is gone (each registry is fresh, nothing to invalidate). - p_get_instructions() is now the single place in the pipeline where $conf['syntax'] is read; from there the flavour travels as a parameter. No code under inc/Parsing/ reads $conf['syntax'] directly anymore — the five syntax-reading modes (Preformatted, GfmHr, GfmEscape, Externallink, GfmQuote) route through $this->registry.
Keep the two concepts apart, as documented in the ModeRegistry and AbstractMode docblocks: the user's configured *preference* stays in $conf['syntax'] for UI code (toolbar, settings), while the active parse's syntax is a parameter carried by the registry.
$PARSER_MODES is demoted to a deprecated, read-only mirror, published during loadPluginModes() — third-party syntax plugins (columnlist, alphalist2, phpwikify, skipentity) and the bundled info plugin read the global directly, often from their constructors, so the taxonomy must stay visible there. No core code reads the mirror.
Fold ModeInterface into AbstractMode while here: getSort()/handle() are abstract, the connect callbacks carry defaults, and the public $Lexer "FIXME should be done by setter" becomes setLexer()/getLexer() injected by Parser::addMode() alongside the registry. Nested-content resolution moves to the allowedCategories()/filterAllowedModes() hooks, resolved once when the registry is attached.
Tests build their own parser/registry through ParserTestBase::setSyntax() instead of mutating $conf and calling the removed ModeRegistry::reset().
show more ...
|
| #
113171bb
|
| 05-May-2026 |
Andreas Gohr <gohr@cosmocode.de> |
Preformatted: strip leading/trailing blank-line padding from body
The lexer's `\n ` continuation pattern eats the indent off blank-but-indented source lines, leaving a `\n` in the rewriter buffer
Preformatted: strip leading/trailing blank-line padding from body
The lexer's `\n ` continuation pattern eats the indent off blank-but-indented source lines, leaving a `\n` in the rewriter buffer for each one. Trim those padding newlines before emitting so the preformatted body starts and ends on a non-blank line, as GFM example #87 requires. Whitespace-only blocks are still skipped entirely.
Adds two PreformattedTest cases pinning the new behavior.
show more ...
|
| #
c2248fda
|
| 05-May-2026 |
Andreas Gohr <andi@splitbrain.org> |
Block: collapse [ \t]*\n[ \t]* to \n inside paragraphs
Strip [ \t]+ on either side of the soft-break joiner emitted for a single eol, and ltrim the first cdata of each paragraph. Without this, DokuW
Block: collapse [ \t]*\n[ \t]* to \n inside paragraphs
Strip [ \t]+ on either side of the soft-break joiner emitted for a single eol, and ltrim the first cdata of each paragraph. Without this, DokuWiki preserved leading/trailing whitespace on continuation lines verbatim, which is invisible in HTML but may visible in plain-text and other renderers. It is also a requirement in the Markdown spec.
Re-baseline the parser-mode tests that pinned the old preserve behavior (cdata adjacent to <code>/<file>/<rss>/header/footnote).
show more ...
|
| #
f57da51c
|
| 05-May-2026 |
Andreas Gohr <gohr@cosmocode.de> |
Preformatted: leave boundary \n in stream when next line has content
Adds a zero-width lookahead exit (?=\n[^ \t\n]) ahead of the existing consuming \n exit. When an indented code block is followed
Preformatted: leave boundary \n in stream when next line has content
Adds a zero-width lookahead exit (?=\n[^ \t\n]) ahead of the existing consuming \n exit. When an indented code block is followed by a non- blank line, the boundary newline now stays available for downstream block-level matchers (GfmHr, GfmHeader, etc.) instead of being eaten on the way out of preformatted mode.
Concretely fixes a thematic-break-after-indented-code case (GFM spec case 85's trailing ----): without this change, GfmHr's \n anchor failed because preformatted had already consumed the newline, and the bare ---- fell through to Entity which converted --- to an em-dash.
The consuming branch is kept as a fall-through for the blank-line and end-of-input cases, where a pure lookahead would trip the lexer's no-advance safety check.
Six PreformattedTest expectations updated: trailing cdata after a preformatted block now carries the leading \n (rendered output is unchanged — paragraph whitespace is trimmed).
show more ...
|
| #
13a62f81
|
| 04-May-2026 |
Andreas Gohr <andi@splitbrain.org> |
rename syntax flavors 'dokuwiki' / 'markdown' to 'dw' / 'md'
Symmetry with the existing 'dw+md' / 'md+dw' setting values.
|
| #
96d096f1
|
| 27-Apr-2026 |
Andreas Gohr <andi@splitbrain.org> |
remove getLineStartMarkers registry — sort order already wins
Preformatted's entry pattern carried a `(?![\*\-])` negative lookahead to defer to list modes on indented bullet lines. 0cecf9d50 (2005,
remove getLineStartMarkers registry — sort order already wins
Preformatted's entry pattern carried a `(?![\*\-])` negative lookahead to defer to list modes on indented bullet lines. 0cecf9d50 (2005, "new parser added") introduced it hardcoded; 7958e6980 (2026, "decouple hardcoded mode names in Eol and Preformatted") refactored that hardcoded knowledge into register/getLineStartMarkers on ModeRegistry so each list mode owned its marker chars. Both preserved the behavior verbatim; neither documented why it was needed.
Tracing the lexer, it isn't. ParallelRegex merges all entry patterns into one PCRE expression; PCRE returns the leftmost match and breaks ties on expression order. Modes are added in sort order via ModeRegistry::getModes(), so Listblock (sort 10) always precedes Preformatted (sort 20) and wins the tie on " - foo" without any lookahead. The only test that caught a difference was testPreformattedList, which happened to register modes in non-canonical order - that was a test bug.
This patch drops the lookahead in Preformatted::connectTo, the registerLineStartMarkers call in Listblock::preConnect, the register/getLineStartMarkers methods on ModeRegistry, and the three registry-API unit tests. testPreformattedList now registers Listblock before Preformatted.
show more ...
|
| #
b1c59bed
|
| 23-Apr-2026 |
Andreas Gohr <andi@splitbrain.org> |
add GfmCode / GfmFile for fenced code blocks
GfmCode (backticks) emits the `code` handler instruction; GfmFile (tildes) emits `file`. Column-0 fences only, no length pairing between opener and close
add GfmCode / GfmFile for fenced code blocks
GfmCode (backticks) emits the `code` handler instruction; GfmFile (tildes) emits `file`. Column-0 fences only, no length pairing between opener and closer, and unclosed fences stay literal — matching DokuWiki's `<code>` tag convention. The info string accepts DW's full attribute vocabulary (language, filename, [options]) through a new shared `Helpers::parseCodeAttributes` that `Code` also uses, with `html` aliased to `html4strict` and `-` meaning "no language".
Preformatted's indent threshold is now preference-gated: 2 spaces in DW-preferred settings, 4 spaces in MD-preferred, matching GFM's indented code block rule. A single tab is a trigger in both.
show more ...
|
| #
504c13e8
|
| 18-Apr-2026 |
Andreas Gohr <andi@splitbrain.org> |
migrate parser tests to modern namespaced classes
Move old-style parser tests from _test/tests/inc/parser/ to namespaced test classes under _test/tests/Parsing/ParserMode/ and _test/tests/Parsing/Le
migrate parser tests to modern namespaced classes
Move old-style parser tests from _test/tests/inc/parser/ to namespaced test classes under _test/tests/Parsing/ParserMode/ and _test/tests/Parsing/Lexer/.
- Add ParserTestBase with assertCalls() helper for comparing handler call sequences with byte index stripping - Split lexer.test.php into ParallelRegexTest, StateStackTest, and LexerTest with a RecordingHandler that extends Handler - Merge handler_parse_highlight_options tests into CodeTest - Add new tests for previously untested modes: Nocache, Notoc, Rss, and all individual formatting modes (Strong, Emphasis, etc.) - Modernize test code: [] syntax, lowercase null, correct assertEquals argument order, replace deprecated withConsecutive and string callables - Renderer tests remain in old location (renderers not yet migrated)
show more ...
|