Add an option to produce fine-grained HTML renderers based on CommonMark's grammar by herley-shaori · Pull Request #635 · Witiko/markdown

herley-shaori · 2026-03-11T12:19:21Z

Summary

Implements the first task of #606 by exposing CommonMark's HTML block type differentiation through individual renderers.

A new boolean option parseHtmlBlocks (default: false) is added. When enabled alongside the html option, the parser produces type-specific renderers instead of the generic inputBlockHtmlElement and inlineHtmlTag renderers:

Block HTML renderers (each receives a filename of a file containing the HTML block contents):

Renderer	CommonMark type	Matches
`inputBlockHtmlCommentElement`	Type 2	`<!-- ... -->`
`inputBlockHtmlInstructionElement`	Type 3	`<? ... ?>`
`inputBlockHtmlDeclarationElement`	Type 4	`<! ... >`
`inputBlockHtmlCdataElement`	Type 5	`<![CDATA[ ... ]]>`
`inputBlockHtmlSpecialElement`	Type 1	`<script>`, `<pre>`, `<style>`, `<textarea>`
`inputBlockHtmlRegularElement`	Type 6	`<div>`, `<table>`, `<form>`, etc.
`inputBlockHtmlAnyElement`	Type 7	Any other complete tag on its own line

Inline HTML renderers (each receives the tag contents as a string):

Renderer	Matches
`inlineHtmlInstruction`	`<? ... ?>`
`inlineHtmlDeclaration`	`<! ... >`
`inlineHtmlCdataSection`	`<![CDATA[ ... ]]>`
`inlineHtmlOpenTag`	`<tag>`
`inlineHtmlCloseTag`	`</tag>`
`inlineHtmlEmptyTag`	`<tag/>`

The existing inlineHtmlComment renderer remains unchanged. When parseHtmlBlocks is false (default), behavior is fully backward compatible.

Changes

markdown.dtx: Added parseHtmlBlocks option definition, 13 new renderer registrations, 13 new Lua writer functions, conditional parser routing in DisplayHtml and InlineHtml, and documentation for all new renderers.
tests/support/keyval-setup.tex: Added test renderer prototypes for all 13 new renderers.
tests/testfiles/regression/github/issue-606-block-html-types.test: New regression test verifying type-specific renderers are produced when parseHtmlBlocks is enabled.

Test plan

New regression test passes across all non-ConTeXt TeX formats (luatex, lualatex, pdflatex, pdftex)
All 44 existing CommonMark_0.30/html_blocks tests pass (backward compatibility)
Both existing CommonMark_0.31.2/raw_html tests pass (backward compatibility)
ConTeXt tests (skipped locally due to missing rename utility — CI should cover this)

Note

This PR addresses Task 1 of #606. Task 2 (renderers corresponding to HTML nodes) would require a more substantial HTML parser and is left for a follow-up.

Continues #606.

…TML types Implement the first task of issue Witiko#606: expose CommonMark's HTML block type differentiation through individual renderers. When the new `parseHtmlBlocks` option is enabled (default: false), the parser produces type-specific renderers instead of the generic `inputBlockHtmlElement` and `inlineHtmlTag` renderers: Block HTML renderers (by CommonMark type): - inputBlockHtmlCommentElement (type 2: HTML comments) - inputBlockHtmlInstructionElement (type 3: processing instructions) - inputBlockHtmlDeclarationElement (type 4: declarations) - inputBlockHtmlCdataElement (type 5: CDATA sections) - inputBlockHtmlSpecialElement (type 1: script/pre/style/textarea) - inputBlockHtmlRegularElement (type 6: div/table/form etc.) - inputBlockHtmlAnyElement (type 7: any other complete tag) Inline HTML renderers: - inlineHtmlInstruction (processing instructions) - inlineHtmlDeclaration (declarations) - inlineHtmlCdataSection (CDATA sections) - inlineHtmlOpenTag (opening tags) - inlineHtmlCloseTag (closing tags) - inlineHtmlEmptyTag (self-closing tags) The existing inlineHtmlComment renderer remains unchanged. When parseHtmlBlocks is false (default), behavior is fully backward compatible. Closes Witiko#606 (task 1)

Witiko · 2026-03-11T13:26:33Z

Hi @herley-shaori, this looks great, especially for a first-time contribution. Thanks for putting in the effort!

Below, I reviewed the code. If you'd like to make the necessary changes yourself, that would be great; otherwise, I'm happy to take over the PR from here.

…option Apply all changes requested in the code review: - Rename `parseHtmlBlocks` (boolean) option to `htmlOutput` (string) with values `basic` (default) and `commonmark`, making the design more future-proof for potential additional values like `nodes`. - Simplify option documentation to not list individual renderers (consistent with other option descriptions in the codebase). - Fix documentation markup: replace `\Mdef` with `\mref` for cross-references to renderers defined elsewhere. - Add `[raw-html]` link reference for inline HTML construct types. - Fix Lua code indentation in InlineHtml and DisplayHtml parser sections. - Remove unnecessary `if: format != 'context'` condition from test file. - Update .gitignore entry from `venv/` to `tests/test-virtualenv`.

Witiko · 2026-03-11T20:52:10Z

Tasks for myself in addition to the comments from the code review:

Set htmlOutput = "commonmark" when the experimental option has been enabled.
Improve the naming of writers, i.e. why block_html_comment_element when it's an HTML node (not element).
- Same for renderers and renderer prototypes, why \markdownRendererInputBlockHtmlCommentElement rather than just \markdownRendererInputBlockHtmlComment?
Add a code example redefining \markdownRendererInputBlockHtmlComment to the user manual, similar to the existing code example for \markdownRendererInlineComment.
- Add other code examples, e.g. redefining \markdownRendererInlineHtmlOpenTag and \markdownRendererInlineHtmlCloseTag, as requested by @u-fischer in handling inline html #597 (comment).
Update CHANGES.md.

herley-shaori · 2026-03-11T21:35:59Z

Hi @Witiko, thanks so much for taking the time to review this! Your feedback is really valuable. I realize I still have a lot to learn — I'll step back and leave this PR to you. If there's anything I can do to help in the future, happy to contribute! 😊

- Add \mref{markdownRendererInlineHtmlComment} to the `basic` option description for completeness. - Add [html-blocks] and [raw-html] link references to the option documentation fragment (each \begin{markdown} fragment needs its own link references). - Split renderer documentation into separate sections: - Rename "HTML Tag and Element Renderers" to "Basic HTML Tag and Element Renderers" for the generic renderers. - Add "CommonMark Block HTML Element Renderers" section for type-specific block renderers. - Add "CommonMark Inline HTML Renderers" section for type-specific inline renderers. - Fix Lua code style: add space after Cs( in else branches of InlineHtml and DisplayHtml parsers for consistency.

Move issue-606-block-html-types.test from regression/github/ to unit/lunamark-markdown/html-output-commonmark.test as requested in review feedback.

Co-authored-by: Andrej Genčur <xgencur@fi.muni.cz>

Witiko

@herley-shaori: I found a few other issues that still need to be addressed. If you have the time and would like to, please feel free to take care of them; otherwise, I can finish them myself.

…:herley-shaori/markdown into feature/issue-606-parse-html-block-types

herley and others added 2 commits March 11, 2026 18:15

Fix style issues

b7bd3f9

Witiko requested changes Mar 11, 2026

View reviewed changes

Witiko added lua Related to the Lua interface and implementation conversion output Related to the output format of the Markdown-to-TeX conversion labels Mar 11, 2026

Witiko reviewed Mar 11, 2026

View reviewed changes

Comment thread markdown.dtx Outdated

Witiko reviewed Mar 11, 2026

View reviewed changes

Comment thread markdown.dtx Outdated

Witiko reviewed Mar 11, 2026

View reviewed changes

Comment thread markdown.dtx Outdated

Witiko reviewed Mar 11, 2026

View reviewed changes

Comment thread markdown.dtx Outdated

Witiko reviewed Mar 11, 2026

View reviewed changes

Comment thread markdown.dtx Outdated

herley and others added 3 commits March 12, 2026 07:07

Implement review comments

995a536

Escape HTML tags

e74e52b

Witiko previously approved these changes Mar 17, 2026

View reviewed changes

Witiko requested a review from lostenderman March 17, 2026 13:39

Witiko reviewed Mar 17, 2026

View reviewed changes

Comment thread tests/testfiles/unit/lunamark-markdown/html-output-commonmark.test Outdated

Move HTML block types test to unit tests directory

251c5a5

Move issue-606-block-html-types.test from regression/github/ to unit/lunamark-markdown/html-output-commonmark.test as requested in review feedback.

herley-shaori dismissed Witiko’s stale review via 251c5a5 March 18, 2026 01:21

Witiko added 3 commits March 18, 2026 09:28

Add more unit tests

b98b7cd

Move around renderer definitions to order sections alphabetically

e7336a6

Strip leading and trailins spaces for inlineHtmlComment renderers

d51349f

Witiko changed the title ~~Add parseHtmlBlocks option for type-specific HTML renderers~~ Add htmlOutput option to produce fine-grained HTML renderers based on CommonMark's grammar Mar 18, 2026

Witiko changed the title ~~Add htmlOutput option to produce fine-grained HTML renderers based on CommonMark's grammar~~ Add an option to produce fine-grained HTML renderers based on CommonMark's grammar Mar 18, 2026

Update CHANGES.md

e040626

Witiko added this to the 3.15.0 milestone Mar 18, 2026

Improve technical documentation

0a7a5e1

Witiko and others added 7 commits March 29, 2026 22:04

Implement review comments

f6d359c

Co-authored-by: Andrej Genčur <xgencur@fi.muni.cz>

Implement review comments

4402524

Co-authored-by: Andrej Genčur <xgencur@fi.muni.cz>

Fix a typo

394bb86

Improve documentation of option html

0594ff1

Use option blankBeforeHtmlBlock in inline HTML comment example

d33dc31

Improve consistency in documentation

c08f853

Rename instruction -> processingInstruction

20328d4

Witiko requested changes Mar 29, 2026

View reviewed changes

Comment thread tests/support/keyval-setup.tex Outdated

Comment thread tests/support/keyval-setup.tex

Comment thread tests/support/keyval-setup.tex Outdated

Comment thread markdown.dtx Outdated

Comment thread markdown.dtx Outdated

Merge branch 'main' into feature/issue-606-parse-html-block-types

e638431

Witiko reviewed Mar 30, 2026

View reviewed changes

Comment thread CHANGES.md Outdated

Witiko and others added 2 commits March 30, 2026 18:45

Update CHANGES.md

599304f

Test blockHtmlStandaloneTag

8ef5f26

Witiko reviewed Apr 2, 2026

View reviewed changes

Comment thread markdown.dtx Outdated

Witiko added 16 commits April 2, 2026 10:52

Test inlineHtml(ProcessingInstruction|Declaration|CdataSection)

06c22df

Test inputBlockHtml(ProcessingInstruction|Declaration|CdataSection)

9a41398

Merge branch 'feature/issue-606-parse-html-block-types' of github.com…

ce7fa4d

…:herley-shaori/markdown into feature/issue-606-parse-html-block-types

Update input filenames

c42b2df

Replace inputBlockHtmlComment with blockHtmlComment

7bca77c

Fix a typo

1bb660b

Fix another typo

415299c

Update input filenames

ccc6f0c

Define blockHtmlComment for tests

6d74b11

Fix the captures in parsers.html_comment

487aad3

Fix a typo in html-output-commonmark.test

ad6c341

Implement renderer prototype fallbacks

857aa6d

Add two missing %s

14f3dba

Reintroduce remove_trailing_blank_lines()

bb49764

Fix a typo in renderer prototype fallbacks

c33fae0

Continue HEAD^^

f4b4a81

Witiko merged commit 5474fb8 into Witiko:main Apr 2, 2026
14 checks passed

Conversation

herley-shaori commented Mar 11, 2026 • edited by Witiko Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Note

Uh oh!

Witiko commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Witiko commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

herley-shaori commented Mar 11, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Witiko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

herley-shaori commented Mar 11, 2026 •

edited by Witiko

Loading

Witiko commented Mar 11, 2026 •

edited

Loading

Witiko commented Mar 11, 2026 •

edited

Loading