markdown-codec construct rows: link titles, task lists, code language, display math, raw HTML residue, blockquote divisions, definitions table - #765
Merged
Conversation
…ge as additive content fields ContentListMembership gains two optionals and ContentParagraph one, all carrying facts a reader previously had to encode in producer-side spellings or drop: - checked: a GFM task-list item's checkbox state, on the membership because it is a fact about the item rather than any one block; markdown-codec previously prepended checkbox glyph runs to fake it. - itemId: the identity of ONE list item, distinguishing "one item, several blocks" from "several items sharing this numId/level", which numId+level alone cannot express; a reader without item identity simply omits it. - codeLanguage: the source-format language identifier of a code-styled block (a markdown fence's info word), a free string because no format's language vocabulary is closed. The styles-table strict entry objects reject all three, so minting never factors them -- each stays a per-node fact, like styleId. JSON-schema $defs fragments updated to match, held by the existing live-comparison regression test.
…e plus quarantined remainder
The info string's first word is the language (CommonMark's own rule)
and lands on ContentParagraph.codeLanguage; everything after it is
pandoc-style attribute syntax with no cross-format meaning and
quarantines as markdown residue on the same paragraph, which this
package's own writer re-emits verbatim after the language word. An
info string opening with { is an attribute block with no language
word at all, so the whole string rides the residue.
The write side rebuilds the info line from the two halves with the
spec's canonical fence-space spelling, which also keeps an info word
beginning with the fence character from fusing into the fence.
Nothing is dropped any more, so CODE_BLOCK_INFO_STRING_DROPPED and
its tests are gone, and five CommonMark exclusions (24, 34, 142,
143, 144) leave the shrink-only list.
…nnotations A titled inline or reference link now opens a run-level link construct extent over its own runs -- the runs keep ContentRun.hyperlink (the standing reconciliation) and the descriptor adds the title, the one fact a flat run field cannot hold. The lowering walk appends into a shared run array so each extent records the paragraph's final run positions. A titled resolved image gets the block-scoped arm instead: a constructStart/End pair bracketing the image block, whose target carries the original destination verbatim, so the writer re-emits  rather than re-embedding bytes. The writer reads a covering extent for each hyperlink group and renders ](dest "title"), falling back to the plain form when no title covers it; an autolink-shaped run with a title uses the bracket form since <...> has no title slot. Paragraphs whose run extents do not name real runs are refused up front (MarkdownInvalidRunConstructExtentError, via document-schema.js's findRunConstructFault) rather than rendered at a guessed position. LINK_TITLE_DROPPED narrows to the shapes still dropping a title: a nested image inside a link or emphasis, and an unresolved image. 24 CommonMark exclusions leave the shrink-only list.
…st membership
Every block of one list item now carries the same minted itemId
(md-i{N}, drawn from the numId counter), so a multi-block item is
distinguishable from several single-block siblings sharing a numId and
level -- the writer groups same-id blocks into one item, one marker
line, later blocks continued on the continuation indent after a blank
line. Memberships with no itemId (every foreign producer) keep one
item per paragraph, unchanged.
The GFM checkbox state moves onto the membership's checked field,
retiring the checkbox-glyph run the lowering used to prepend; the
state now survives an item whose first block is one a glyph could
never be prepended to. The writer still recognises the old glyph
spelling for documents produced before the field existed.
LIST_ITEM_MULTI_BLOCK_FLATTENED is gone -- nothing is flattened any
more -- and 17 CommonMark exclusions leave the shrink-only list.
A blockquote now lowers to a division construct pair bracketing its blocks, one pair per nesting level -- the container boundary and the exact depth, both facts indentLeftPt alone never carried (adjacent same-depth quotes were indistinguishable from one multi-block quote; depth beyond level 1 was an approximation). The indent and Quote styleId stay on the inner blocks as the materialised formatting, so a construct-ignoring consumer still sees an indented quote paragraph. The writer renders a division as a blockquote wrapper only when the wrapped paragraphs carry the quote indent -- this package's own dual carry. A foreign division (an ODF text:section, a tagged-PDF /Sect) renders transparently under CONSTRUCT_UNREPRESENTED, because a named section is not a markdown blockquote and rendering it as one would invent a construct the source never had. One quote shape cannot carry the pair at all: a quote containing a heading anywhere in its subtree, because a marker extent may not open a heading scope and the last heading inside an extent always leaves one standing at the closing marker. Those quotes keep the old indent-only approximation (BLOCKQUOTE_CONTAINER_SKIPPED) so the heading itself keeps its heading fidelity -- flattening it to text, the footnote treatment, would regress a heading that today round trips. BLOCKQUOTE_NESTED_DEPTH is retired: nesting is exact now, and 4 more CommonMark exclusions leave the shrink-only list.
A $$ block now lowers to the one block-level carrier a wordprocessing ContentDocument has for a ContentFormula -- an embeddedObject of objectKind formula -- with the literal LaTeX verbatim in the formula's rendering-authoritative presentation layer, no MathML (markdown carried none to read), and no semantic content layer (nobody has lowered this LaTeX to semantics; that is a documents.js question, not something this package does on the way past). The frame is the zero box: an in-flow markdown block has no page position or intrinsic size to record, and ContentEmbeddedObjectBlock's frame is required -- origin-and-zero is the same "positioned by the flow, no size fact" spelling ooxml.js's inline OLE reader established for the identical constraint. The writer re-renders an embedded formula carrying presentation LaTeX as a fresh $$ pair; any other embedded object (another document kind, or a formula with only MathML) still drops silently, as it always did. The MathBlock styleId remains the write-side spelling for a paragraph that arrives styled that way. A display-math block inside a list item now reports LIST_ITEM_BLOCK_UNLISTED alongside tables and resolved images, the same gap it shares with them.
…n residue Preserved block-level and inline raw HTML now quarantines its verbatim source on the node that carries it -- the paragraph for an htmlBlock, the run for each inline rawHtml tag -- and this package's own writer re-emits that residue as-is, per the residue channel's restorable tier. The block path re-emits the untrimmed original in place of the separator-trimmed run text; the inline path bypasses escaping entirely for a residue-carrying run. That marker is also what finally distinguishes genuine preserved HTML from literal text that merely looks like a tag, so escapeMarkdownText loses its matchHtmlTag exemption and always escapes a literal '<' in ordinary text -- the exemption could only ever have been firing on literal text, since a real tag never reaches it as text any more. CommonMark exclusion 493 (the raw-text/tag ambiguity) leaves the shrink-only list.
…the package tree
The reference definition table src/block/definitions.ts builds is no
longer built and immediately discarded: readMarkdown splices it into
the package's own definitions root as the link tenant ({ kind: 'link',
destination, title? }, keyed by normalised label), and writeMarkdown
renders this package's own entries back out as [label]: dest "title"
lines after the body -- so PACKAGE_TABLE_DROPPED now reports the
definitions table only when it holds a foreign tenant this package
cannot render. The verbatim front-matter block rides the package-level
source residue table under the frontmatter key, and writeMarkdown
re-emits it as-is in place of the metadata-generated block whenever
frontMatter: true asks for one at all, so unmapped keys and original
spellings survive the round trip.
Both carries are tree-only, which is the design decision this row
turned on: document-schema.js's tables and package-level residue are
tree-only by construction, the flat ContentDocument has no root to
carry them, and reference USE sites already resolve to
hyperlink-carrying runs at parse time -- only re-emission fidelity
ever needed the table. The flat pair therefore behaves exactly as
before, and the tree pair is assemblePackage of the flat document
plus the two splices, added only when non-empty so definition-free
and front-matter-free sources keep the exact assemblePackage shape.
The lower pipeline grows lowerMarkdownDetailed for this: the same
composition lowerMarkdown always ran, additionally surfacing the
reference map and the raw front-matter block for the tree-level read.
…ula carry markdown-codec's fidelity-constructs row changed what a $$ block looks like on the way out of its reader: no longer a MathBlock-styled paragraph carrying raw LaTeX text, but an embedded formula object holding the verbatim presentation layer and nothing else. The math pass here recognises that block shape directly and replaces it in place with the fully lowered two-layer formula -- same position in the flow, now carrying content, MathML, and provenance. The inline \( \) span recognition is untouched (still a Cambria-Math-marked run), and the write side needs no change: it rebuilds the display/ inline spellings markdown-codec's emitter still renders. The CLI's openDocument diagnostic test leaned on the now-retired info-string diagnostic as its guaranteed-fire vehicle; it moves to the raw-HTML preserved-as-text diagnostic, which still fires, since a fence's language word now carries semantically and nothing drops.
… writer markdown-codec's fidelity-constructs row makes its read side emit marker pairs of its own -- a division pair for every blockquote, a link pair around a titled resolved image -- so buildMarkdownText's MarkdownConstructUnsupportedError refusal would have refused this package's own editor and conversion round trips: opening a markdown file containing a blockquote and saving it threw. The refusal's stated reason (markdown-codec's writer has no arm for either marker kind) no longer holds: the writer resolves marker pairs as balanced brackets, renders each construct it has a spelling for, and renders the rest transparently under a diagnostic. The markers now pass through, and a genuinely unbalanced list still throws, by markdown-codec's own MarkdownUnbalancedConstructMarkersError -- the shared definition of that check. The math-lowering and diagnostic-vocabulary notes in the READMEs move to the new shapes, and the fidelity table in markdown-codec's README picks up the shrunk conformance numbers.
Contributor
|
🎉 This PR is included in version 4.8.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 1.0.9 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 4.1.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 5.1.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 4.4.2 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 3.2.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 4.4.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 3.2.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 2.1.32 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 1.26.38 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 1.1.1 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 1.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Contributor
|
🎉 This PR is included in version 1.2.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands the markdown semantic-construct rows from the fidelity inventory, plus the additive schema fields they need and the documents.js pass-throughs to keep conversions carrying them.
Schema (additive):
ContentListMembership.checked(GFM task state) and.itemId(list-item identity — "one item, several blocks" vs "several items sharing numId/level"), andContentParagraph.codeLanguage(a fence's info word).markdown-codec
linkconstruct'stitlefield — a run-level extent for an inline or reference link, a block-scoped marker pair around a resolved image, which also restores the image's original destination on the way out.codeLanguage) plus a quarantined pandoc-style{.class #id}remainder in residue.$$math lowers to a real embedded formula document (presentation layer); inline\(...\)stays a Cambria-Math-marked LaTeX run, documented as such.rawHtml: 'quarantine'.definitionstable and render back out as[label]: dest "title"lines; unmapped front-matter keys quarantine as package-level residue instead of dying at the key switch.divisionconstruct pair (exact boundary and depth) with the materialised indent kept; a quote containing a heading degrades to indent-only with a diagnostic, since a marker extent may not open a heading scope.documents.js: construct markers pass through to markdown-codec's writer on
markdownToX-shaped conversions, and display math lowers from the embedded formula carry on the way into markdown.Closes #722
Testing
Known remaining, tracked on #750's run-level arm: footnote reference sites still lower to marked runs rather than anchor extents.