feat!: promote DocumentPackage to the tree form at the package boundary - #634
Merged
Conversation
document-schema.js 4.0.0 retires the flat { formatVersion, content, pages }
DocumentPackage envelope for the tree form, drops the per-arm ContentDocument
formatVersion literal, and moves the Layout* item family to pdf-codec; ooxml.js
3.0.0, odf.js 4.0.0, and markdown-codec 3.0.0 are their schema-4 migrations.
All four bump together: a schema-4 sibling beside a schema-3 one leaves two
copies of document-schema.js in the pnpm tree, whose nominally-unrelated
ContentDocument types fail typecheck.
Schema 4.0.0 made ContentDocument a purely in-process codec-exchange type: versioning lives only at the serialised package boundary, in the release-pinned $schema URI documentPackageWithSchema stamps. Every reader wrapper, variant bridge, and fixture builder that constructed a content literal with CONTENT_FORMAT_VERSION drops the field; nothing about the content model itself changed. BREAKING CHANGE: CONTENT_FORMAT_VERSION no longer exists and ContentDocument arms carry no formatVersion field.
Schema 4.0.0 demoted LayoutDocument and the whole Layout* item family to pdf-codec: a layout document is that codec's own read/write artefact, not a shared pivot, so the layout engines, reconstructors, PDF editors, the metadata write path, and the codec registry take the item types, LAYOUT_FORMAT_VERSION, and LayoutDocumentSchema from pdf-codec's barrel. LayoutMetadata, LayoutFont, and LayoutFrame stay schema-sourced -- the content model itself references them. The registry states its two-function layout entry as a local structural type, the shape document-schema.js's retired LayoutCodec port used to give it. BREAKING CHANGE: LAYOUT_FORMAT_VERSION and the Layout* item types re-export from pdf-codec; layoutDocumentWithSchema and LayoutDocumentJson no longer exist.
…mbering
ContentListMembership.numId became optional at schema 4.0.0 -- OOXML drawing
paragraphs carry only a level, and a de-numIded bridge product is legal
content. The wordprocessing layout engine's marker pass treats an absent numId
as carrying no ordering information (only markdown-codec's minted md{n}: ids
ever name an ordered list, so the bullet cycle is the only honest marker). The
docx numbering pre-pass keys its map by numId-or-absence so every membership
lands on a numbering definition, with all no-numId paragraphs sharing one w:num
-- exactly right for a synthesiser whose every level is the same bullet
template anyway -- and DocxParagraph.list now refuses a numId-less membership
loudly, naming the pre-pass that mints one, rather than writing a w:numPr that
references nothing.
The engine's import line also picks up the pdf-codec Layout flip this change
builds on.
The lossless tree <-> flat pair, ported from document-outline.js's phase-1 reference implementation (that repo's pre-re-charter history) onto schema 4's own tree vocabulary: decompose takes the flat ContentDocument and embeds its node objects into per-container groups -- mandatory section groups carrying pre-layout geometry, one shape group per shape so a slide's paragraphs never flatten across its boundaries, a sheet's grid riding its node while children carry images and embedded objects, embedded documents staying whole leaves. flattenPackage is the exact inverse, with style-ref resolution fused into the walk because the flat codec-exchange form is always fully materialised. The one forced adaptation beyond types: the reference decomposed a 3.x package's content field and rebuilt envelopes through a separate helper, while schema 4's DocumentPackage IS the tree, so decompose takes the content directly and the envelope rides the package root assemblePackage builds. canonicalise is the shared canonical-key recipe (stableContentHash's own, ported so tuple identity and the bijection comparator cannot drift from it). Ownership discipline unchanged from the reference: decompose wraps, it never copies, so the frames a layout pass stamped onto content are the same objects in both encodings; flatten without a styles table returns those same objects.
document-schema.js#21's factoring pass, which the schema deliberately leaves to documents.js: assemblePackage = decompose then factorStyles, separate passes so minting idempotence stays independently testable. The mint walks the fresh tree outermost-first and hoists property tuples occurring two or more times onto a group-wrapper ref plus a styles-table entry -- pure compression over the one tree a conversion just built, never a second authority for content. Exactness (the promotion's law ii) rests on two rules, both consequences of the resolution helpers being gap-fill-never-overwrite: a minted tuple's keys must be carried by every paragraph and every extent run of the wrapper's whole subtree extent, and a key already minted by an ancestor entry is frozen for everything below it so a nested re-mint can never shadow the value the ancestor's ref restores. Mintable keys are the schema's own style halves minus list -- membership is a grouping signal the tree itself must keep stating, and frames/sourcePath/styleId are per-node facts the schema's strict entries refuse outright. Ids are s1, s2, ... in descending-frequency, first-occurrence, canonical-tuple order, so the pass is deterministic; factorStyles flattens its input first, which makes re-factoring mint the identical table by construction.
One helper, all sites: assemblePackage(content, pages?) = decompose then
factorStyles, called wherever a conversion used to hand-construct the flat
{ formatVersion, content, pages } envelope. The composition engine's three
executors (bridge, toPdf, fromPdf) and convert.ts's special cases (odfToPdf,
odbReportToPdf, odbToXlsx, odbToCsv) all emit the tree now, and every site
fires onDocument AFTER the output bytes are built -- executeBridge
historically fired before the build and is reordered, so a callback that
inspects the tree can never observe a half-built conversion. The codecs'
readers and builders keep producing and consuming the flat ContentDocument
unchanged; the tree exists only at the boundary.
index.ts exports the boundary itself (assemblePackage, decompose,
flattenPackage, factorStyles, PackageChildren) beside the pdf-codec item-family
re-exports, and drops the retired CONTENT_FORMAT_VERSION /
layoutDocumentWithSchema / LayoutDocumentJson surface.
BREAKING CHANGE: onDocument and ConversionResult.package carry the tree-form
DocumentPackage of schema 4.0.0 -- children hold the decomposed groups and the
minted styles table instead of the flat content envelope.
Every package consumer flattens once and keeps working over the flat form: buildDocumentBytes feeds content.write the materialised ContentDocument, layoutDocumentFromPackage walks the flattened content (one tree-to-flat authority, and the frames walk stays the flat-document walk it always was -- the deliberate one-implementation choice, documented on the function), and lintMathCoherence flattens at entry while keeping its exported DocumentPackage signature so callers hand back exactly what onDocument gave them. BREAKING CHANGE: a tree-form package is the only accepted input shape.
ConversionResult.package changed type with the tree promotion, which is exactly what the version-history comment on contractVersion exists to record: a consumer reading the field must flatten (documents.js exports flattenPackage) or walk the tree, where the flat envelope it used to read is gone. The supported-conversions table is unchanged -- the port's pairs were always derived from the composition pathfinder, whose routes did not move.
The promotion's merge gate: the three laws re-run over this repo's actual output -- reader outputs for every format, editor builds encoded and read back, xlsx through ooxml.js's own pair, csv and svg text, standalone and sheet-embedded formulas, an .odb table extraction, a rendered .odb report, a multi-section concatenation, reconstruction captures, and onDocument captures from every conversion family carrying real frames and pages. Law (i) asserts flatten(assemble(c)) reproduces c up to the declared empty-embeddedObjects normalisation, law (ii) asserts the materialised flat form is ref-free and structurally identical to the source, law (iii) asserts reassembly and re-factoring mint identical trees -- each against a structuredClone snapshot taken before the round trip, never an identity assertion, because decompose shares node objects by design. A closing assertion pins that the corpus really exercises minting, so the gate cannot pass vacuously.
…ldens One committed package golden per variant now -- wordprocessing, presentation, spreadsheet, drawing, and formula -- each captured from that variant's own to-pdf conversion so the committed tree carries real frames, real pages, and a real schema-valid shape; layout-document.json is dropped because the demoted pdf-codec LayoutDocument has no $schema-stamped JSON envelope to validate a golden against. The README's intermediate-package section states the tree, the three laws with their merge-gate suite, and the assemblePackage boundary, and the pivot/architecture/references wording follows the demotion (the shared model is ContentDocument; LayoutDocument is pdf-codec's view).
The spreadsheet layout engine's fixtures follow the same content-literal drop as every other test file, with the Layout item types the engine imports picked up from pdf-codec alongside it.
Code Coverage OverviewLanguages: TypeScript TypeScript / unitThe overall coverage in commit 8ccb27b in the Show a code coverage summary of the most impacted files.
Updated |
… list The dist-parity smoke check still asserted the retired schema-io helper; the boundary functions that replaced the export's neighbours in the public surface (assemblePackage, decompose, flattenPackage, factorStyles) join the list so the built artifact keeps proving both builds carry the package boundary.
The live-view gotcha predates the tree promotion and still spoke of mutating a content field the package no longer carries; the tree's own content nodes are what a post-layout mutation would touch now.
This was referenced Aug 18, 2026
Contributor
|
🎉 This PR is included in version 3.0.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #623.
The keystone of the DocumentPackage promotion: schema 4.0.0 defines the tree, and this PR makes it real at this package's boundary. One helper,
assemblePackage(content, pages?)= decompose thenfactorStyles, behind every construction site;flattenPackageat every consumer; the readers and builders keep producing and consuming the flatContentDocumentunchanged.Port — decompose/flatten brought in from document-outline.js's phase-1 reference (its pre-re-charter history) onto schema 4's own tree vocabulary. The one forced adaptation beyond types: the reference decomposed a 3.x package's
contentfield and rebuilt envelopes through a separate helper; schema 4'sDocumentPackageis the tree, so decompose takes the flat content directly and the envelope rides the root. Ownership discipline intact — decompose wraps, never copies, so layout-stamped frames are the same objects in both encodings.Minting —
factorStyleswalks the fresh tree outermost-first and hoists property tuples occurring two or more times onto a group-wrapper ref plus astylesentry. Exactness (law ii) rests on two rules forced by gap-fill-never-overwrite resolution: a tuple's keys must be carried by every paragraph/run of the wrapper's whole extent, and an ancestor's minted keys are frozen below it.listnever factors (a grouping signal the tree must keep stating);frames/sourcePath/styleIdare per-node facts the schema's strict entries refuse outright. Deterministic s1, s2, ... ordering; idempotent by construction.Sites — the composition engine's three executors plus convert.ts's special cases (odfToPdf, odbReportToPdf, odbToXlsx, odbToCsv) all emit the tree, and every site fires
onDocumentafter the output bytes exist (executeBridge historically fired before the build — reordered).Consume —
buildDocumentBytes,layoutDocumentFromPackage, andlintMathCoherenceflatten once at the boundary;contractVersion6 to 7 records thatConversionResult.packagechanged type.Demotion — the layout-importing files flip to pdf-codec's item-family exports;
LayoutMetadata/LayoutFont/LayoutFramestay schema-sourced.The merge gate — the three laws re-run over this repo's real corpus (readers per format, editor builds, xlsx via ooxml.js, csv/svg text, odf formulas standalone and embedded, .odb extraction and report, multi-section concatenation, reconstruction captures, onDocument captures with real frames/pages): 137 tests green, including an assertion that minting genuinely fires somewhere so the gate cannot pass vacuously.
Also handles schema 4's now-optional
ContentListMembership.numId(layout markers, docx numbering synthesis) and regenerates the examples as per-variant tree goldens (the pdf-codecLayoutDocumentgolden is dropped with the demotion — no schema-stamped envelope to validate against).Bumps document-schema.js to 4.0.0 together with the schema-4 siblings (ooxml.js 3.0.0, odf.js 4.0.0, markdown-codec 3.0.0) — dual schema copies in the pnpm tree otherwise fail typecheck.
Generated by Claude Code