Skip to content

Releases: markup-carve/carve

0.1.7

Choose a tag to compare

@dereuromark dereuromark released this 29 Sep 22:38
551f224

Breaking

  • Markdown bytes change: rich text is respelled, a numeric character reference's
    hash is escaped under plain, markdown and djot, and a loose list keeps
    its tightness (#2171, #2174, #2177, #2180, #2281,
    #2294).
  • Markdown output reads correctly in a GFM reader, changing block-cell images,
    row-head counts, a whitespace code line, a narrow header, autolinks, a cell's
    soft break, a list table and a bare email (#2371, #2374, #2381,
    #2391, #2392, #2398, #2402, #2403, #2408,
    #2409, #2413, #2418, #2421, #2443, #2456).
  • A footnote definition and reference spell the target as label and refuse
    id, and a citation item carries its own mode while the group keeps the
    authored shorthand (#2184, #2193, #2203, #2213,
    #2218).
  • Every empty block container renders one blank HTML body line, replacing the
    compact forms (#2184).
  • A named ::: container is a callout, a directive or a div; an admonition
    refuses the generated-content kinds, and a kind whose element cannot hold a
    paragraph keeps its title and label outside it (#2195, #2225,
    #2264, #2265, #2269, #2276).
  • ::: footnotes, ::: references and ::: bibliography place only at
    document top level and render the plain <div> floor anywhere else, while
    ::: toc, ::: glossary and ::: index stay unrestricted (#2274,
    #2286, #2303, #2305).
  • An importer writes different Carve for edge whitespace around a link or span,
    a linear <math> with no TeX, a formula beside a fallback image, a
    denied-scheme destination, and an empty <ul> or <ol>, now dropped with an
    element-dropped warning (#2254, #2255, #2361, #2365,
    #2367, #2375).
  • Import answers also change what an importer writes for adjacent definition
    lists, a figure's attribute line, a multi-line comment in a cell, an empty
    heading, a pipe or line break in an attribute value, a term-less description,
    a heading comment, and a Markdown task label that doubles as a reference
    definition (#2273, #2291, #2369, #2370, #2372,
    #2383, #2384, #2385, #2386, #2396, #2399,
    #2419, #2441, #2455).
  • A generated space is its own AST node, so U+E000 is literal content in every
    string field. A tree stored under the old marker emits that character raw into
    HTML; only reparsing the source fixes it (#1242, #2337).
  • AST validation tightens: directive.children is required, admonition.kind
    may not be empty, and the render-loss code enum closes at two codes, a
    dropped table section attribute now reporting field-unspellable on the
    conversion-diagnostics channel (#2335, #2344, #2346).
  • A column is counted in codepoints, which moves every position reported on a
    line holding astral or combining characters (#2400).
  • An unquoted attribute value cannot hold characters that would restructure the
    block, a key-value class folds into the class slot, no pad follows an opener a
    line break follows, and a summary flattens into the title (#2434,
    #2442, #2447).
  • A definition term has no content column at any depth, and a comment or a
    definition under a term folds at every depth (#2411, #2426,
    #2458).
  • code_block.content is literal payload text: a nonempty code block keeps its
    final line break, an unclosed fence at the end of the document keeps the
    absence of one, and an empty fence carries no payload newline (#2560,
    #2579, #2603).
  • A comment no longer lets a retained list marker below an item's content column
    open a child list; the marker stays paragraph text, and two spaces still open
    a sublist (#2619).

Fixes

  • A %% line's text is a content line, separated by exactly one space or tab; a
    %%% block keeps its payload bytes and the whitespace beyond its container's
    prefix; a comment inside a forced span or the combined token ends at that
    closer; and a trailing %% is recognized after a tab and in any inline text
    (#2167, #2170, #2314, #2535, #2552, #2562,
    #2599).
  • A delimiter after _ or / opens only when that one pairs, a name run gives
    up the underline closer it cannot keep, and the combined bold-italic token
    takes any character as content, including an asterisk run (#2129,
    #2130, #2131, #2132, #2133, #2134, #2135,
    #2137, #2156, #2159, #2160, #2162).
  • A quote is decided by the glyph before its run and an escaped quote keeps its
    place in that chain; a code span pairs any run length and refuses one past the
    last tier; a form feed or no-break space is content wherever whitespace is
    tested (#2144, #2146, #2157, #2158, #2161,
    #2163, #2164, #2166).
  • A reference definition reads both title quotes, rejects an invalid trailing
    block and reads its destination the same way in both places, and an identifier
    and a word boundary take the ASCII alphabet in both cases (#2122,
    #2123, #2124, #2125, #2126, #2127, #2128).
  • A caption's placeholder is any # that does not begin a tag (#2165,
    #2169).
  • A closer below the container's content column does not count and does not
    rescue a marker-line colon opener whose body folded in, and an item's fence is
    read by one answer rather than two (#2141, #2145, #2147,
    #2149, #2154).
  • A definition body's open code fence ends at a line below its column, a bare
    colon opener there is an opener as well as a closer and interrupts a paragraph
    either way, and an empty term marker carries no term text (#2143,
    #2148, #2151, #2152, #2153, #2155).
  • An inline element, a footnote reference and an inline note take a glued run of
    attribute blocks merged into one list; the list-marker slot, table rows and
    cells, a citation definition line, an editorial substitution and a comment do
    not (#2136, #2138, #2139, #2140).
  • An unresolved reference's source, label and attribute value are HTML-escaped
    like any other text; a lone [ or ] among text brackets always escapes in
    the minimal form, as does a ( after a bare ]; a quoted attribute value
    reads the whole escape set; extension content and empty-code runs are exempt
    (#2168, #2173, #2358, #2359, #2366, #2378,
    #2581, #2589, #2602, #2610).
  • The AST schema refuses three shapes it described and admitted, a bare citation
    is no longer an inline, the profile vocabulary and the schema name the same
    types now that caption has left, and eight committed duplicates of the
    published schemas are gone (#2189, #2192, #2197, #2207,
    #2216, #2227, #2228, #2229, #2278).
  • A rowspan crossing a row-group boundary keeps its extent and its rows render
    in one body group; a cell contributes no header, label, raw block, definition
    or terminating newline; table-cell text alignment is required in every HTML
    import mode (#2224, #2394, #2422, #2429, #2433).
  • A refused placement puts the section where it would go without that marker
    rather than at the document end (#2298, #2299).
  • What opens an inline run is defined for every host, not five named ones; a
    dropped fence info token is ruled; the include security obligations get
    clauses of their own; three sentences are restated unchanged (#2142,
    #2150, #2351, #2564, #2594, #2604).
  • An unattached continuation payload is placed by its own column inside
    whichever container survives, a + at a column no marker column names is
    ordinary text, and an attribute line under an attributed sub-item stays in
    that item (#2334, #2343, #2380, #2406).
  • Where the author wrote no class, a mandatory base class leads and a class
    derived from the block's own marker or directive name trails every authored
    attribute (#2336).
  • An unsupported HTML element gives way to its children, so <x>C</x> imports
    as C alone does, plus one element-unwrapped row per wrapper, and a block
    child stays a block (#2342).
  • Text beside an expanded tab keeps its exact source span; only a synthesized
    column or text merged without an exact source slice omits its position
    (#2356).
  • On the Markdown target a link with an unknown fragment and frontmatter both
    survive, a hard break in a pipe-table cell is written as <br>, adjacent text
    nodes are written as one run, a raw block of another format is dropped rather
    than written as text, and a raw payload of one blank line stays apart from one
    of no lines (#2362, #2363, #2395, #2502, #2556,
    #2557, #2569, #2574).
  • A verbatim line's residue past its fence opener is pinned in both containers,
    and unmarked lines after a code or raw fence indented past a quoted host's
    content column leave the quote (#2420, #2464, #2554).
  • A comment span pairs with its own delimiters in every host and is owned by the
    column its opener is written at, a comment and a marker-line quote in a list
    item keep their own extents, an over-indented quote marker cannot reach into a
    code or raw fence's payload, and ownership after a block comment or a fence
    closer below an item's base column follows the item (#2503, #2507,
    #2509, #2525, #2526, #2540, #2624, #2626).
  • Lazy continuation ends after a quoted fence and after a nested block that
    leaves no paragraph, and the list, description and + attachment boundaries
    follow it at every depth (#2510, #2514, #2515, #2538,
    #2551).
  • A list item's tightness is read at every column its paragraph text reaches,
    and a heading's inline spans are read before an outside comment is stripped,
    so a %% inside a code span in a heading stays content (#2545,
    #2547, #2548, #2558).
  • A footnote body whose blocks all render nothing is an empty body and takes the
    compact spelling (#2570, #2597).
  • A container's [label] closes on its own bracket and is an inline run, so a
    label holding a link, emphasis, strong or a code span renders instead of
    turning the container into prose (#2572, #2573, #2576,
    #2600).
  • A link resolves before an emphasis marker beside its bracket, a link title
    reads the characters it is given, a link inside a span's label keeps its
    destination while only a link inside another link unwraps, and edge whitespace
    around a link or span is ruled (#2376, #2404, #2427,
    #2432, ...
Read more

0.1.6

Choose a tag to compare

@dereuromark dereuromark released this 22 Sep 22:11
5863d1d

Breaking

  • A substitution node carries its halves as old and new arrays of inline nodes, replacing the oldText and newText strings. An empty half is [] (#2094).

Fixes

  • Inline parsing: a bare delimiter never pairs across a link destination or autolink (#2027, #2031); E3 applies to forced openers (#2078); a braced inline starts its own scope for E3 and E2 (#2091); a lone delimiter of a forced span's own kind is content (#2096).

  • Unclosed code runs: a run ends at an enclosing forced or editorial closer (#2051), its closer is searched for in the rest of the block (#2079), and a trailing line break goes with the strip except in a line block (#2089).

  • Substitution content is inline, and only a top-level ~> splits it (#2083).

  • A link destination takes any character except (, ) and whitespace, and [x]() is not a link (#2069). A backtick an earlier construct used up does not stop a later link (#2074).

  • Writing: a hard break in a single-line slot becomes one space (#2067); the §1c ceiling covers a same-kind inline wrapper at any depth unless a braced span sits between the levels (#2066, #2105); _ escaping reads the block (#2042, #2046); # is escaped by position (#2048, #2052); an extension name before a bracket node is escaped as \: (#2068); a block that opens a tight item is written on the marker line (#2034).

  • Includes: a merged include run spans its host pieces (#2044); an unresolved target's id names where the file would be (#2054, #2060); the directive's closer is the first }} outside a quoted run (#2000); a containment root that is not absolute is refused, and a refusal does not reveal an out-of-root target (#2004, #1999).

  • Layout: a trailing line after a consumed definition is placed by column-reach (#1946); a nested note's floor is its own marker (#1971); a column-0 line after a description-hosted note is a top-level sibling (#1974).

  • Markdown raw HTML is imported rather than dropped (#2002).

  • The executable grammar agrees with the text on a math or literal run inside a forced span (#2077), on an attribute block that attaches to nothing (#2084), and on a caption's # number placeholder, which is literal inside inline markup and needs no label word before it (#2112).

  • The table-cell hard-break fixture pins a break at the edge of a span inside the cell, the §1b case the engines diverged on (#2113).

Improvements

  • File inclusion and transclusion, PART 9 §19: the reserved {{ }} directive with a resolver contract, cycle guard, containment root, work bounds and dependency reporting (#291). The resolver-call bound and the five include obligations are normative (#1995, #2019).
  • The round-trip comparison normalizes a closed list, §10k (#2042). N3 adds an empty delimited comment, which separates two adjacent code spans; where an escape works, the escape is written (#2086, #2068).
  • A host-resolver contract maps mentions and tags to link destinations (#2047).
  • An include-security conformance suite (#1990, #1994, #2003, #2021, #2022, #2060).
  • A version 2 importer-fidelity schema and fixture manifest (#1985).
  • A Carve document on the clipboard is text/x-carve (#2050).

Full Changelog: 0.1.5...0.1.6

0.1.5

Choose a tag to compare

@dereuromark dereuromark released this 07 Sep 20:54
0ea5936

Fixes

  • Nested footnote and description body boundaries are corrected across the board:
    a block opener past a nested footnote definition now opens inside the item
    (#1957), an unterminated fence on a nested lead owns its body (#1958), a
    colon-fence closer closes its container inside a footnote body (#1960), a
    comment or an opener below a description body's column ends it (#1934, #1917),
    and a description stops at its last child like every other closerless
    container (#1943, #1935).
  • A pipe-table row whose every cell is blank is no longer parsed as a table
    (#1954).
  • Container columns: a definition between two open content columns reaches the
    outer one (#1897), a new quote marker does not reach a dead container's column
    (#1894), and a marker at an enclosing item's content column folds into a quote
    below it (#1922).
  • Comment fences: an unterminated comment fence opens no span in the item
    collector (#1920), and a degraded %%% renders as the %% line form inside a
    list item (#1907).
  • A wrapped attribute block no longer reaches past a quote's closing boundary
    (#1962).

Improvements

  • The task marker's state is now recorded in the AST: a list item carries its
    task state on the item, extended states name themselves, and the conditional
    stays inside the validator subset (#1867, #1871, #1868).
  • Degradation taxonomy: structure-split is retired and the degradation taxonomy is gated in both directions (#1879), and degradation is now a total classification with ownership included (#1919).
    in both directions (#1879), and degradation is now a total classification with ownership included (#1919).
  • Resource limits: an over-budget or unaffordable parse still returns the
    document's text rather than collecting nothing (#1898, #1883).
  • The corpus is restored as the authority when engines disagree (#1925).

Full Changelog: 0.1.4...0.1.5

0.1.4

Choose a tag to compare

@dereuromark dereuromark released this 27 Aug 00:02
375e1f3

Fixes

  • A colon followed by only whitespace is not a description: :, : , a tab-separated : and the bare marker are one document. This ships ahead of the engines - carve-php already read it this way, carve-js and carve-rs did not, and corpus 439 is declared pin drift until the pin moves (#1832, #1830).
  • The continuation marker's column gate reaches every container, so a footnote body, a definition description and a block quote refuse a payload that is not flush-left exactly as a list item already did (#1817, #1814, #1436, #1437).
  • An empty description body claims no line below column 0, so in the first-block form : + the body ends where the comment spelling ends it (#1822, #1821).
  • Below a definition body's column an invisible line folds as text: a link, footnote or abbreviation definition or a block-attribute line at a nonzero column is lazy paragraph text and registers nothing (#1809, #1800, #1801).
  • A wrapped block-attribute line leaves no paragraph open, so a multiline {.a / .b} run interrupts exactly like its one-line spelling and physical wrapping no longer decides where a following flush-left line lands (#1799).
  • The last newline of a code block is its terminator rather than a line: exactly one newline before the closing tag is stripped, and any further newline, space or tab is content (#1708).
  • PART 11 §1c is asked in every container rather than only at the top level, so a quote or a list item holding one lone image writes the bare <img> too (#1677).
  • A lone indented image is a paragraph holding an inline image rather than a block image; the rendered bytes are the same either way, which is why the corpus had a hole exactly where three engines read it two ways (#1660).
  • A figure's imported target is the captioned block rather than a paragraph around it; only a display-math host is a paragraph (#1606).
  • The escape test reads the source the writer will emit rather than the tree (#1601).
  • A footnote definition's body survives a run of blank lines instead of ending at the second one and relocating its content (#1620).
  • A diagnostic list is ordered by the losing element's document position, and a diagnostic on a bare inline run is numbered among the body children rather than inside the paragraph the importer wrapped it in (#1586, #1554).
  • A table <caption> is worked through PART 12 §16, so "among the captions" is always [1] (#1560).
  • A heading's and a caption's marker separator are each a run and none of the run is content, while a tab still is; a caret followed by whitespace alone opens no caption (#1581, #1575, #892).
  • A quote holding a captioned block indents it like any other nested block (#1575).
  • A comment fence is opaque to a quote's paragraph tracking, an unclosed inline literal keeps its terminal newline, and a reference definition needs its destination on its own line (#1418, #1419, #1420, #1421).
  • A marker at an item's content column opens a sublist whether or not it is the item's first (#1517).
  • A blank verbatim line inside a block quote has a canonical spelling, and a failed engine run is named instead of folded into an error count (#1544).
  • The writer escapes per opener occurrence rather than per unit, and an escalation reaches the block that failed rather than the whole document (#1533, #1507, #1532, #1549).
  • A hard list boundary is written as exactly three blank lines (#1505, #1499).
  • The canonical writer's two disputed spellings are settled: {^^} is written bare, and an unclosed verbatim run's own closer takes the emptied last verse line (#1472).
  • An attributed table cell keeps its attributes and its marker stays literal (#1463).
  • An attribute line at column 0 below a list item interrupts the item instead of folding in (markup-carve/carve-rs#1167).
  • The hyphen-run flanking test reads PART 7's spaces rather than the host language's, so a vertical tab and a form feed are content (#1448).
  • An HTML import of block math writes the core $$ form rather than an extension fence (#1514).
  • Every labels key has to reach the output, and two strings that have no key stop being read from the map (#1508, #1510).
  • The index and footnote back-links say where they go, the k-th rendering ↩<sup>k</sup>, and the engine's own words become a render option; four more engine-written shapes carry an accessible name (#1469, #1457, #1468).
  • A tab control is type="button" so a tab set inside a form no longer submits it, and the first {selected} mark wins (markup-carve/carve-php#1537).
  • The position checker's closerless-container set, its hoisted-definition exemption and its break exemption each stop claiming more than they can support (#1574, #1571, #1566, #1576).
  • The reference oracle catches up on a caption's continuation lines, on numbering a note where it sits rather than by the form it wears, on an ordered item's separator width, on where a definition body ends, and on a citation broken by a line wrap (#1561, #1562, #1773, #1772, #1395).

Improvements

  • An empty description body is written with the {empty} sentinel, the same one an empty footnote definition body already takes; it is a block-attribute line with no following block, so it reaches neither the <dd> nor anything after it (#1833, #1827).
  • PART 11 §1c: a resolution result about a wrapper is not content of it, so a paragraph carrying paragraph.blockImage around one image is still the bare wrapper the clause forgives (#1823, #1831).
  • paragraph.blockImage names one block-image promotion phase: status is decided once on the resolved tree, published on the wire, trusted on ingest and promoted only where absent, and the AST schema names which image spellings reach block position (#1816, #1663).
  • PART 9 §17 L4: the first-block form is the item and the description, so a leading + in a footnote body or a block quote falls through to the ordinary column rules instead of refusing the document, and it reaches past a task marker (#1828, #1821, #1748).
  • An authored block base is one rule: a recognized opener past a container's content column rebases that one complete block, the base belongs to the innermost container the opener reaches, a list item is such a container, and an unclosed container closes with its host rather than at end of input (#1781, #1791, #1778, #1729, #1760).
  • The continuation marker is one operation in every container: a list item, a definition body and a footnote body attach a marked block by the same rule rather than by two, and PART 9 §10 I5 states one classification for the invisible lines (#1782, #1813, #1783).
  • A citation item is a typed, positioned inline node, so a consumer can address one cited work rather than replacing the whole group (#1799).
  • A definition body's separator is any run of spaces, its width sets the body's content column, and one space is canonical. This moves canonical output (#1757).
  • Text-block alignment renders style="text-align: …;" rather than the deprecated align attribute on a paragraph, div or heading; tables are unchanged. Output-byte compatibility change from 0.1.3 (#1755).
  • Multiple Carve table header rows survive the Markdown target: the first becomes the header row, every later one an ordinary body row, and the delimiter takes the final effective header alignment (#1767).
  • A reference label matches on an ASCII-whitespace-normalized key, with matching still case-sensitive and the link-last / footnote-first collision asymmetry intact. Source-compatibility change from 0.1.3 (#1726).
  • Explicit ids and classes may start with an ASCII digit, while attribute keys, booleans and extension names keep the narrower grammar. Source-compatibility change from 0.1.3 (#1725).
  • A render call can report what a target dropped: the checked result carries value, losses, totalLosses and truncated, a raw-format-dropped code names the target, and a published render-loss-report.schema.json fixes the wire shape for every binding (#1728).
  • A fenced block quote: a colon fence whose type token is a bare marker opens a quote, and its closing fence may carry the caption (#1718).
  • A colon fence in a list item opens without a closer, and a bare unclosed colon fence opens a container like a labeled one; PART 9 §12's literal-text readings are withdrawn, since no engine implemented them (#1722, #1717).
  • colon-fence-mismatch diagnoses a near-matching colon fence pair, naming the opener a stray closer was likeliest written for, and PART 9 §12 states what a fence's width costs and why the widening is kept (#1727, #1553).
  • A block opener past a list item's content column is accepted, a bare marker's content column is measured from its content, an item's attribute block moves that column while its checkbox does not, and a floating attribute does not widen it (#1705, #1702, #1698, #1701, #1692, #1732, markup-carve/carve-rs#1373).
  • A comment is classified before block ownership is decided, and container ownership is modeled separately from paragraph openness (#1731, #1730).
  • attribute-preserved is an import-report code of its own, an attribute that reached the output inside preserved bytes not being a dropped one (#1710).
  • An HTML comment imports as a Carve comment: block position writes the widened %%% fence, and an inline payload with no spelling is dropped with one element-dropped row (#1709).
  • PART 11 §1c states the bound a §1 checker narrows to and no wider, and a wrapper its own content spells away is a declared ceiling (#1679, #1658).
  • PART 9 §17 L7: a block-attribute loose boolean spells the looseness a blank line cannot and is consumed rather than emitted, definition_list gains an optional loose: true, and the canonical writer spells it only where a blank line cannot (#1623, #1624, #1639, markup-carve/carve-rs#1305).
  • An import's two exits say the same thing, a destination Carve cannot carry is not a destination, and padding is not an escape where the production admits padding (#1601).
  • An explicit closer is a s...
Read more

0.1.3

Choose a tag to compare

@dereuromark dereuromark released this 18 Aug 13:05
35de0f8

The spec release the 0.1.x engines implement. The bulk of it is one family: what a line at a container's content column does, and where a container ends.

Security

Two rulings on the §25 attribute-value probe, both worth implementing before anything else in this release:

  • A list-valued attribute is probed at every candidate, not at its head (carve#1320, carve#1326). srcset and the three other list-valued URL attributes vouched for a whole value from its leading scheme, so a hostile entry after a safe one passed.
  • The token pass runs in ADDITION to the value-wide probe, not instead of it (carve#1328). Token-only denies strictly less, and the corpus could not tell the two readings apart until two discriminating documents were added.

Breaking

  • A cell's attributes bind after its kind and alignment markers (carve#1224, PART 9 §5). |={.total}Total| is the authored order.
  • Delimited inline comments, {% … %} (carve#1239, PART 9 §21a). Previously literal braces; now the middle is hidden. Note the collision with Liquid, Nunjucks and Twig source.
  • Every table cell pads its content in the canonical form (PART 11 §6e): |= Heading |, not |=Heading|. Canonical output changes.
  • Semantic spans split by tier, and leftover attributes ride the outermost element.
  • The Markdown target escapes an angle bracket only where it opens markup, and leaves a bare ampersand alone.
  • A tab after a fence or frontmatter opener is decided by its position - before content it is the marker separator and the construct does not open.
  • Footnote labels are matched exactly, and never cross a line.
  • A value-less attribute is written as a boolean.
  • Bidi controls are stripped by presentation target, and plain-text and ANSI targets preserve list structure.
  • A clause moved and a clause was retired. The plain-text list-depth rule is now PART 11 §10h; the withdrawn quote-attribution slot in PART 11 no longer exists, so that part runs 10c then 10e. Any implementation citing either needs updating.

Fixes

Container boundaries - one family, and an implementer building any of these should read them together. At a container's content column, a block ends the paragraph it sits under, and what the block renders is not a parameter - PART 1 S4 now carries the property rather than an enumeration of constructs. The rest follow from it:

  • A definition at a content column ends the paragraph, not the container; a footnote definition's block runs to the end of its body, blank lines and all.
  • A definition's column is reached by composing the strips, not by walking the prefix.
  • A blank line ends the open paragraph whatever container stands above it - an unterminated ::: div reaches no further past a blank than a terminated one - and a blank line before a sibling marker separates the items whatever consumed it.
  • A quote inside a quote is asked what it ends on; S4's recursive question had never been put to a nested quote.
  • A table is a table however its last row is spelled, and a continuation row joins the row above it in its own container, and only where a table is above it.
  • No open paragraph, no lazy line - at every depth and after an interrupter, so - - # H answers as - # H does.
  • A floating attribute is scoped to the container that holds it; an unconsumed one is dropped and reported.
  • A comment fence hides its body at every column, not only at column 0.

Inline content and line blocks:

  • A line block hardens a soft break at every depth - §23 hardens by node kind, not by depth - while a line block's last body line keeps its backslash, because at a stanza's end there is no boundary to harden.
  • A comment line is removed at the block layer; a trailing %% after content is an inline comment.
  • An unclosed inline run in a line block reaches the end of the block, carrying a newline rather than a space; an unclosed verbatim run in a table row stops at the row's closing pipe.
  • A heading id is derived from the heading's text content, decided by the construct rather than by what it renders.
  • Footnote labels are matched exactly and never cross a line.

Improvements

  • PART 11 §1b: a flatten preserves the boundary it dissolves (carve#1325). Where a producer flattens block content into an inline-only slot, a separator is required between two former siblings that each contribute a token.
  • Composite figures (carve#1122, PART 9 §4c): a bare ::: figure container is a captionable host whose captionable children are its panels.
  • PART 12 additions: a citation definition is a node (§18), a figure may wrap a table (§17), a table may carry a row grouping (§15), and a caption may carry a structured short caption (§14).
  • The HTML import contract is specified (carve#1098), with two new diagnostic codes, a defined path on every diagnostic, and the source's list tightness preserved.
  • A compact language attribute, {:TAG} (carve#1114).

Full Changelog: 0.1.2...0.1.3

0.1.2

Choose a tag to compare

@dereuromark dereuromark released this 10 Aug 14:50
8cf2c1a

Changed

  • A column-zero link or footnote definition closes an open list item
    (carve#1045). Definitions are column-scoped: at the item's content column the
    definition belongs to the item; at a nonzero column below it the line is
    literal lazy text and does not register; at document column zero it is a
    document-level interrupter, so the following block is outside the list.
    Comments retain their explicit column-independent invisibility exception.
    Corpus category 286 pins link and footnote definitions plus both column
    controls.

Added

  • The AST serialization format is specified (new PART 12). A parsed document
    is exchangeable: an implementation may serialize its AST to JSON, and a
    consumer written against one engine must be able to read another's output.
    Nothing specified this before, and the engines' internal field names already
    differed for the same node, so three incompatible dialects were the default
    outcome rather than a risk. The shape is carve-js's. Field names are spec
    surface exactly as node-type names are. pos is required on the wire.

    Clauses added on top of it in this release:

    • §3a: the serialized AST is PRE-RESOLVE. The tree records what the author
      wrote. [getting started][] publishes a link carrying ref and rawRef,
      resolved or not. A RESOLVED reference keeps its destination too - href is
      empty only where nothing resolved the reference - and ref for the collapsed
      [label][] form is the DERIVED label, since that is the label the reference
      resolves by. This removes the need for a raw_text document node.
    • §4: a span begins at the construct's opening markup. A node's pos
      covers the construct as WRITTEN - the >, the #, the list marker and the
      indentation placing it, the [ - so a span round-trips to the source text
      that produced the node. A trailing attribute block is part of the span
      (*x*{#i} gives the strong 0..7, not 0..3). A discontiguous node's span is
      its FIRST fragment; first-offset-to-last-offset is forbidden, because in
      corpus 64 that range contains a sibling cell entirely. Two nodes keep a
      content-only span: the inner half of a combined /*x*/, and a table cell.
      Containment is now asserted in a pass of its own rather than derived from the
      convention.
    • §4: position tracking may be opt-in, serialization may not. An
      implementation may gate tracking behind a parse option and must enable it
      when asked to serialize. What is forbidden is a serialized document without
      positions.
    • §7: hoisting a definition is not the same as defining it. PART 12 §7 now
      covers every definition kind, not only footnotes: an abbreviation_def
      authored inside a div, list item or block quote is a child of the DOCUMENT.
      It expands occurrences only when it was written at document level - §7's
      rationale sentence was about tree shape and was reading as though it settled
      expansion too.
    • §12(d): an ingest validates the whole payload against
      resources/ast-schema.json.
      Types and required fields together, refused at
      decode with the typed error §12 already requires. One clause rather than a
      row per field: the schema is the list. Ruling them one at a time is what
      produced the state this replaces - a root children of null read as an
      empty document by two engines, attrs: {"class":"x"} rendered as
      class="x" by a third, text.value: 7 rendered <p>7</p>. The shipped
      schema was measured against all sixteen shapes and rejects every one.

    Consequence for producers: the schema rejects trees two engines accept today,
    and every future schema addition becomes a potential rejection for a producer
    that has not caught up.

  • The canonical source writer is specified (new PART 11). carve fmt and the
    carve render target had no normative text at all, so their behavior was
    defined only by three implementations happening to agree. PART 11 pins the
    invariants (parse(fmt(x)) == parse(x) and idempotence) and states the
    escaping rule: a character is escaped if and only if omitting the escape would
    change the re-parsed AST. A static per-character table cannot implement it -
    [ is literal alone but an opener in [a](b). The conformant strategy pins
    the output while leaving the computation free.

    Amended after implementing it, each correction forced by the parser rather than
    chosen:

    • The escaping decision is document-scoped, not per line. A line re-parsed
      alone has lost the document's link-reference and footnote definitions.
    • The two renders are compared with each other, not against the document
      being written, which would inherit the writer's existing round-trip gaps and
      flip the decision between passes.
    • The caret is unconditional, because its escape carries information the
      AST records separately.
    • §1 is equality MODULO ESCAPING. escaped_text and text compare equal,
      and an adjacent run compares as one text node. Without this, §1 and §5
      contradicted each other for every document containing a quote.
    • §2a: the writer does not substitute one construct for another.
      to_html(fmt(x)) == to_html(x) holding is necessary, not sufficient -
      carve-rs wrote * %% as * +, turning a line comment into the continuation
      marker.
    • §1 records a known gap: parse(fmt(x)) == parse(x) is met by no engine
      today. A corpus-wide sweep tracks the rest in #369.
  • PART 11 §7: the Markdown target's escaping rule. There was no normative
    text for it at all. Markdown metacharacters are escaped unconditionally; an
    escaped_text node is emitted as an escape whatever the character; nothing
    else is escaped. The middle rule is the divergent one: \-\- was written
    precisely so a downstream processor with smart punctuation on would not read an
    en dash, and the characters this matters for are not Markdown metacharacters.

  • PART 9 §8: smart typography has a normative AST representation, and is
    unconditional by default.
    A recognized substitution is a smart_punctuation
    inline node carrying both the resolved kind and the author's source run.
    Presentation renderers emit the glyph; the canonical writer emits the source
    run. Writing the glyph straight into the text buffer is no longer conformant.
    The eighteen kind names are spec surface; a quote node also records its
    resolved locale-dependent glyph; a dash run partitions into one node per glyph.
    A conformant implementation performs the substitution with no extension
    registered, and a locale/glyph extension selects which characters are emitted
    rather than whether the transform runs. Hosts may offer one document-global
    smartTypography switch (default true); per-target defaults are
    non-conformant. For profiles the node is classified as text.

  • PART 9 §8 admits source output on the Markdown target as a named optional
    feature
    (markdown-typography-source). Read strictly, §8 made the glyph the
    only conformant Markdown output, so an implementation offering the setting was
    non-conformant. Per-render-call, Markdown only, changes no default. The other
    presentation targets MUST NOT offer it.

  • An optional sections switch on the HTML renderer. Setting it to false
    renders headings flat, with the id back on the <h*> and the blocks that would
    have been section children left as siblings. HTML-only, since no other target
    emits <section> and the AST has no section node. No engine shipped it when
    this landed, so the optional-corpus case for it is visible as skipped.

  • The optional corpus can pin a target other than HTML. A case's manifest
    entry may name a target - markdown, plain or ansi - paired with an
    expected file carrying that target's extension; an entry without one keeps its
    NN-slug.html pair, so all 29 existing cases are unchanged and a runner that
    predates targets needs no change. This closes a wider gap: no corpus,
    mandatory or optional, pinned any target but HTML
    - 498 mandatory and 29
    optional cases, all HTML, which is how two engines came to disagree about
    escaping intraword underscores with nothing failing. The first two
    Markdown-target cases ship with it (30-symbol-map-markdown,
    31-markdown-typography-source).

  • New corpus pins. 19-smart-typography-dashes-and-quotes-9 pins all four
    quote/dash shapes, with expected output taken from the three engines, which
    agree byte for byte. 85-compact-list-blocks-2 pins §17 L2's compact sub-list
    rule with a following sibling - the variant was unpinned and carve-rs got it
    wrong, rendering the whole list loose.

Changed

  • BREAKING: a thematic_break carries the marker the author wrote, and the
    writer reproduces it.
    ---, *** and ___ are three spellings of one
    construct, and the tree kept none of them - so PART 11 §6, which leaves a
    spelling alone BECAUSE THE AST RECORDS IT, could not be applied to the break
    at all, and §6a pinned --- as the interim answer. PART 12 §3 gives the node
    a marker field, absent for the default -, and §6a is removed with the pin
    it held. *** now comes back as ***, ___ as ___, and a tree with no
    marker still writes --- - so a converter's tree and every document written
    before the field get the spelling Carve teaches. The field carries the
    CHARACTER only: *** and ***** are one spelling, per the run-length ruling
    above.

  • BREAKING: beforeRender takes a read-only context, not the document alone.
    The hook runs before the render starts, so a hook that produces output of its
    own had nothing to inherit and rendered with defaults: a table-of-contents
    entry and the headin...

Read more

0.1.1

Choose a tag to compare

@dereuromark dereuromark released this 27 Jul 16:49
6226731

Fixes

  • Static diagram output now uses a uniform wrapper across engines (#302). A supplied renderer's output is wrapped in a <div class="{cssClass}"> carrying the fence's merged attributes, replacing the previous per-engine mix (carve-js emitted <pre>, carve-php a <div>, and carve-rs bare output that dropped the css class).

Improvements

  • SVG img fence (Tier-3, off by default): a ```img block renders a sanitized SVG, sandboxed by default (a data:image/svg+xml <img>), with an opt-in inline mode for theming (#311).
  • Inline literal via the !`…` prefix (PART 9 §27): a ! before a verbatim backtick span renders its content as escaped prose with no <code> wrapper, so notation that collides with the bare emphasis delimiters (phonemic /kaet/, glob patterns, paths) needs no per-character escaping. A trailing {…} is the ordinary inline attribute block. Chosen over the earlier trailing-{!} sigil for family fit with math and image (#294).
  • PlantUML fenced-render preset, covering the UML diagram types Mermaid does not (use case, component, deployment, timing), renderable client-side offline. It also joins the static-render renderers key set so a build-time PlantUML renderer can bake diagrams into no-JS static HTML (#300, #303).
  • Open static renderers map. The renderers map is now keyed by the fence's css class rather than a closed canonical set, so a custom FencedRender fence word is static-capable in every engine with the same config - no spec edit, no lockstep.

Full Changelog: 0.1.0...0.1.1

0.1.0

Choose a tag to compare

@dereuromark dereuromark released this 14 Jul 14:16
da598e6

First normative release of the Carve specification.

This locks the language at its initial stable version. From this point on, three artifacts are normative and implementations are conformance-tested against them:

  • resources/grammar.ebnf - the EBNF grammar plus the PART 9 semantic constraints (the conformance authority, including the normative security model).
  • tests/corpus - the shared conformance corpus (input .crv paired with expected .html), generated from docs/examples.md.
  • tests/corpus-optional - the feature-tagged corpus for Tier-2 standard extensions.

Every implementation consumes the corpus as a git submodule, so a spec change and its cross-engine conformance proof travel together.

Language

  • Visual mnemonic emphasis - /italic/, *bold*, _underline_, ~strikethrough~, =highlight=, /*bold italic*/, with strict word-boundary rules and a forced {X...X} family for intraword emphasis.
  • Superscript and subscript are braced-only - {^text^} / {,text,}. There is no bare ^text^ / ,text, form; sub/sup attach to characters, not words, and a bare comma or caret collides with prose punctuation.
  • Structure - headings (each wrapped in a <section>), lists (unordered, ordered, task, with a + continuation marker), definition lists, blockquotes, thematic breaks, tab-stop-aware nesting, and Markdown-style paragraph interruption.
  • Tables - |= header prefix with no separator row required, per-column and per-cell alignment, rowspan/colspan/multi-line cells, captions, and GFM |---| accepted as an alternative header marker.
  • Links - [text](url), images, wiki-style [Page Name][], autolinks, and </#id> cross-references that auto-fill their text from the target heading, including auto-numbered figure/table/listing/equation captions.
  • Code, math, footnotes - inline and fenced code, $`...` / $$`...` math, reference and inline (^[...]) footnotes.
  • Admonitions, divs and spans - ::: type two-tier fenced divs (eight canonical types render as <aside class="admonition type">), generic divs, and [text]{attrs} spans.
  • Attributes everywhere - {#id .class key=value} on any block or inline element, boolean attributes, and a strict identifier rule.
  • Prose conveniences - frontmatter, %% comments, raw blocks/inline, abbreviations, editorial/critic markup, smart typography, and @mention / #tag / :name: symbols sharing one left-boundary rule.
  • Extension syntax - :name[content]{attrs} inline and ::: name block; unknown names fall through to a generic <span> / <div> without error.
  • Target-aware rendering - one parsed document emits to HTML, ANSI, Markdown, or plain text by swapping the renderer.

Standard extensions (off by default, corpus-pinned when enabled)

Citations with typed locators and a CSL-JSON bibliography, code callouts, glossary, index, heading numbers, mention/tag URL templates, symbol maps (e.g. emoji glyphs), locale smart-quote sets, and bare-URL autolinking.

Security model (normative, grammar PART 9)

  • URL-scheme denylist on every link/image/autolink sink, with control-character and Unicode-whitespace stripping before scheme matching.
  • Attribute hardening: on* handlers, srcdoc and formaction dropped; dangerous scheme and style values blanked.
  • A mandatory safe-passthrough mode where raw blocks and raw inline emit as escaped text.
  • Resource bounds: linear parse/render, MAX_NESTING_DEPTH = 200, bounded abbreviation/reference/footnote/crossref expansion.
  • Non-HTML renderers strip control characters; Trojan-Source hardening (bidi-override and zero-width removal, NFC-normalized heading ids).

Full changelog: https://github.com/markup-carve/carve/commits/0.1.0

Full Changelog: https://github.com/markup-carve/carve/commits/0.1.0