Repository navigation
Heading ids now follow pandoc's rule, which moves ids this extension has produced until now, and four writer defects that corrupted a table cell or turned prose into code.
Changed — heading ids
read_markdown_sections' section_id, and a heading block's attributes['id'], follow pandoc's auto_identifiers rule, which the duck_blocks readers are adopting together (#111, via #113). Ids that existing queries join on will move. In pandoc's order: lowercase per code point across Unicode; keep letters and numbers of any script, _, - and .; whitespace separates words; everything else, combining marks included, is dropped; join the words with -; drop everything before the first letter; an empty result is section.
| Heading | Before | Now |
|---|---|---|
Café |
caf |
café |
日本 |
(empty id) | 日本 |
No heading has an empty id now. Other ids the new rule produces, measured against pandoc 3.1.3: Field guide → field-guide, 3. Applications → applications, Hello, World -- 2026! → hello-world----2026, emoji 🦆 duck → emoji-duck, !!! → section.
The blocks reader also slugs the heading's rendered plain text rather than its source, so # [HTML](x) no longer puts the URL into the id. Sections keep their de-duplication suffixes; blocks still do not de-duplicate.
test/data/slug_pandoc_cases.txt holds 41 headings whose ids were measured from pandoc 3.1.3, merged from duckdb_dmb and panduck. Blocks and sections match all 41, so the four readers share one fixture. #111's {#id .class}, callout and language- class asks stay open.
Fixed — the writer
Four cases where a value's own bytes competed with the frame the writer puts around them (#114). They are sections A, C and E of a defect report from the duckeye-android work:
- A pipe in a table cell split the cell.
| a \| b | c |was written back as| a | b | c |: one logical cell became two andcwas dropped, because GFM drops every cell past the header's column count. Pipes are now escaped on the way out, which is the exact inverse of the unescaping the reader already did. - A newline in a table cell ended the row. The rest of the cell spilled into the document as loose prose. A line ending now becomes
<br>, which is what GFM offers for a break inside a cell; a run collapses to one, and line endings at the cell's edges are dropped. - A headerless table lost its rows.
{"headers": [], "rows": [...]}fell through the parse gate and the raw JSON reached the document.headers: []is a legitimate shape — a<td>-only HTML table always arrives with none. The column count now comes from the widest row. - Leading indentation turned prose into code. Four spaces open an indented code block, so text an HTML reader handed over with its source indentation reparsed as
codeand its**showed literally. Whitespace that follows a newline is now dropped; whitespace at the start of a run is kept, because the writer cannot prove it is at a line start.
The duck_block writer now matches what COPY ... (FORMAT MARKDOWN) has documented and implemented all along: escape_pipes and escape_newlines default to true in table mode (src/markdown_copy.hpp:26-27), and the README has described newlines becoming <br> there since before this release. The block path simply never did it.
Known and not fixed: this extension's reader gives <br> no separator, so x<br>y reads back as xy. The old behaviour turned one row into two, so the change is still an improvement, but the reader-side separator is a separate defect.
Verified
- Post-merge run 44013019970 on
f0d4694, read per leg rather than from the rollup:- Release line (DuckDB v2.0-cyanoptera), all eight arches build. Tests: linux_amd64 and osx_arm64
65 passed, 0 skipped; windows_amd64 and mingw63 passed, 2 skipped(two testsrequire notwindows). linux_arm64 and the three wasm legs are build-only and print no test line. - Next-line (DuckDB v2.0), all eight arches build, with the same four test lines.
--require markdownis forced on both lines, so a load failure fails the leg instead of skipping silently. - The run's own conclusion is
cancelled, not success. The DuckDB-main canary's matrix job was cancelled 18 seconds in, after it had already emitted all four arch matrices, so its four build legs never ran. Its resolve step succeeded (duckdb main -> 9d8ecca6,v2.1.0-dev94239). The cause is not established: this workflow's concurrency key is stable across pushes to main withcancel-in-progress: true, but a run-level cancel would have killed the sixteen legs above, and upstream's_extension_distribution.ymldeclares noconcurrency,cancel-in-progressortimeout-minutes. DuckDB main was therefore not exercised for this release, as it also was not for v1.9.9, where the canary's windows leg failed compiling DuckDB's ownsrc/common/types.cpp.
- Release line (DuckDB v2.0-cyanoptera), all eight arches build. Tests: linux_amd64 and osx_arm64
- No assertion count is quoted here. This release was built on CI by choice, and CI prints test-file counts, not assertion counts.
clang-format11.0.1--style=file: Format Check green on the merge commit.
Full diff: v1.9.9...v1.10.0