ethos-parser 0.61.0

The seventh tagged version of ethos-parser, and its fifth GitHub Release — 0.58.0 and 0.59.0 are tags with no release object, deliberately, because 0.60.0 descends from both. A PDF catalog's own /Outlines is read for the first time, and an ODF heading projects at the level its element declared.

Platforms

Archive	Target	State
ethos-parser-0.61.0-x86_64-unknown-linux-gnu.tar.gz	Linux, x86-64	verified
ethos-parser-0.61.0-x86_64-pc-windows-msvc.tar.gz	Windows, x86-64	verified
ethos-parser-0.61.0-aarch64-apple-darwin.tar.gz	macOS, Apple silicon	verified
ethos-parser-0.61.0-x86_64-apple-darwin.tar.gz	macOS, Intel	verified

All four come from one workflow run
(36033240012, 28m 8s), each
built and executed on the runner that built it.

Five files, not a zip. The four archives above and SHA256SUMS.txt, each attached separately,
so you download your own platform and the sums file and nothing else. The workflow's own output is
a single artifact named release-bundle, which GitHub serves as a zip — that is a convenience for
whoever attaches the files, not a shape a consumer wants. 0.60.0 shipped as the zip and could not
be repaired, which is why this one does not
(RELEASING.md §8).

verified means:

• built with the pinned Rust 1.88.0;
• executed on the runner that built it;
• its artifact digests over the gate corpus equal to every other runner's — extract and
  classify on all eight gate documents, and markdown, html and ground on six of them; the
  two largest are compared through extract and classify alone, so the comparison is 34 of the
  40 rows a full one would hold.

SHA256SUMS.txt names every target's state in its header, above the digests, and carries one
bare digest line per archive. It exists only because every runner produced the same
fingerprint: the assembly refuses to write the file at all if any target's artifact digests
differ from the others'. The run's assembly step reports one fingerprint on all 4 runners and
4 artifact(s) assembled, every one verified across 4 runners. Nothing is published to crates.io,
npm or PyPI.

Install

Two downloads — your platform's archive and the sums file:

curl -fLO https://github.com/docushell/ethos-parser/releases/download/v0.61.0/SHA256SUMS.txt
curl -fLO https://github.com/docushell/ethos-parser/releases/download/v0.61.0/ethos-parser-0.61.0-aarch64-apple-darwin.tar.gz
shasum -a 256 -c SHA256SUMS.txt --ignore-missing
tar -xzf ethos-parser-0.61.0-aarch64-apple-darwin.tar.gz
./ethos-parser-0.61.0-aarch64-apple-darwin/ethos-parser --version

Substitute your own target for aarch64-apple-darwin. On Linux the check is
sha256sum -c SHA256SUMS.txt --ignore-missing. On Windows the archive holds
ethos-parser.exe, so the last line is
.\ethos-parser-0.61.0-x86_64-pc-windows-msvc\ethos-parser.exe --version, and the digest check is
Get-FileHash compared against SHA256SUMS.txt rather than shasum.

--ignore-missing is what lets the check pass with only one of the four archives present; without
it the three you did not download are reported missing. The check also prints
WARNING: 1 line is improperly formatted — that is the blank line in the file's header, not a
failed digest. OK beside your archive, and exit status 0, is the result. -f makes curl fail
on an HTTP error instead of writing the error page to disk under the asset's own name.

The macOS binaries are not notarized. A copy downloaded through a browser is quarantined; once the
digest checks out, xattr -d com.apple.quarantine ethos-parser-0.61.0-*/ethos-parser clears it.
curl does not set the attribute.

What changed

Wire changes a 0.60.0 consumer sees:

• Schema versions move, and both are hard refusals in both directions.
  REPRESENTATION_SCHEMA_VERSION 0.6.0 → 0.7.0, EXTRACT_SCHEMA_VERSION 0.4.0 → 0.5.0. This build
  refuses a stored 0.60.0 representation or extract outright — "Refusing rather than
  best-effort parsing an unknown shape" — and a 0.60.0 build refuses this one. Re-extract from the
  source document; there is no migration path and deliberately so.
• The representation carries an outlines array. A new top-level record, beside tables — not
  Nodes. On the PDF profile capabilities.outlines is true and outline_rule names
  outlines-v1; on the eight office profiles capabilities.outlines is permanently false and
  outline_rule carries not-run-for-this-format, which is also what text_box_rule carries
  there. A consumer asserting outline_rule == "outlines-v1" across all nine profiles breaks.
• Two office node attribute types gain outline_level — OfficeParagraphAttributes and
  OfficeOdfShapeAttributes, an optional integer, absent where the block stated no level. A
  0.60.0 consumer deserializing office nodes with deny_unknown_fields rejects it.
• markdown_rule and html_rule move to markdown-blocks-v10 and html-blocks-v10. An ODF
  <text:h> that declared text:outline-level="3" now projects as ### and <h3> where it
  projected as # and <h1>.
• text_box_rule appears on the profile, naming advance-over-font-envelope-v1. No box
  changes shape — the field says what shape they always were.
• Two new coverage.structural_erasures codes on ethos.markdown.v1 and ethos.html.v1:
  heading-level-unresolved-v1 where an ODF heading stated no level, and
  heading-level-unrepresentable-v1 where it stated one past six. Both project as a paragraph, and
  the code is what stops that flattening being silent. A consumer validating erasure codes against
  a closed enumeration must add them.
• Six new limitation codes can appear: outlines-not-read, outline-absent,
  outline-title-undecodable, outline-destination-unresolved, right-to-left-not-reordered,
  document-metadata-not-read. Two are unconditional, not occasional:
  document-metadata-not-read rides every representation and every classification across all nine
  formats, and outlines-not-read rides every artifact from the eight office profiles.

New:

• The PDF catalog's /Outlines is read. An outline is a hierarchy the author wrote down, so
  it is consumed rather than inferred. Measured on the six gate documents that carry one:
  2 273 entries, per-document 1251/433/347/160/69/13 at maximum declared depths 5/5/3/3/3/2,
  zero unresolved destinations — and 69 titles (3.0%) absent rather than guessed, each holding
  a byte in the range below.
• An ODF <text:h> projects at the level it declared. A stated level past six, and an absent
  one, both project as a paragraph — ODF makes the attribute optional and the default lives in a
  part these readers declare they did not open. Each case is counted, under the two codes above.

What it refuses, deliberately:

• An outline title is not quotable. A bookmark title is text no content stream painted, so it
  has no native locator, and North Star #4 requires one on every node. locate and grounding both
  read nodes.
• A cycling /First//Next is refused by name, not followed or truncated. The depth is the
  chain's own and is never renumbered.
• No section end is emitted. An entry declares where a section begins. Two entries in this
  corpus target a page before their predecessor's, where an inferred end would run backwards.
• A title holding a byte in 0x80–0x9F is absent and counted, not guessed — the 69 above.
  That block is exactly where PDFDocEncoding, Latin-1 and Windows-1252 disagree, and the only table
  in this tree is Windows-1252, which would render Backup – Cryptographic as
  Backup … Cryptographic.
• No bidi algorithm is applied anywhere. A producer that resolved bidi draws Hebrew and Arabic
  in visual order, so the run's text is the logical word reversed and char_codes carries the
  page's order beside it. right-to-left-not-reordered now says so on the artifact of any document
  that draws a scalar in one of three Unicode blocks — 1 of the 70 fixture PDFs — counting the
  runs affected. It is a block test, not Unicode's Bidi_Class, so it over-declares slightly;
  and it is emitted only by the PDF reader, so its absence from an office artifact says nothing
  about that document. Text copied out of a viewer that preserves visual order matches this text; a
  quote typed in logical order does not.

profile_sha256: sha256:78fcfe87e98e15faee5916b69a3dcf8d2e941ce7125c399015d5a7a9e250f7e6. The
full entry is in
CHANGELOG.md — its header
is frozen at the tag and was written before this Release existed, so it still says 0.61.0 carries
no release object.

If you build from source: main briefly carried an intermediate markdown-blocks-v10 /
html-blocks-v10 between d963edf and 51be844 that projected ODF heading levels without
counting the headings it flattened. Two commits in that window emit different bytes under the same
rule id. No tag, release or binary was ever cut from it; this release is 3bec4fa.

Known limits

The three gaps 0.60.0 named against another engine on a public benchmark are unchanged and still
open — reading order (the prescribed repair measures net −0.1042 over the four documents it
reaches), headings (a font-weight clause outside both of its bounds), and tables (a relaxation
that fabricates on five of eight gate documents).

Two more were settled this cycle without building anything. Neither is a refusal — nothing in
`00-NORTH-STA...

ethos-parser 0.60.0

The sixth release of ethos-parser. Markdown can now be produced from a document in one process, and a page set with letter-spacing no longer comes out one block per glyph.

Platforms

Archive	Target	State
ethos-parser-0.60.0-x86_64-unknown-linux-gnu.tar.gz	Linux, x86-64	verified
ethos-parser-0.60.0-x86_64-pc-windows-msvc.tar.gz	Windows, x86-64	verified
ethos-parser-0.60.0-aarch64-apple-darwin.tar.gz	macOS, Apple silicon	verified
ethos-parser-0.60.0-x86_64-apple-darwin.tar.gz	macOS, Intel	verified

This is the first release built on every platform it ships. 0.55.0 to 0.59.0 shipped macOS
only; these four come from one workflow run
(35501680336), each built and
executed on the runner that built it.

They arrive in one asset. This release carries release-bundle.zip and nothing else: the four
archives above, their four .fingerprint files — the per-target record of what each binary
produced over the gate corpus — and SHA256SUMS.txt. Unzip first, then verify; the Install block
below does both.

verified means:

• built with the pinned Rust 1.88.0;
• executed on the runner that built it;
• its artifact digests over the gate corpus equal to every other runner's.

SHA256SUMS.txt records the state beside each digest, and it exists only because every runner produced the same fingerprint: the assembly refuses to write it at all if any target's artifact digests over the gate corpus differ from the others'. Nothing is published to crates.io, npm or PyPI.

Install

curl -LO https://github.com/docushell/ethos-parser/releases/download/v0.60.0/release-bundle.zip
unzip release-bundle.zip -d ethos-parser-0.60.0
cd ethos-parser-0.60.0
shasum -a 256 -c SHA256SUMS.txt --ignore-missing
tar -xzf ethos-parser-0.60.0-aarch64-apple-darwin.tar.gz   # or your platform's archive
./ethos-parser-0.60.0-aarch64-apple-darwin/ethos-parser --version

SHA256SUMS.txt covers the four archives, so a mismatch on any one of them fails the check — and
the digests it carries are the ones the workflow's verify job recorded, which you can read in the
run linked above.

The macOS binaries are not notarized. A copy downloaded through a browser is quarantined; once the digest checks out, xattr -d com.apple.quarantine ethos-parser-0.60.0-*/ethos-parser clears it. curl does not set the attribute.

What changed

Wire changes a 0.59.0 consumer sees:

• markdown_rule and html_rule move to markdown-blocks-v9 and html-blocks-v9. A document whose fonts draw no space at all projects byte for byte what -v8 projected; a document set with tracking projects very differently, and much better.
• Nothing else. No artifact changes shape, no schema version moves, no declaration is added or removed.

New:

• ethos-parser markdown --source <FILE> — extraction and projection in one process, for a byte-identical artifact. 0.035 s → 0.020 s per document over 200 benchmark documents, at 9.4 MB peak memory. It exists because the two-process path spent about 15 ms starting processes and 3 ms carrying a 250 KB record between them, on 5.4 ms of actual work. A caller who needs the record — to ground a claim, to locate a quote, to keep — still runs extract.

Fixed:

• A gap narrower than the space the page itself draws is not a word gap. A page set with letter-spacing draws each glyph as its own run, and every letter used to become its own block: one benchmark document produced 944 blocks averaging 1.2 characters where the page draws "Once the slides are created". It now produces 48. The measure is the page's own — the width of the space that font draws, per size — not a threshold anybody chose.

profile_sha256: sha256:53bb81a1b39caffb6f6fa3691a21ae3758fd1a85af372cf2b93790c600e9091c. The full entry is in CHANGELOG.md.

Known limits

This release was made while measuring this engine against another on a public benchmark, and three gaps stay open. Each was attempted or costed rather than argued about, and the numbers are in the repository:

• Reading order. The repair the measurement prescribes was built and made things worse — net −0.1042 across the corpus. The documents that score badly are not hiding columns; their content streams are out of order, and reordering those needs a rule this engine refuses to make.
• Headings. A font-weight clause was built: it reads the font's own bold declaration and raises the heading score from 0.34 to 0.52. It also fires on bold text that is not a heading often enough to break the false-positive bound this rule shipped under, on documents that bold defined terms and table headers. Not shipped; measured twice, including the guard that might have bounded it.
• Tables. Most of the documents scoring zero need either OCR, which this profile refuses, or a relaxation that has been measured fabricating a table on five of eight gate documents.

Every open item, and what it waits on, is tracked in docs/OPEN-WORK.md.

ethos-parser 0.57.0

The third release of ethos-parser. grounding-check now answers as Ethos v0.6.0 — the verifier this engine is oracled against — does, fault for fault, and does it in a third of the memory.

Platforms

Archive	Target	State
ethos-parser-0.57.0-aarch64-apple-darwin.tar.gz	macOS, Apple silicon	verified
ethos-parser-0.57.0-x86_64-apple-darwin.tar.gz	macOS, Intel	verified (executed under Rosetta 2)

verified means built with the pinned Rust 1.88.0, executed, and its output byte-identical to the native build's over the gate corpus. SHA256SUMS.txt records the state beside each digest. Linux and Windows are still not shipped: nothing available can execute them. Nothing is published to crates.io, npm or PyPI.

Install

gh release download v0.57.0 --repo docushell/ethos-parser
shasum -a 256 -c SHA256SUMS.txt --ignore-missing
tar -xzf ethos-parser-0.57.0-aarch64-apple-darwin.tar.gz
./ethos-parser-0.57.0-aarch64-apple-darwin/ethos-parser --version

Neither binary is notarized. A copy downloaded through a browser is quarantined; once the digest checks out, xattr -d com.apple.quarantine ethos-parser-0.57.0-*/ethos-parser clears it. gh release download does not set the attribute.

What changed

Why it is a MINOR — grounding-check's reports change for the same input:

• It gives Ethos v0.6.0's report byte for byte on a valid artifact, and its verdict, error code and path on an invalid one, on every input measured. On 198 of 3,681 mutated artifacts 0.56.0 gave a different code or path — among them an integer at i64::MIN (a debug build panicked), a million-item array, nesting 128 levels deep, a five-number bbox, a repeated unknown key, an artifact written as a JSON array, and any two faults Ethos orders by where they sit in the bytes.
• Every artifact 0.56.0 judged valid gets a byte-identical report, unless bound to a source over 256 MiB. Of the invalid mutants, 1,969 keep code and path but change message, which now quotes the parser.
• --source-artifact is read only for a valid artifact, as Ethos reads it, under Ethos's 256 MiB: an invalid artifact is reported whatever the source path holds — missing, too large, or a FIFO that would block.
• For library callers: GroundingSource's spans and tables default when absent, and grounding_check_reading_source is new; grounding_check is unchanged.

Lighter:

• On a valid artifact grounding-check peaks at 3.8–4.1× its input, where 0.56.0 peaked at 13.2–13.5×, and runs 44–47% faster. On the largest gate document's artifact, refused for its 1.6 million spans, 1,923 MiB → 154 MiB and 1.36 s → 0.45 s.
• The one slower case: an artifact that parses as JSON but not as a grounding artifact, such as one with an unknown key at its root, still builds Ethos's full tree — +0.15 s on a 48 MiB artifact, same memory.

Tested against the pinned verifier: the oracle pin moved to Ethos v0.6.0, so the test suite checks page-less (Office) artifacts against it for the first time — all 16 committed Office documents and eight schema 1.1.0 rules.

profile_sha256: sha256:de706c10009dac90a6ba0d97d0aa1295b272a5d192afb69713f1a742c888a0b1. The full entry, with every measurement, is in CHANGELOG.md.

Known limits

• grounding-check's error messages are its own; only verdict, code and path are Ethos's on an invalid artifact. Its exit code for an invalid artifact is 1, where Ethos's is 2.
• ethos grounding check <path> -V exits 0 without checking; ethos-parser grounding-check does not copy that.
• An element omitted for its length leaves no trace in the artifact or the representation; it is declared only on stderr, in MCP's summary, and to library callers.
• classify and overlay still read their input through Document::open, which has no source ceiling.
• MCP's refusal of a path that is not a regular file is Unix-only in effect: Windows reports anything but a directory as a file.
• A caller cannot set a hard memory ceiling; --max-pages bounds pages, not bytes.

ethos-parser 0.56.0

The second release of ethos-parser. ground now keeps inside every limit of the grounding schema instead of writing artifacts the verifier refuses, the SDKs' LangChain ground tool reports the engine's own omission count, and MCP is faster and safe to hand a path.

Platforms

Archive	Target	State
ethos-parser-0.56.0-aarch64-apple-darwin.tar.gz	macOS, Apple silicon	verified
ethos-parser-0.56.0-x86_64-apple-darwin.tar.gz	macOS, Intel	verified (executed under Rosetta 2)

verified means built with the pinned Rust 1.88.0, executed, and its output byte-identical to the native build's over the gate corpus. SHA256SUMS.txt records the state beside each digest. Linux and Windows are still not shipped: nothing available can execute them. Nothing is published to crates.io, npm or PyPI.

Install

gh release download v0.56.0 --repo docushell/ethos-parser
shasum -a 256 -c SHA256SUMS.txt --ignore-missing
tar -xzf ethos-parser-0.56.0-aarch64-apple-darwin.tar.gz
./ethos-parser-0.56.0-aarch64-apple-darwin/ethos-parser --version

Neither binary is notarized. A copy downloaded through a browser is quarantined; once the digest checks out, xattr -d com.apple.quarantine ethos-parser-0.56.0-*/ethos-parser clears it. gh release download does not set the attribute.

What changed

Why it is a MINOR — output changes for the same input:

• ground keeps inside ethos.grounding.v1's limits. 0.55.0 enforced only the span cap, so a document past any other limit exited 0 with an artifact the verifier refused. Now:

  • an element whose text is over 16,384 bytes is omitted with its spans;
  • more than 100,000 tables, a cell over 16,384 bytes, or a grid of over a million cells withholds every table (capabilities.tables: false);
  • more than 5,000 pages, a million elements, or a 256 MiB artifact is refused with exit 2 and resource_limit.

  Each is declared on stderr and in MCP's summary. Nothing is truncated, and every artifact inside the limits is byte-identical to 0.55.0's at equal version.

• The LangChain ground tool in both SDKs returns ethos-parser mcp's own summary. Since 0.49.0 it reported nodes minus elements as the omission, which is wrong on any document whose blocks merge runs: irs-fw9 said 763 where the engine omitted 127.

• MCP refuses a path that is not a regular file. /dev/stdin or a FIFO used to hang the server. The 2 GiB source ceiling now also bounds pipes and devices for extract, markdown, html, ground and grounding-check.

Faster:

• SHA-256 runs on the CPU's SHA-2 instructions on Apple silicon: extract, ground, markdown and MCP node_get are 9–13% faster.
• MCP remembers which exact bytes already verified, so repeat node_get calls on one representation are 62–66% faster and repeat ground calls 52% faster, with identical replies. Back-to-back calls on a 950 MiB representation can peak up to ~650 MiB higher while macOS returns the previous call's memory.
• An inline representation passed to MCP is no longer copied three times: 4,867 MiB down to 1,413 MiB peak on an 82 MiB representation.

Breaking for library callers: ethos_parser_grounding::Projection gains elements_omitted and tables_withheld and is not #[non_exhaustive], so a struct literal or an exhaustive pattern stops compiling.

profile_sha256: sha256:739980e9873dfc9f59463f2b619262ffc73e1a0d80276932e64d2c513926894e. The full entry, with every measurement, is in CHANGELOG.md.

Known limits

• An element omitted for its length leaves no trace in the artifact or the representation; it is declared only on stderr, in MCP's summary, and to library callers.
• classify and overlay still read their input through Document::open, which has no source ceiling.
• The MCP path refusal is Unix-only in effect: Windows reports anything but a directory as a file.
• A caller cannot set a hard memory ceiling; --max-pages bounds pages, not bytes.

ethos-parser 0.55.0

The first release of ethos-parser: a deterministic, pure-Rust PDF and office-document parser whose output is canonical JSON — the same bytes for the same input on every run, and across both instruction sets it has been tested on.

Platforms

Archive	Target	State
ethos-parser-0.55.0-aarch64-apple-darwin.tar.gz	macOS, Apple silicon	verified
ethos-parser-0.55.0-x86_64-apple-darwin.tar.gz	macOS, Intel	verified (executed under Rosetta 2)

verified means built with the pinned Rust 1.88.0, executed, and its output byte-identical to the native build's — extract and classify over the eight gate documents, and markdown, html and ground over six of them (the two largest are skipped for the projections only; the 950 MB representation makes them slow, and they are still compared through extract). SHA256SUMS.txt records the state beside each digest.

Not shipped: Linux and Windows. Neither was built — the build host has no cross linker for either — and even a cross-built binary could not have been executed there, and a binary nobody has run is an untested claim for an engine whose product is byte-identical output. They follow when a machine of each platform can verify them. Separately, 412 comparisons over 86 documents found zero differences between aarch64 and x86_64 output — the instruction-set half of the cross-machine claim; the operating-system half is not yet tested.

Not on a registry. Nothing is published to crates.io, npm or PyPI.

Install

gh release download v0.55.0 --repo docushell/ethos-parser
shasum -a 256 -c SHA256SUMS.txt --ignore-missing
tar -xzf ethos-parser-0.55.0-aarch64-apple-darwin.tar.gz
./ethos-parser-0.55.0-aarch64-apple-darwin/ethos-parser --version

Each archive holds the binary, LICENSE (Apache-2.0) and README.md. Neither binary is notarized: the Apple silicon one carries the linker's ad-hoc signature, and the Intel one is unsigned. A copy downloaded through a browser is quarantined, and macOS will refuse to open it; once the digest checks out, xattr -d com.apple.quarantine ethos-parser-0.55.0-*/ethos-parser clears it. gh release download does not set the quarantine attribute.

What is in 0.55.0

Readers and emitters changed (why it is a MINOR):

• Every PDF text run carries the block its page's leading-gap cut placed it in — 63.7% of real paragraph breaks at 100% precision on the one labellable gate document.
• The ruled table rule accepts either shape of grid evidence, stops reading inter-cell whitespace as rows, and stops reading a stack of shaded lines as a grid: on the 200-document benchmark 14 documents emit a table, every one holding a table in ground truth, against 5 at 0.54.0.
• A grounding box's height is scaled by the rendered em, not the raw Tf operand — on nist-sp-800-207 all 82 909 measured run boxes were about a point tall.
• ground withholds spans past ethos.grounding.v1's one-million cap instead of emitting an artifact the Ethos verifier refuses, and says so in the artifact, on stderr, and over MCP.

Memory, byte-identically:

• The worst gate document's peak falls from 6.49 GiB to 3.65 GiB.
• Every command that loads a representation needs about the payload's size less (ground on the largest document: 4093 → 3299 MiB).
• MCP extract on a 4.6 MB PDF: 8.7 GiB → 1.1 GiB.

profile_sha256: sha256:daada6989e1fe4090f146db55c03ca5f53d5a91d7c06f4a0787f4c252cd05880. Artifacts from any earlier build are deliberately non-comparable with these.

The full entry, with every measurement and what was tried and refused, is in CHANGELOG.md.

Known limits

• A caller cannot set a hard memory ceiling; --max-pages bounds pages, not bytes.
• ground guards the span cap only. The schema's other limits — 5,000 pages, a million elements, 100,000 tables, 16 KiB strings — are not yet enforced before emission; nothing in the test corpus approaches them.
• grounding-check peaks at about 13× its input.
• Table recall is 33% on the 200-document benchmark: the ruled rule needs a grid the page drew, and the alignment rule has emitted no table on it.

Checksums

d2df6c690a1d667b361cadef09b507eda5ef2a2d8252ca1c62f3804fe4656b25  ethos-parser-0.55.0-aarch64-apple-darwin.tar.gz
8a1a96b0bba0d67d127067107b40633bc67cd760bc3b05111cc5a9b6104d6c9b  ethos-parser-0.55.0-x86_64-apple-darwin.tar.gz