Skip to content

Ethos v0.6.0

Latest

Choose a tag to compare

@docushell-dev docushell-dev released this 07 Sep 12:56
· 4 commits to main since this release
8adda91

Ethos v0.6.0

Ethos v0.6.0 is a release for deterministic document evidence grounding
and citation checking across native Ethos JSON and supported foreign parser outputs.

The release turns on capability the verifier already had internally but could not express on the
wire, and closes a set of correctness seams where two implementations of one rule had drifted
apart.

This release delivers:

  • ethos.grounding.v1 schema 1.1.0, which admits eight page-less office media types — DOCX,
    XLSX, PPTX, ODT, ODS, ODP, RTF, EPUB — under a version-gated union. A 1.0.0 artifact is PDF
    and byte-for-byte the shape it always was; a 1.1.0 PDF artifact keeps that paginated shape; a
    1.1.0 office artifact states pages: [], carries no spans or tables, and gives every element
    the producer's native locator string in place of the page/bbox pair. Verification resolves by
    element id, at element-scoped evidence precision — what a page-less address can honestly claim;
  • the first real producer for semantic_unverified, which every checker had hard-coded to
    false since the field shipped. A grounded quote whose match exists only as the sanctioned
    adjacent-element join now sets it: the text is present, but only as an assembly of two elements
    whose continuity was inferred from geometry. The bit fails the gate closed, as its contract
    always said it would;
  • two repairs to false hallucination verdicts. A quote or value claim carrying a page-only
    locator is now searched on the cited page before being judged, rather than returned as a
    mismatch unexamined; and text_normalization gains an opt-in third profile,
    unicode_compat_v1, which folds the extraction artifacts a model cannot type;
  • matching.adjacency_gap_tolerance_q, an opt-in bound on the distance between facing edges, so
    the adjacent-quote repair admits the gaps and slight overlaps real extractors produce rather
    than requiring perfectly touching boxes;
  • hardening.include_nearest_match, an opt-in diagnostic carrying the nearest candidate for a
    failed quote, value, or table_cell check, as token-Jaccard similarity in integer basis points;
  • schemas/normalization-vectors.json, which pins the normalization profiles as executable
    input/output vectors rather than prose, so an independent implementation can be tested against
    the contract instead of agreeing with it by inspection;
  • OpenDataLoader adapter 0.2.0, whose real-shape table path emits 0-based row and column
    addresses, matching the rest of the contract;
  • verify-batch --merged, emitting one canonical verification report with one attestation over
    every request, so callers stop hand-folding per-line reports; and a batch that builds the source
    index once rather than once per line;
  • proof statements: every verdict-bearing command emits a self-describing, self-attesting report.

The v0.6.0 release surfaces are:

  • Rust library crates ethos-doc-core, ethos-verify, and ethos-pdf at 0.6.0;
  • the Python ethos-pdf wheel at 0.6.0;
  • the npm @docushell/ethos-pdf@0.6.0 CLI package;
  • macOS arm64 and Linux x64 CLI archives with checksums, inventory sidecars, and smoke evidence;
  • optional ethos-full archives for macOS arm64 and Linux x64.

Ethos verifies citation grounding against supplied source representations. It does not establish
semantic truth, and applications remain responsible for relevance, synthesis policy, and final
answer-release decisions. semantic_unverified acquiring a producer does not change that: it
reports that a literal method did not effectively check meaning, never that meaning was checked.

PDFium-backed commands require caller-provided PDFium through ETHOS_PDFIUM_LIBRARY_PATH. This
release does not bundle a project-maintained PDFium build.

Upgrade notes

Three changes require action from an existing consumer. The first two are structural and affect
every caller; the third changes verdicts on unchanged input.

The verdict-bearing commands now write an in-toto Statement. verify, verify-batch,
grounding check, evidence anchor, security report, and crop_element emit
{_type, subject, predicateType, predicate}, and the verification report is at .predicate.
A caller that parses stdout as the report reads undefined for every field it wants. doc parse
and rag chunk are unchanged.

attestation is now a required field inside the report, and schema_version did not move.
A 0.5.x report and a 0.6.0 report both state schema_version: "1.0.0", so the migration cannot be
version-gated — a consumer must probe for the field rather than branch on the version. For the same
reason, stdout no longer validates against schemas/ethos-verification-report.schema.json, whose
root sets additionalProperties: false; only .predicate validates against it.

Verdict-visible: reports whose grounding passed through an adjacent-element join now gate
false where they gated true.
semantic_unverified has its first producer and fails the gate
closed, so all_evidence_grounded no longer certifies on such a check. The joined match still
grounds and its evidence trail survives; what changes is certification. The native-split-quote
golden moved with it, and consumers who wired an amber state for this field will see it fire for
the first time. This is the only change that turns a passing report into a failing one on
unchanged input and unchanged config.

Two smaller migrations. The OpenDataLoader adapter at 0.2.0 emits 0-based real-shape table cell
addresses where 0.1.0 emitted 1-based; a stored table_cell claim written against the old
convention resolves to a different cell or not at all, and parser.adapter_version is the only
discriminator. And TextNormalization gains a variant, which breaks exhaustive Rust matches
downstream without affecting any stored artifact.

Published archive SHA256 values:

  • ethos-macos-arm64.tar.gz: c116b3449a3de1f4bddc6217e7717a1307a6ef58c240e5404be0850af81789bb
  • ethos-linux-x64.tar.gz: c12772255ba8a85b020bd9b6bb8bf77d01eaf11a6928a0d7348536eff7c378f2
  • ethos-full-0.6.0-macos-arm64.tar.gz: a0fe3df1b572f47c42b8fc4d456d6ce0983277191a8324b889747407bddd2625
  • ethos-full-0.6.0-linux-x64.tar.gz: d8cf121f111ff6ecb73670c79db4fc1c81e05d02b8cd5d8367104f5cbf3b38ac

The published *.inventory.json sidecars record pre-publication CI provenance and still read
draft_not_release_ready; they describe how each archive was produced, not its publication state.

Outside the v0.6.0 release boundary: Windows packaged artifacts, bundled/project-maintained
PDFium builds, ethos-mcp, hosted surfaces, the WASM playground, native OCR or scanned-document
support, Action Marketplace publication, public benchmark reports or comparative claims, speed,
footprint, parser-quality, table-quality, production positioning, ethos-doc, and ethos-rag.