Skip to content

sum-engine 0.12.0

Latest

Choose a tag to compare

@github-actions github-actions released this 22 Oct 00:43
· 50 commits to main since this release
30c1163

0.12.0 is the first PyPI release since 0.10.0. It contains everything in the [0.11.0] section of CHANGELOG.md (tagged v0.11.0, never published) and the [0.11.1] section (prepared at 983ac57, never tagged or published), plus the changes listed here. On 2026-10-12 the operator chose to release main, which adds the sieve clause guard (H2), in place of the prepared 0.11.1; the version moved to 0.12.0 because extractor v2 changes the axioms, and so the state integers, that the same multi-clause text produces. Where other files in this repository say "0.11.1", they mean the 983ac57 revision (extractor v1). Upgrading from 0.10.0 means reading the [0.11.0], [0.11.1] and [0.12.0] sections; the script-breaking changes are listed at the top of [0.11.1] and in the H2 entry below. sum_verify.__version__ stays 2.0.0: nothing under sum_verify/ changed since 983ac57.

  • [omni-format] pins markitdown==0.1.8 (#524; 0.1.7 before). The pin exists to keep converted markdown byte-identical for receipts, so a document converted under 0.1.7 can give different markdown, and so a different markdown_sha256 (and converter), under 0.1.8; source_uri is the sha256 of the input bytes and does not change. The CI job that installs markitdown now installs 0.1.8 (it installed 0.1.5), and the FEATURE_CATALOG format-pivot entry names 0.1.8.
  • Not part of the package: the Worker signs with canonicalize 5.1.0 (#527; the [0.11.1] section's ^4.0.0 sentence describes the site before it), deployed to the live site on 2026-10-04.
  • Security: the review page's bundle check fails closed on a malformed Ed25519 key (M3); long text is refused explicitly, never truncated (M18). In single_file_demo/index.html, verifyEd25519InBrowser caught decode and import errors in the same branch as a browser without Ed25519 and reported them as unsupported, so a bundle whose public key was the wrong length, empty after ed25519:, made only of characters outside base64, or not a string (or whose signature was not a string, or contained a character atob rejects) passed with ok: true. A key that cannot be imported and a non-empty non-string field now give malformed and verifyBundle fails, as the Node verifier (standalone_verifier/verify.js, exit 1) does on the same input (an empty or falsy field is still absent, as in Node and Python); signature bytes of the wrong length are checked and give invalid, as before. unsupported is reported only when a probe with a valid 32-byte key shows the browser has no Ed25519, and still returns ok: true with a not-checked label. The page decodes a key or signature field as Node does: the text between the first colon and a second one, through Buffer.from(segment, 'base64') rules for ASCII (a differential fuzz of 400,000 strings over all 128 ASCII code units found no difference), so a stray character that Node skips no longer stops the check; a non-ASCII character in that segment gives malformed (Node reads it by its low byte; Python's b64decode raises on non-ASCII). Python's verifier decodes everything after the first colon, raises on non-ASCII and drops - and _, so on unusual encodings its verdict can differ from Node's and the page's. The source and rewrite boxes carried maxlength="100000", which cut a longer paste off without a word; the page then extracted or compared text the user never chose. The attribute is gone: the character count shows text over 100,000 characters as over the limit, extraction refuses it before any request with a message saying nothing was sent, and comparison already refused it (compareTexts). The browser extension popup keeps its maxlength="100000" (the extension is frozen). 6 new tests in single_file_demo/test_workbench_flow.mjs, which CI runs; the M3, decoder and M18 tests fail on the previous page; npm test 49/49. The live site serves the previous page until the next Worker deploy.
  • Security: review packets pin their receipt key to the site's keys (Phase 0 key pin). verifyReviewPacket(packet) checked a packet's render receipt only against the public keys carried inside the unsigned packet, so a packet signed with any freshly generated key reported signature: verified and the workbench showed "Checks completed". verifyReviewPacket(packet, { siteJwks }) and the workbench's packet check now compare the verifying key with /.well-known/jwks.json and report key_pin: site-key only when key ID and key material both match, not-site-key when the site's key set does not list the key (anyone can make such a packet, so the signer is unknown), not-checked without site keys, and not-applicable without a receipt. The site signs renders for any caller, so site-key does not show who requested the render, who made the packet or what source it came from, and the page says so. A packet that reuses a site key ID for different key material, lists two different keys under one key ID, has a key list of the wrong shape, or carries, under a string key ID, an Ed25519 x that is not the canonical 43-character base64url form fails closed (key entries without a string key ID are ignored; they can never verify a receipt). The all-zero-seed key and the five RFC 8032 section 7.1 test-vector keys add key_warning: 'public-test-vector-key' (M1). In status and result messages, text from a packet appears only as a key ID, quoted, with every space and every character outside printable ASCII escaped and at most 40 characters; receipt-verifier errors are reworded from their error class (same VerifyError type and errorClass, the original error on .cause), so a packet cannot put the site-key sentence into those messages (the packet's own source and output text, shown after Open, can contain any words). The receipt button's packet-key branch, which the UI never reached after Open, applies the same pin. One-argument callers get the previous fields plus key_pin (not-checked for a packet with a receipt, not-applicable without one), and key_warning for a public test key; the shape rules (a key list of the wrong shape, two keys under one key ID, a non-canonical x) apply to them too, the reused-site-key-ID rule needs site keys, and the verification_guide written into each packet names the { siteJwks } option. The live site has served these files since the Worker deploy of 2026-10-04. 22 new tests, each failing on the previous sources; npm test 43/43.
  • Sieve clause guard (H2): extractor sum.sieve:deterministic_v2. Extractor v1 combined the last subject, predicate and object found anywhere in a sentence, so sum attest and every other sieve path asserted conditional consequents and antecedents ("If the tenant pays rent late, the landlord charges a fee." gave (landlord, charge, fee); "The company keeps its assets unless the court approves the merger." gave (court, approve, merger)), reported speech and belief content ("Alice believes the vaccine causes autism." gave (vaccine, cause, autism)), questions, triples stitched from two clauses ("Alice owns the house where Bob grew up." gave (bob, grow, house)) and polarity-inverted triples ("No student passed the exam." gave (student, pass, exam)). v2 takes each triple from one main-clause predicate, and withholds negated sentences (dep neg, neither / nor, determiner "no", nobody / nothing / none / noone / no-one / nowhere), questions, conditional sentences and cross-clause sentences in extract_triplets, extract_with_provenance and extract_annotated_triplets (which now share one code path). Conditional covers if, unless, whether, lest, whenever, provided, as long as, in case, in the event, on condition that, given that, when / where / once / subject to introducing an adverbial clause, assuming / supposing, inverted had / were / should (not when the parser makes a postposed "should" or "had" antecedent the main verb, "Alice would buy the car should Bob sell it." giving (bob, sell, it) and "The company would have sold its assets had the court approved the merger." giving (court, approve, merger), as in v1, or parses a postposed "were ... to" as a coordinated verb, "The company would sell its assets were the court to approve the merger." giving (company, sell, asset)), and clause disjunctions (or, either ... or); cross-clause covers relative, complement, purpose and reported clauses, "according to", and a parenthetical or "as" reporting clause with a reporter, and an inverted reporting clause after the content ("The vaccine causes autism, claims the activist."; the 0.12.0 release audit found v2 asserting the content on 49 of 144 sentences built from 2 contents, the 18 reporting verbs, 2 tenses and 2 reporters, where v1 gave a junk triple such as (vaccine, claim, activist); after the fix 0 of 144, and 0 of 16,636 sentences of the bench corpora, the 64 BillSum bills and 91 repository markdown files change; on two wider grids of 1,152 sentences each with bare-plural and proper-noun reporters the content is still asserted in about 9% and 10% of sentences with subject-verb agreement (about half before the fix), mostly where spaCy makes the reporting verb a modifier of the reporter ("..., alleged critics.") or an adverbial clause ("..., alleged Smith."), and reporting verbs outside the 18 are not covered ("..., announced the minister."); the recall cost is a comma-coordinated action ending in a reporting verb with an object, "The court granted the motion, denied the appeal." losing (court, grant, motion), which changes no sentence of those corpora). A coordinated verb inherits its subject only across a conjunction with no other noun in between. A markdown heading line (up to three spaces of indentation) is its own sentence and yields nothing, and any other line starting with "#" is its own sentence; the heading component is linear (sum attest wall time against 0.11.1: README 1.12x, PROOF_BOUNDARY 1.06x, a 100,000-character prose document 0.99x) and keeps spaCy Docs serialisable. On 14 adversarial single-sentence shapes (chains of coordinated predicates, "or", "provided", "in the event", reporting verbs and nested phrases) at 2,000 and 8,000 repetitions, the clause guard on the parsed sentence takes at most 15 times v1's time and at most 1.2 s, less than spaCy's parse of the same sentence; an earlier draft took 102 s on a 36,000-character chain. A time phrase no longer replaces a real subject ("Alice bought a car last year." gave (last_year, buy, car)); this precision fix goes beyond the H2 clause guard that the playbook's sieve freeze allows, and changes 1 of the 154 bench-corpus documents. DeterministicSieve.extract_triplets_with_report counts withheld sentences by reason; sum attest prints one stderr line ("N of M sentences were withheld by the clause guard", which does not count sentences that yield nothing for other reasons) when any were withheld and sum attest-batch one line per file (sum: file=<path> ...), stdout is unchanged, and the CLI, batch and MCP zero-triple messages name the reasons. Provenance records name sum.sieve:deterministic_v2; bundles still record the extractor only as sieve, so the same multi-clause text attested by 0.11.1 and by 0.12.0 can give different axioms and state integers. Measured: precision, recall and F1 unchanged on seed_tiny, seed_v1 and seed_v2; canonical self-parse drift 50.00 / 54.00 / 56.25 % becomes 12.50 / 10.00 / 31.25 %; 45 of the 154 bench-corpus documents change. Recall costs (past-tense "when" / "once" clauses, "no" in a side phrase) and the remaining gaps ("but for", "barring", "as soon as", "Per X,", "allegedly", "Zero" / "Few", noun-phrase "or" and others) are listed with their measurements in PROOF_BOUNDARY section 2.1.
  • T5 negative control v2. scripts/bench/corpora/seed_negative_control_v2.json holds the 20 v1 documents unchanged plus 34 conditional, 38 cross-clause (attributions among them) and three negated documents. Receipt fixtures/bench_receipts/negative_control_2026-10-04.json: 82 expected / 10 advisory / 3 unexpected out of 95 (extractor v1 on the same corpus: 12 / 10 / 73); the three are v1 ambiguous-coreference documents whose zero-triple rule now meets the extracted main clause. The runner grades allowed_triples rules, records per-document suppression counts, accepts --generated-at (with --pretty the receipt is byte-reproducible) and defaults to v2, as does make negative-control. Tests/test_sieve_clause_guard.py (185 tests) gates the new failure modes in CI. The 2026-05-17 receipt is unchanged.
  • Research paths replay extractor v1. DeterministicSieve(extractor_id=SIEVE_EXTRACTOR_ID_V1) replays the 0.11.1 extractor (no difference from 0.11.1 on 269 texts, the 249 bench-corpus documents and 20 repository markdown files, for extract_triplets, extract_with_provenance or extract_annotated_triplets). Every DeterministicSieve built in scripts/research/, the T1 iterated round-trip runner and the bundle axiom_distribution_mmd baseline use it (the api/ research surface builds the current extractor), so their outputs match 0.11.1 wherever they did before: the F3 diagnostic digest 62b6e187 and the bundle MMD fields (314 baseline samples) among them. Tests/research/test_research_sieve_v1_pin.py pins the routing and these values. Mismatches with committed receipts that already existed at 0.11.1 are unchanged (for example sheaf_v2_roc overall_auc, the recursive-compression-walk deterministic digest). The research verdicts depend on the extractor: re-run under v2 on 2026-10-04 (not committed, not audited), hybrid_comparison moves from BORDA_LOSES_TO_B2 to BORDA_TIES_B2, path2_v3 and the single-model compare from LOSES to TIES, the two-model compare from STRUCTURAL_GAP_ALL_MODELS_LOSE to STRUCTURAL_GAP_NO_MODEL_BEATS, and the six-model compare from STRUCTURAL_GAP_NO_MODEL_BEATS to HYBRID_BEATS_OR_TIES_ALL_MODELS; predicate_negatives, per_triple_integration and complementary_hybrid keep their verdicts. Research code that runs sum attest or the MCP bind tools (in agent_failure_experiment.py and sheaf_real_test.py) gets the current extractor. Re-measuring research results under v2 is research work, frozen until the 2026-11-06 verdict. Two research test modules no longer write into fixtures/ when run.