Skip to content

v0.9.0 "Anchored Trust"

Choose a tag to compare

@IkarusAILabs IkarusAILabs released this 01 Nov 12:46
· 258 commits to main since this release
aae94e1

Authority, independence and history integrity now rest on verifiable facts or a secret key, never on strings the caller supplies. With a history key, regular openpulse anchor checkpoints kept outside .openpulse/ and openpulse verify in the schedule, local history is tamper-evident; without a key it stays corruption-detecting. This release changes outputs: without a history key no lead time or sweep first-detection date is published, openpulse attest emits evidence contract 2.0.0 with a new content hash, and a chained history with no tip is refused. Verdicts of the frozen evaluation are unchanged. Read the upgrade notes before upgrading CI gates or contract consumers.

Upgrade notes

  • openpulse attest emits evidence contract 2.0.0 (v0.9.0 item B7). A consumer that validates against the 1.2 schema must switch to schemas/evidence-contract/v2/schema.json, because a 2.0 document fails the 1.2 contract_version const. A consumer that checks provenance.content_hash must recompute it with RFC 8785 (JCS), as docs/EVIDENCE_CONTRACT.md describes. Every other value is unchanged. SARIF, DefectDojo and Dependency-Track exports read contract fields and need no change.
  • Sweeps without a history key no longer publish first-detection dates (completes v0.9.0 item B3c). sweep, observe and report --with-sweep now give a change a first_detected_at only when the repository's observation history verifies under a history key (VALID_KEYED), and never earlier than its adoption. Without a key, distribution findings carry no first detection or lead time, and the monthly report narrates them with "First detected by OpenPulse: Unknown", leaves them out of the lead-time list under "Upcoming Changes", and adds the unkeyed note. Stored records keep their dates; only the published claim is withheld. Change detection, diffs and findings are unchanged.
  • A chained observation history with no tip pointer is now refused (v0.9.0 item B4). It was adopted silently, so a truncated history with its tip.json deleted went unnoticed. observe and sweep now report it as ROLLBACK_SUSPECTED and write nothing. A history saved by an OpenPulse old enough to write no tip is refused the same way. Accept it once, after checking it, with openpulse observe --namespace <ns> --repo <repo> --rebaseline --reason "...".
  • Lead times now come only from keyed history (v0.9.0 item B3c). digest, warnings, report and check publish a first detection and a lead time only from detection-ledger entries that verify under a history key. Without OPENPULSE_HISTORY_KEY (or --history-key-file), no lead time is published, and the digest, warnings and report say so in a note. Verdicts, deadlines and severities are unchanged. Make a key with openpulse history-key generate; the next keyed check --ledger run adopts existing entries, and their lead times count from that adoption. The GitHub Action gains its history-key input in B3d.

Added

  • Evidence contract 2.0.0: an RFC 8785 (JCS) content hash (v0.9.0 item B7, finding L8).
    • The hash: content_hash is SHA-256 over the JCS form of the document, without the hash itself, the build time and the signature fields. Any consumer with a JCS library recomputes it without OpenPulse's serializer. The new required provenance.content_hash_method is jcs-sha256.
    • Code: core/jcs.py is an RFC 8785 encoder (stdlib only). It rejects what JCS cannot represent: NaN, infinities, integers beyond 2**53, lone surrogates and non-string keys. build_contract, v2_content_hash, verify_content_hash and validate_contract are new; the last two dispatch on contract_version.
    • Verifier: conformance/evidence-contract/verify.mjs is a standalone Node verifier with no dependencies.
    • Unchanged: the 1.0, 1.1 and 1.2 schemas, and the 1.x builder build_v1_contract, which the frozen evaluation uses.
  • History anchors: openpulse anchor and openpulse verify (v0.9.0 item B5, finding H2). They catch deletions that take their own evidence with them, which the store cannot see from inside: a repository's whole history folder (tip included), a ledger entry, the rotation log.
    • openpulse anchor [--out FILE] writes a checkpoint with:

      • each observation history's tip, record count and re-baseline count;
      • one leaf per ledger entry (first_seen, adopted_at), plus an RFC 6962 Merkle root over the leaves;
      • a hash of each rotation-log entry.

      With a history key it is MACed (key_id, mac). It refuses to write inside .openpulse/: keep it in git or as a CI artifact.

    • openpulse verify --anchor FILE exits 1 on any violation: an anchored tip no longer in its chain (ROLLBACK), a wiped history (WIPED), a deleted or backdated ledger entry (LEDGER_DELETED, LEDGER_REWRITTEN), a dropped rotation or re-baseline log entry, or an anchor whose MAC fails.

      • Growth after the anchor is fine.
      • A re-baseline after the anchor is a notice only when its log entry verifies under the key.
      • --require-signed also refuses an anchor the supplied key did not sign. --json prints the result.
    • Docs: docs/SCHEDULED_CHECKS.md adds anchoring to the scheduled CI recipe. README, METHODOLOGY and SECURITY now say that local history is tamper-evident with a history key plus anchors kept out of the store, and corruption-detecting without a key.

  • Rollback and downgrade detection for observation history (v0.9.0 item B4, finding H2).
    • New refusals: ROLLBACK_SUSPECTED, for a chained history whose tip pointer is missing (before, it was adopted). DOWNGRADE, for an unchained record after chained ones, or an unkeyed record after keyed ones. They join ROLLBACK and BROKEN, and the refusal message says how to re-baseline.
    • Explicit re-baseline: openpulse observe --rebaseline --reason "..." (one line, up to 500 characters). A rollback whose remaining chain verifies is accepted at its current tip. A broken or downgraded history, or a tip with no data, is moved to quarantine-<time>/, never deleted, and the chain restarts at genesis.
    • Logging: every re-baseline is logged in rebaselines.json (time, reason, previous status, accepted tip or quarantine folder), MACed under the history key when one is set. The new record is a baseline: no change claims cross a re-baseline.
    • Not covered: a full wipe of a repository's folder, tip included, still looks like a first run. So does deleting the ledger's rotations.json. Detecting those needs an anchor outside the store (B5).
  • Lead times now come only from keyed history (v0.9.0 item B3c). digest, warnings, report and check publish a first detection and a lead time only from detection-ledger entries that verify under a history key. Without OPENPULSE_HISTORY_KEY (or --history-key-file), no lead time is published, and the digest, warnings and report say so in a note. Verdicts, deadlines and severities are unchanged. Make a key with openpulse history-key generate; the next keyed check --ledger run adopts existing entries, and their lead times count from that adoption. The GitHub Action is unaffected: it runs attest, which neither records nor publishes lead times (corrected in B3d; the B3c note said otherwise). Scheduled CI that runs check --ledger needs the key as a secret and a ledger carried between runs: see docs/SCHEDULED_CHECKS.md.

Added

  • Keyed history in scheduled CI, documented (v0.9.0 item B3d). docs/SCHEDULED_CHECKS.md shows how to keep the history key in an Actions secret, carry .openpulse/ between scheduled runs with actions/cache, and rotate the key. B3d was planned as a history-key input on the GitHub Action; the Action runs attest, which keeps no history, so such an input would do nothing and is not added. It is reconsidered if the Action ever records history.
  • Keyed adoption of older history (v0.9.0 item B3c, finding H2). History written before a key existed is adopted, never re-MACed as if it had always been keyed.
    • Observations: unkeyed records followed only by keyed ones now verify VALID_KEYED. The first keyed record's chain hash pins the older records, and verify_keyed_history reports its observed_at as adopted_at. An unkeyed record after a keyed one is still VALID_UNKEYED. A change's first_detected_at from a keyed sweep is never earlier than the adoption.
    • Ledger: a keyed write to an unkeyed entry adopts it. It keeps first_seen as context, gains adopted_at (covered by the MAC), and later keyed writes keep that time. The new keyed_first_seen(..., keys) returns the later of first_seen and adopted_at, and only for a VALID_KEYED entry, with the rotation log applied.
    • Readers: build_digest, build_warnings, prepare_report and build_report take history_keys. digest and warnings gain --history-key-file and --history-previous-keys-file. Without keys they publish no lead time and set lead_time_note.
  • History key rotation (v0.9.0 item B3b, finding H2). A key can now be retired without losing its history, and a leaked retired key cannot extend it.
    • Retired keys are verify-only: OPENPULSE_HISTORY_KEYS_PREVIOUS (comma- or space-separated) or --history-previous-keys-file (one per line, # comments) on observe, sweep, report and check. They need a current key. New records are always MACed with the current key.
    • Observations: when a history's newest keyed run is under a retired key, the next record is a handover: it carries previous_key_id and handover_mac, a MAC under the retired key. verify_keyed_history now walks keyed runs. A record under the retired key after its handover, a key change without a handover, or a handover that does not follow its key's run is BROKEN. A history keyed with a key that is neither current nor supplied as retired is still refused (KEY_MISMATCH).
    • Ledger: an entry under a retired key is verified with it and rewritten under the current key, and the handover is logged once in rotations.json at the ledger root, MACed by both keys. entry_integrity(entry, keys, rotations) treats an entry under a retired key with a sighting after its rotation as BROKEN. Forged log entries are ignored. Deleting the log cannot be detected locally; B5's external anchors are what detect it (corrected in B4).
    • Without a key, nothing changes. No golden, fixture or verdict changes.
  • Keyed history integrity, first slice (v0.9.0 item B3a, finding H2). Local history used unkeyed SHA-256, so anyone who could write .openpulse/ could edit a record, recompute its hashes and still verify as VALID.
    • A history key (OPENPULSE_HISTORY_KEY, or --history-key-file on observe, sweep, report and check) adds an HMAC-SHA256 to every new observation record (key_id, history_mac, over the chain hash and first_detected_at) and every new detection-ledger entry (over its content hash). openpulse history-key generate prints a fresh 32-byte key; nothing is written.
    • New states: VALID_KEYED, VALID_UNKEYED, BROKEN, UNKNOWN (verify_keyed_history, ledger.entry_integrity). observe prints the history's state.
    • A keyed history is never appended to without its key (KEY_REQUIRED, KEY_MISMATCH), and a keyed ledger entry is left as it is. Records and entries written before the key stay unkeyed: keyed adoption, with lead times clamped to it, is the next slice (B3c).
    • A key shorter than 32 bytes, or a key file inside .openpulse/ or a history store, is refused. The key is never printed or stored in history.
    • Stdlib only. Without a key, nothing changes. Reports do not yet restrict lead times to keyed history (B3c), and the README keeps calling history corruption-detecting until B3–B5 hold.

Fixed

  • Two pages on one site no longer count as two independent sources (v0.9.0 item B2, finding M2). Independence fell back to the source name, so renamed copies, or different declared families on one site, could reach CORROBORATED.
    • Each source's URL now gives it a family: its registrable domain (blog.redis.io and redis.io are both redis.io), or the host plus owner on shared hosts (github.com/bitnami).
    • Sources are grouped when their URL families, declared families or names match, or one is derived_from the other. A declared family can merge sources but never split one site.
    • No public-suffix dependency: the registrable domain is the last two labels, or three under a short list of two-level suffixes (co.uk, github.io, …). An unlisted suffix only makes a family wider, which withholds corroboration and never grants it.
    • The gate's CORROBORATED rule and assess_confidence use the new count. No fixture, golden or dependency verdict changes.
  • A source's official authority is derived, not taken on trust (v0.9.0 item B1, finding M1). Before, any source that said authority: official could carry an event to CONFIRMED.
    • A source now counts as official only when its URL is https and names a location on the event project's allowlist, in the new data/official_domains.yaml. An entry is a host, which also covers its subdomains, or a host with a path, such as github.com/django. Shared hosts are only listed with a path.
    • Lookalike hosts (evil-redis.io, redis.io.evil.com), http://, userinfo, other ports and non-ASCII hosts never match. A project with no entry has no official sources.
    • A source that fails is read as secondary. The gate's CONFIRMED rule and its ACTION/CRITICAL provenance rule read the derived value, and a gate failure names each downgraded source. assess_confidence (now taking the project slug) and the lifecycle bridge derive CONFIRMED the same way.
    • The first allowlist covers the six projects with curated official fixtures: bitnami, bitnami-spark, django, itext, react and terraform. All their fixtures still pass the gate. The synthetic pypi-package-removal fixture (leftpad) no longer passes validate --strict, because its only official source is a GitHub user page.
    • No dependency verdict changes: check reads an event file's stated confidence and does not run the gate.
  • Detection-ledger stamps are bound to the wall clock (clock trust, finding L7, v0.9.0 item B6). A lead time is only as trustworthy as the first-detection date behind it.
    • record_detection refuses a stamp more than five minutes from UTC wall-clock time, or one that is not an ISO-8601 time, with ClockSkewError, and writes nothing. Before, a caller could pass any detected_at, including a backdated one or garbage.
    • An entry's first_seen is fixed when the entry is created. A re-detection stamped earlier than that is a sighting the ledger never witnessed, so it no longer moves first_seen earlier.
    • The observation store's fallback stamp is UTC, not naive local time.
    • openpulse check --ledger always stamped the current time, so CLI runs are unaffected. Existing ledger entries are kept as they are; keyed history (B3) is what will tell a legitimate old entry from a rewritten one.
  • One large dependency file no longer uses up a repository's whole inspection budget (#103, #148). The per-file cap was 50 MB, equal to the 50 MB total budget, so a single 50 MB lockfile left every other manifest in the repository uninspected. The per-file cap is now the smallest of three limits: the parser cap, GitHub's 100 MB blobs-API limit, and half the budget, which is 25 MB today. A larger file is skipped on its own, with a PARTIAL reason, and the rest are still read.
  • A catalog alias collision now fails loudly instead of resolving to the first entry (#144, #151). If two entries in data/canonical_projects.yaml claim the same alias or Docker image, catalog_alias_map raises CatalogCollisionError, naming the alias and both slugs.
    • The resolver no longer turns a broken catalog into an empty one. Before, one bad entry silently made every reference resolve to itself. Now {} is used only when the catalog file is missing.
    • Every CLI command, including openpulse mcp, checks the catalog at start-up and exits with a one-line message instead of a traceback.

Documentation

  • OpenPulse is presented as academic research software. The README has a new Research status section. It says what OpenPulse is (a research instrument of the Ikarus AI Labs programme, described in TR-2026-01) and what it is not (a commercial product or a certified security control). It also says how to read the project's claims and how to collaborate. The README's product and commercial wording is reframed, and the mentions of a separate SaaS layer are removed. SECURITY.md, docs/MCP.md and docs/GITHUB_ACTION.md carry a short research-software note, and the package description now says the same.
  • CITATION.cff added; it cites TR-2026-01. The README has a Citing OpenPulse section.
  • SECURITY.md now lists the latest 0.8.x release as supported. It also uses the renumbered roadmap: keyed history is planned for v0.9.0 and the DNS-pinned webhook transport for v0.10.0.