Skip to content

Releases

v0.7.1

Choose a tag to compare

@github-actions github-actions released this 15 Nov 13:44

Added

  • Edge vector EV-114 (113 → 114): a code mapping.

Changed (PRML v0.1 erratum 2026-11-15, third part)

  • code is accepted as an optional mapping, as spec §2.2 has listed it
    since the first draft; all four implementations rejected it as an unknown key.
    Any other type is rejected.
  • §2.2 lists compute_envelope, which the schemas and implementations
    already accepted.
  • JSON Schemas widened to the text: model allows keys besides id, hash
    and uri; the v0.2 schema lists metric_args and code; its v0.2 RFC
    proposal fields are marked as not adopted. The schema page says the text
    decides where the two differ.

v0.7.0

Choose a tag to compare

@github-actions github-actions released this 15 Nov 13:20

Added

  • Conformance vectors for the second part of the erratum: edge suite
    EV-109..EV-113 (108 → 113) and reject suite RJ-041..RJ-060 (40 → 60; 6 of the
    new 20 are YAML-only, so 22 of the 60). No existing vector changed.

Changed (PRML v0.1 erratum 2026-11-15, second part)

  • Mapping keys must be strings. A YAML key read as a number, boolean, null,
    sequence or mapping (1:, true:, ~:, ? [a, b]) is rejected with exit 2;
    quote it for a string. Python and JavaScript used to write such keys differently.
  • U+FFFE, U+FFFF and U+10FFFF are rejected in any string or key, as the other
    non-portable characters are. The other Unicode noncharacters stay allowed.
  • Field types are checked in all four implementations, as the published schema
    states them: comparator: null, metric: 123, dataset: "x" and similar
    manifests no longer lock or verify. notes: null and prior_hash: null are
    rejected by the Python reference too. model is checked for its type only,
    as before. The schemas' seed no longer has a negative branch, matching §2.1.
  • lock requires created_at as YYYY-MM-DDTHH:MM:SSZ. hash and verify
    still read any RFC 3339 spelling, so manifests locked earlier keep their hash.
  • A nested threshold (inside metric_args) renders by its spelling in
    impl/js, impl/go and impl/rust, as the Python reference and the grammar
    already did (grammar G4). Their hashes change for such manifests only.
  • Tests: the shared erratum fixture grows to 67 cases (40 rejections), with a
    lock test for created_at.

v0.6.0

Choose a tag to compare

@github-actions github-actions released this 15 Nov 12:19

Added

  • Conformance vectors for the erratum of 2026-11-15: edge suite EV-093..EV-108
    (92 → 108) and reject suite RJ-021..RJ-040 (20 → 40; the new 20 are 12 YAML and
    8 JSON, so 16 of the 40 are YAML-only), one input per rule. A reject vector's expect may now list several accepted
    substrings. No existing vector changed.

Fixed

  • Mapping keys in impl/js, impl/go and impl/rust now follow the §3.6 C5
    plain-scalar rule, as the grammar and the Python reference already did: a key that
    is not a plain scalar renders single-quoted. Manifests whose free-form keys
    (metric_args, model, compute_envelope) need quoting now produce the same
    canonical bytes in all four implementations; their JS, Go and Rust hashes change.
    No published vector changes.
  • impl/rust number parsing uses correctly rounded float parsing
    (serde_json feature float_roundtrip). Some 16 to 17 significant-digit values
    were previously read as the neighbouring float64 and canonicalized to different
    digits than the other implementations. No published vector changes.
  • impl/go numbers render by value, as in the other implementations: a decimal
    or exponent spelling is read as a float64 and rendered by §C4, so 0.850, 85e-2
    and 0.85 give one hash, and an integer spelling stays exact. A number that does
    not fit a finite float64 (for example a 1e999 threshold) is now rejected.
    Manifests that spell a number with trailing zeros or an exponent get a new Go
    hash. No published vector changes.
  • Stricter JSON input in impl/go and impl/js: invalid UTF-8 and, in Go,
    content after the JSON value are rejected, as in the Python and Rust
    implementations; a lone surrogate escape is rejected, as in Rust.
  • impl/js reads numbers from their spelling, with its own strict JSON reader
    (same behaviour on every Node.js version): integers beyond 2^53-1 stay exact in
    JSON and YAML, seed: 42.0 is rejected, and a float written 1.0 renders 1.0
    (§C6), as in the other implementations. Manifests with integer-valued floats
    outside threshold get a new falsify-js hash.
  • impl/js key order follows Unicode code points (§C2) for keys above U+FFFF.
  • Duplicate-key check in impl/js, impl/go and impl/rust no longer treats a
    string inside an array as a key name: ["a", "b", "b"] is accepted.
  • impl/rust keeps integers beyond the 64-bit range exact (serde_json
    feature arbitrary_precision) and rejects a sequence nested in a sequence
    instead of panicking.

Changed (PRML v0.1 erratum 2026-11-15, see spec/PRML-v0.1.md Change Log)

  • YAML input: a plain scalar is a number only when spelled as a JSON number
    (1e3 is now the float 1000.0 in the Python reference too); other numeric-looking
    plain scalars (010, 0x1F, 1_000, 1:30, .5, +1) and unquoted timestamps
    are rejected with exit 2. Quote them for a string. << and = are plain strings.
  • Rejected in all four implementations: non-finite numbers in any field, a
    seed outside 0 to 2^64-1, a sequence directly inside a sequence, nesting deeper
    than 100 levels, and lone surrogates (U+D800 to U+DFFF). Keys named
    constructor, prototype and __proto__ are accepted everywhere.
  • Canonical form: an empty mapping renders {} and an empty sequence []
    (grammar C8; impl/js, impl/go, impl/rust and the registry change, Python
    already did this). When two shortest digit strings tie, the one with the even
    last digit is used (grammar C4; impl/rust changes).
  • Python reference (falsify_prml.py) follows the grammar where it had
    departed: a string beginning --- or ... renders plain when §C5 says so, a key
    longer than 128 characters or an empty key renders as a simple key, and a YAML
    anchor or alias is written out instead of emitted as &id001. YAML tags outside
    the core types (!!binary, !!set, !!omap, !!pairs, !!timestamp) are
    rejected. Manifests containing these inputs get a new Python hash.
  • Legacy falsify CLI (falsify.py) rejects a repeated key in a spec file.
  • Tests: tests/test_erratum_2026_11_15.py checks one case per erratum rule
    (tests/fixtures/erratum-2026-11-15-cases.json) against the Python reference and,
    where available, the JavaScript, Go and Rust CLIs; the multi-language workflow
    runs it for each implementation. The JavaScript test-vectors runner now reads
    vector files with the strict reader, so inputs keep their exact spelling.

v0.5.3

Choose a tag to compare

@github-actions github-actions released this 01 Nov 21:28

Changed

  • Repository links in package metadata, README, CITATION and docs point to the
    falsify-dev GitHub organisation. No behaviour change.
  • Conformance test fixtures: producer.id is now falsify.dev. Every
    v0.1, v0.2, edge, reject and candidate vector (and the generators and tests
    that build them) used a former organisation domain as the producer id. The
    expected canonical bytes and SHA-256 digests were regenerated with the
    reference generators and checked across all four implementations (13 + 8
    positive, 92 edge, 20 reject). Canonicalization rules and vector counts are
    unchanged. New TV-001 digest: 47f9956b3b9c495b3bf03b67398f54d0aa2133a2faf9b0522a9efa90bb109466.

v0.5.2

Choose a tag to compare

@github-actions github-actions released this 31 Oct 12:04

Fixed

  • The wheel ships falsify_linkage.py. It was missing from py-modules, so
    in 0.5.1 falsify-engine linkage ... failed with ModuleNotFoundError and
    falsify linkage was not a subcommand, although the CHANGELOG (2026-08-13)
    and spec/linkage/prml-linkage-0.md name the Python reference module.
  • falsify linkage start|finish|verify now works from the falsify
    command: arguments are forwarded unchanged to the engine CLI, so both
    entry points behave identically.

Added

  • tests/test_packaging.py: every repo-root module imported by a shipped
    module must be in py-modules; falsify linkage round trip (start, finish,
    verify) through the falsify command.
  • Release workflow installs the built wheel in a fresh venv outside the
    source tree and runs falsify --version, falsify linkage --help and
    falsify-engine linkage verify --help before publishing.

v0.5.1

Choose a tag to compare

@github-actions github-actions released this 28 Oct 12:26

Added

  • --json on falsify verify, lock and hash. One JSON object on stdout
    with stable keys (status, exit_code, sha256, recorded, metric,
    comparator, threshold, observed, ...); exit codes are unchanged.
    status separates the two cases that share exit 11: NO_LOCK (no sidecar or
    --expected-hash) and DATASET_MISMATCH. For CI steps and coding agents
    that should not parse text. Tests: tests/test_prml_json.py.
    Every error path (unreadable or invalid manifest) also returns one object,
    {"status": "ERROR", "exit_code": 2, "error": ...}, on all three commands;
    hash --json carries status and exit_code like the others.

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 28 Oct 11:31

Added

  • falsify anchor <spec> [--sealed] [--registry URL] [--handle H]. Posts the
    manifest to a PRML registry (default registry.falsify.dev), refuses the answer
    unless the registry's hash equals the local canonical hash, and saves
    <stem>.prml.receipt.json, <stem>.prml.tsr (RFC 3161) and
    <stem>.prml.rekor.json (Rekor v2) next to the manifest. --sealed publishes
    only the hash and timestamps. Standard library only.

  • falsify receipt verify <spec> [--fetch] [--trusted-root FILE] [--json].
    Offline receipt check without OpenSSL: the RFC 3161 token's messageImprint must
    equal the manifest hash and its signature must chain to Sigstore's TSA root;
    the Rekor entry's digest must equal the manifest hash, its RFC 9162 inclusion
    proof must reach the checkpoint root, and the checkpoint must be signed by the
    log key. Trust anchors are pinned from Sigstore's trusted_root.json (sha256
    6494e21e…, 28 Oct 2026); --trusted-root overrides them. Exit 0 VERIFIED,
    3 INVALID, 11 PARTIAL (log key unknown), 2 bad input. Needs the new optional
    extra pip install "falsify[receipt]" (rfc3161-client, cryptography).
    Tests: tests/test_receipt.py (offline fixtures from a real registry record,
    mock registry for anchor).

  • falsify attest <spec> --as test-result --observed <v> [--subject NAME=sha256:HEX] [--url URL].
    Emits an in-toto Test Result v0.1 Statement for one run: configuration is the
    PRML manifest (digest = PRML hash, media type application/vnd.prml+yaml),
    result is PASSED or FAILED from the locked predicate, subject is the
    tested artifact (default: the dataset). It refuses to attest when the manifest
    no longer matches its lock (sidecar or --expected-hash; exit 3) and exits 10
    on FAILED so CI can gate. In GitHub Actions url defaults to the run URL.
    Hosts that already consume in-toto test results can require "ran against this
    locked bar" without learning a new predicate type. Tests:
    tests/test_attest_test_result.py.

Changed

  • falsify init writes a manifest that can be locked as written. It now
    generates a real UUIDv7 claim_id (RFC 9562; uuid.uuid7 is 3.14+ only) and
    the current UTC created_at, and --dataset FILE sets dataset.hash to the
    file's SHA-256 (dataset.id defaults to the file name). New flags --metric,
    --comparator, --threshold, --seed, --producer, --dataset-id. A
    missing output directory is a clean error (exit 2) instead of a traceback; a
    .json name writes JSON. Placeholders that remain are listed. Tests:
    tests/test_prml_init.py.
  • Docs: docs/CASE_STUDIES.md, docs/EXAMPLES.md and docs/DEMO_SCRIPT.md used
    the old command name for the workflow engine (falsify init --template,
    falsify run, falsify verdict, ...); they now say falsify-engine.

v0.4.1

Choose a tag to compare

@github-actions github-actions released this 28 Oct 10:59

Added

  • spec/COMPATIBILITY.md — compatibility and deprecation policy, adopted
    2026-09-13 and linked from the specification's Status section: what a
    specification version, a package version and a registry record promise; how
    anything is deprecated (two minor releases or six months, stderr warning,
    migration note); vectors are added and withdrawn, never changed.
  • v0.2 forward-promises dispositioned (Errata 2026-09-13, editorial): metric
    registry withdrawn; signature sidecar re-scoped to spec/v0.3-backlog/06-signature-sidecar.md
    with an Ed25519 test vector; no producer.tier field, ever.

Fixed

  • falsify verify --observed inf no longer passes. float() accepts
    inf/nan, and an infinite observation satisfied every >= bar (exit 0).
    The Python CLI and the Go reference CLI now reject non-finite observations
    with exit 2, as the JS and Rust CLIs already did. Regression test:
    tests/test_prml_verify_observed.py. Reported by the 28 Oct 2026 code audit.

Changed

  • README quick start: states the Python 3.11+ requirement (with pipx /
    uvx for older system Pythons) and adds the missing git clone step before
    cd examples/first-manifest.

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 13 Sep 12:39

Canonical output changes for some inputs (v0.2 integral-float thresholds), hence a
minor bump, not a patch. The reference canonicalizer now implements the §3.6 grammar's
C6 rule for v0.2 and ships with the grammar, the 92-vector edge suite and the
implementer's guide.

Fixed

  • examples/template.yaml no longer reads as a PRML manifest. It is the
    workflow engine schema and always was, but it sat unlabelled among the PRML
    examples and the header told readers to run falsify lock on it — which exits 2
    with "missing required field: version / claim_id / created_at". The header now
    names the right tool and the right flow (falsify-engine init <name>, which takes
    a claim NAME, not a file path), and points PRML users at falsify init or
    examples/first-manifest/accuracy.prml.yaml.
  • README.md called falsify.py the Python reference implementation. It is not;
    falsify_prml.py is. falsify.py is the workflow engine, a different tool with a
    different schema, installed as falsify-engine. Two places corrected.

Added

  • examples/README.md — routes a reader to the right tool in one table. Every
    command in it was executed before the file was committed.
  • README.md reference-implementation section now states which implementations are
    distributed (Python on PyPI, JavaScript on npm) and which exist in-repo only
    (Go, Rust), with each one's command set. Go and Rust implement the verification core
    and deliberately omit authoring commands; there is no Go module and no crates.io
    crate, and none is planned for the 0.x line. Previously a reader had to guess.
  • pyproject.toml description now says the package also ships falsify-engine.
  • §3.6 Formal Grammar (spec/grammar/): an ABNF for the canonical byte
    sequence plus constraints C1–C7, normative. Verified in both directions by
    spec/grammar/check_grammar.py, an emitter and recogniser written from the
    grammar alone (no PyYAML): 21/21 conformance vectors reproduced byte-for-byte,
    16 malformed canonical texts rejected. tests/test_grammar.py runs the same
    checks in CI. No canonical byte or digest changes.
  • spec/grammar/candidate-vectors-2026-09-13.json: 83 inputs at the edges of
    the plain-scalar predicate and the float rule, with expected bytes and hash
    and each reference implementation's agreement on 2026-09-13 — kept as the
    dated pre-correction snapshot; 82 of the 83 were promoted to the edge suite the
    same day (see below).

Changed

  • §3.5 no longer calls the reference canonicalizer "normative for this
    rendering"; constraint C4 of the grammar is normative and the canonicalizer
    conforms to it. Recorded as a dated erratum in the spec.
  • multi-lang-conformance.yml summary notice said "14 reject-vectors"; the
    suite has 20 (16 for the JSON-only implementations) since 0.3.13.

Fixed (cross-language, same day as the grammar)

  • JavaScript, Go and Rust reference implementations corrected to the §3.6
    grammar.
    The 83-input battery run through all four implementations gave 62
    agreements and 21 divergences — none covered by a conformance vector, so the
    daily CI had stayed green. The three hand-rolled plain-scalar predicates
    (ported from one another) over-quoted ?x :x y n 1e5 0o17 and
    under-quoted - << = 1_000 1:30 0b101; Rust rendered a threshold
    of 1e-05 as 0.00001. All three predicates are now transcriptions of README
    §C5 (P1–P6 plus the YAML 1.1 resolver patterns verbatim); Rust's float
    rendering implements §C4 from the exponent rather than from ryu's choice of
    notation (Rust gains the regex crate). The registry's canonical.js was
    corrected in step; all six publicly listed registry records re-derive their
    stored hash under the corrected canonicalizer. Result: Python 83/83, Go 83/83,
    Rust 83/83, JavaScript 82/83 — 83/83 in all four after the v0.2 decision below.
  • Edge suite spec/test-vectors/edge/edge-vectors.json (92 vectors,
    EV-001..EV-092) added and wired into multi-lang-conformance.yml for all four
    implementations. Separate from the 21 normative vectors of Appendix B by
    design (spec/test-vectors/edge/README.md). FACTS.env gains
    VECTORS_EDGE=92.

Changed (specification, v0.2)

  • v0.2 threshold canonicalizes by value (RFC "Post-freeze clarification
    (2026-09-13)"): integral and below 2^53 → integer digits, otherwise the §3.6 C4
    float. 1300.0 and 1300 are one manifest. The frozen RFC had never stated a
    rule; the ROADMAP proposed the opposite of what the vectors did; JavaScript and
    the registry could not observe the type distinction at all. Python, Go and Rust
    now render 1300.0 as 1300; no published digest changes (no vector spelled an
    integral threshold as a float). falsify_prml.canonicalize applies the rule at
    the root mapping only, like the v0.1 float rule.
  • Edge suite grows to 92 (EV-083..EV-092 pin the boundaries: -0.0→0,
    2^53-1, 2^53→9007199254740992.0, 1e16/10000000000000000→1.0e+16,
    1300.5, 1e-05). FACTS.env VECTORS_EDGE=92. All four implementations and
    the registry pass 92/92.

v0.3.14

Choose a tag to compare

@github-actions github-actions released this 24 Aug 11:45

Packaging and presentation only. No change to canonicalization, verification
or exit codes; falsify verify behaves identically to 0.3.13.

Changed

  • The README no longer describes the current product as a three-day
    hackathon build.
    The README is this package's long description, so that
    line was the first thing a reader of the PyPI page saw about what falsify is
    today. The origin is not removed: the 0.1.0 entry below and SUBMISSION.md
    both still record it, with dates.
  • Package keywords dropped claude and hackathon and now read
    falsifiability, pre-registration, ai-evaluation, reproducibility, audit, CI.
    Neither removed keyword described what the package does.

Fixed

  • falsify stats --html emitted a broken link. The generated report footer
    pointed at github.com/<USER>/falsify-hackathon, a placeholder that never
    resolved, in every HTML report a user produced. It now points at
    github.com/studio-11-co/falsify.