Skip to content

v0.4.0

Latest

Choose a tag to compare

@MrBinnacle MrBinnacle released this 23 Oct 04:52
· 47 commits to main since this release
96a9e57

This section records the user-visible changes since v0.3.0. Internal test, CI
and docs-only commits are omitted unless they change a published surface.

Read the first entry under Changed before you upgrade. One rename in this range
breaks consumers silently. Nothing raises an error and nothing logs a warning.
A program that matched the old value stops matching and reports nothing.

Changed

  • The domain glossary is renamed from CONTEXT.md to GLOSSARY.md (#722).
    Pocock v1.3.1 names the domain glossary GLOSSARY.md; a reader following
    any live pointer now lands on that file. Git recognizes the move as a
    rename. The title line names the glossary; the body is byte-identical.
    docs/agents/domain.md is replaced with the v1.3.1
    setup-matt-pocock-skills template, and AGENTS.md names GLOSSARY.md on
    both live instruction lines. Existing CHANGELOG entries that name the old
    path stay as written.

  • BREAKING. report_schema_version goes from 1.4.0 to 2.0.0, because the
    aggregation_method value bh_fdr_fallback is renamed to
    bounded_pooling_refused
    (#360, #441, in 03ee27d). The constant lives at
    src/skill_harness/aggregation/report.py:44. The new value is written at
    src/skill_harness/aggregation/fit.py:759. There is no alias on the write
    path and none on the read path. The old string survives only in comments at
    fit.py:255 and report.py:17.

    What breaks. Any consumer that compares against the literal
    "bh_fdr_fallback" stops matching. The comparison returns false rather than
    raising, so the consumer reports nothing instead of failing. Code that
    branches on the value takes the wrong branch. Code that filters on it returns
    an empty result.

    What to do. Match "bounded_pooling_refused". If you must read both
    generations, branch on report_schema_version rather than accepting either
    string, because the two values report different outcomes. The fit is refused
    and pooled at the admission bound, rather than falling back to BH-FDR. A
    consumer that cannot migrate yet can stay on 0.3.0
    (pip install skill-harness==0.3.0). That release reports
    report_schema_version 1.4.0 and still writes "bh_fdr_fallback".

    docs/PRD.md:757 states the rule this follows. Removals, renames, and type
    changes take a major bump, because they break diff skill consumers.

  • The runtime dependency floors rise to the values pyproject.toml declares
    at this PR's head.
    An environment that pins any package below its floor
    no longer satisfies this package's requirements. Measured from
    git diff v0.3.0 HEAD -- pyproject.toml:

    • anthropic>=1.15.0,<2 (was >=1.2.0; the floor passed through 1.5.0 in
      #548 and 1.11.0 in #727, which re-asserts the post-patch bound for
      GHSA-q5f5-3gjm-7mfm and GHSA-w828-4qhx-vxx3, and landed at 1.15.0 in
      #749). The <2 cap is new (#732). anthropic 2.0.0 is a breaking major
      that no audit has reviewed, so an environment that already resolved
      2.x no longer satisfies this package.
    • openai>=3.31.0,<4 (was >=2.41,<4; the lower bound rose in #604 and
      again in #732. 3.17.0 is the first release that requires the patched
      httpx2>=2.12.0; the floor takes Dependabot's 3.31.0, inside the
      audited 3.x range. The <4 cap is unchanged).
    • click>=8.5.0 (was >=8.1; #606).
    • scipy>=1.18.1 (was >=1.11; #729 is the last of the floor bumps in
      the EB-MoM numerical stack).
    • statsmodels>=0.15.0 (was >=0.14; #605).
    • pydantic>=2.14.1 (was >=2.6; #750).
    • tiktoken>=0.14.0,<1.0 (was >=0.7,<1.0; #751. The <1.0 cap is
      unchanged).

    rich did not move. Dev-extra floors moved with Dependabot and are not
    runtime requirements.

  • The ablation runner accepts a whole prompt assembly as an arm, not only a
    clause of one card
    (#554, #615, in 9d956e8a). RunConfig carries a
    declared arm set serialized into runs.config_json. An assembly may name a
    foreign skill body by path. Duplicate assemblies are refused before any
    spend. Arm samples are written to the append-only evidence.arm_samples
    table (migration 1200), and the A41 reconciler sums them with samples.usd.

  • Every external-check refusal reason is persisted (#555, #626, #629,
    in 6b6b74a8). Migration 1100's CHECK constraint listed ten
    UnmeasuredSubReason literals; external_check_missing was never added, and
    INSERT OR IGNORE silently dropped the rows. Migration 1300 rebuilds
    clause_run_outcomes with all eleven members, adds a nullable
    sought_oracle column, and recreates the append-only triggers. Historical
    runs that lost those rows stay lost; new runs record the reason.

  • A renamed published skill no longer loses its value_class (#601, #618,
    in d629227e). The registry key closure-mode-at-boundaries still pointed
    at the pre-rename card name, so value_class_for("closure-mode") returned
    None and the skill took the unclassified path instead of
    trap-discipline. The key is now closure-mode, and a staleness test
    fails when any non-retired key stops naming a live skill.

  • A clause whose scoring axis reaches no comment or clarity outcome is
    refused before sampling, with a clause-specific reason
    (#555, #626, in
    6996e68d). The blanket BLOCKER-1 refusal is replaced by an
    external_check_missing sub-reason that names the axis.

  • The CLI's rich Console reads COLUMNS at render time, not at import
    (#564, #624, in ddf1373e). A module-level Console() froze the width for
    the life of the process, so terminal resize and
    CliRunner(env={"COLUMNS": "200"}) were inert.

  • The published receipt pages stop printing machine tokens as values
    (#583, in 330eca7b). A boolean renders yes or no; a None whose
    schema field names its own null case uses that word (value_class renders
    unclassified); everything else uses the absent class treatment. The
    site also gains page types, a shell with a current-page marker, a 404,
    schema-sourced measurement-table labels, and --base-url so the generator
    stops baking its own host into the output. The landing page sits behind
    --landing (default off) and its one instruction is a command that runs.

  • The ablation results table gains a gate-2 (registered floor) column
    (#546, #558, in c2fa804). A clause that previously rendered a green
    PASSED now renders PASSED (scalar only), HARM (registered floor),
    EQUIVALENT (registered floor), or UNRESOLVED (registered floor). The
    verdict words did not change meaning. The report now shows which decision
    produced them. Anyone who parses this table by column position must re-read
    it.

  • Tie-bearing ablation clause decisions route through the Gate-2 discordant
    rule
    (#368, #493, in 0274dd1). A clause whose decision turned on a tie
    can receive a different verdict than it did under 0.3.0.

  • paired-gate2 refuses a run that carries undecided epochs (#438, #491,
    in 1a5027a). A hazard entry is counted per simple command rather than per
    tool-call blob. When an arm carries undecided epochs, the command emits a
    HAZARD_UNDECIDED refusal and exits 1. A run that previously produced a
    score now produces a typed refusal instead.

  • The PyPI package description is replaced (#497, in 3ba4f69). It now
    carries the repository's GitHub About text.

Added

  • skill-harness run pi-paired CONFIG [--execute], a paired subject lane
    for the Pi runtime (src/skill_harness/cli/main.py:3452). The default is a
    dry run that performs the pre-spend gate only. --execute runs the Full and
    Null epochs and spends money. The subcommand requires a pi binary that this
    package does not install. Without it the launcher raises
    PiExecutableNotFoundError
    (src/skill_harness/subject/pi/launcher.py:164). The runtime version is
    measured from the binary that ran and is never assumed. No sized Pi
    measurement has been run, so no Pi skill effect is admissible yet.

  • SERS 1.4.0 and SERS 1.5.0. 1.4.0 splits the subject and extractor model
    roles and adds subject_identity.subject_model, required only under the
    1.4.0 conditional (#487, in c398cca). 1.5.0 adds four optional top-level
    keys from the On-Irreducibility additions (#526, #547, in cfb5cec). Both
    are additive for existing receipts. A consumer holding an older copy of the
    schema rejects a new receipt, which the standard treats as intended
    generational non-comparability.

  • SERS 1.6.0. Separates the verdict from its present validity (#643, #654,
    in c879633, finished in #655/#660, in 282a5064). A receipt reader must
    handle three new optional top-level objects that are absent on pre-1.6.0
    receipts: verdict_scope (required on a 1.6.0 KEEP, with model_id,
    task_family and delivery_mechanism), currentness (VALIDATED requires
    basis FULL_VALIDATION; CARRIED_FORWARD requires verdict_scope.tested_at
    and renders a carry-forward disclaimer; STALE names a newer metric version
    or hash mismatch), and drift_policy. Receipts from different
    sers_version values are not comparable.

  • SERS 1.7.0. Adds the online-FDR ledger facts and the claim-level ladder
    to the receipt (#644, #686, in bd3467c; ladder fields from #647, #671, in
    3f8d4a90). A receipt reader must handle two changes. First, a 1.7.0 KEEP
    carries a required multiplicity object (LORDdep procedure, target FDR
    0.05, anytime_valid_p, wealth before/after, batch order) and a
    verdict_scope.control_world_result that must equal pass; a 1.7.0 CUT
    must not carry multiplicity at all. Second, KEEP receipts gain
    designer, designer_independent, family_replication, model_replication
    and claim_level (KEEP / REPLICATED / ROBUST), with the ladder thresholds
    enforced on write.

  • The claim-level ladder (#647, #671, in 3f8d4a90). A family record
    carries five mechanism fields and registration checks. Receipts record who
    designed the claim, whether the designer is independent, how many
    independently designed families replicated, and how many materially distinct
    families a robust claim needs (three, with at least one independent
    designer). Render sentences carry the same-designer qualifier.

  • Collection screens record what each run actually delivered (#664, #672,
    in ef69c192; #667, #698, in 2787c13c). The delivered-surface extractor
    marks each subject card visible (with position), truncated (with delivered
    length) or dropped, never guessing an empty listing. The collection screen
    gains a seeded random-half arm: the seed and the drawn card names freeze
    into runs.config_json, and a delivery-integrity flag marks a run whose
    intended card was dropped or truncated from the delivered listing.

  • The [geometry] extra (#589, in the rendered-geometry harness).
    pip install "skill-harness[geometry]" pulls playwright>=1.63.0, the
    browser behind src/skill_harness/sitegen/geometry.py. The core package
    never imports playwright; the module raises BrowserNotInstalledError with
    the extra named. Installing the distribution is half the job: the Chromium
    build is fetched separately with python -m playwright install chromium.

  • Drift-check rows AC-2, AC-3 and AC-4. AC-3 covers the public vacuity
    claims (#543, #565, in f3f2ba9). AC-4 covers workflow configuration (#571,
    in dc7520e). Rows DC-17 (mirror records, #514), DC-18 (Vale version
    pins, #488) and DC-19 (asset pairs, #592) are also live. The drift check
    reports 22 live contracts at this version.

  • Storage migrations 1100_clause_run_outcomes.sql and
    1101_structural_covariates.sql (in e047967), plus 1200 for arm samples
    (#615) and 1300 for the clause_run_outcomes CHECK rebuild (#629). All
    create or rebuild tables additively from a v0.3.0 store's point of view;
    1300 is the one that replaces a table in place.

  • The release gate reports its own coverage (#576). Every gate that does
    not run prints a named skip line, and both summary lines state how many of
    the eight gates ran. At 0.3.0 the gate prints 7 of 8 gates ran; skipped G6.
    Off the 0.3 minor line it prints 5 of 8 gates ran; skipped G6, G7, G8. G7
    and G8 previously returned in silence, so RELEASE GATE: PASS covered six of
    eight checks and did not say so.

Fixed

  • Ingest finds the skill listing where current Claude Code writes it
    (#675, #687, in a9fb533; #674, in 6505b42). Current Claude Code puts the
    Skill-tool listing in a system message, not in the first user message. The
    extractor and the Stage 1A screens reader searched only the first user
    message, so a delivered card was recorded as no_listing and a look
    crashed after both samples completed. Both now scan messages in order until
    the first assistant message and read the listing from the first message of
    any role that holds Claude Code's header. A listing that first appears at or
    after the first assistant message is still not attributed to the run.

  • requirements-ci.txt pins urllib3==2.8.0 for
    CVE-2026-97687/97688/97689
    (#701, in e235bc6). This is the CI
    constraints file, not a runtime floor in pyproject.toml; the bump closes
    the advisory on every environment that installs from that pin.

  • The SERS schema held the UNMEASURED sub-reason vocabulary twice and one
    copy was untested
    (#578). $defs/rate_or_refusal is referenced by ten
    measurement fields and was short tier2_uncalibrated and
    length_confounded, so a receipt recording either of the runner's
    pre-sampling refusals could not be written. Both copies are now asserted
    equal to UnmeasuredSubReason.

  • ExtractedClause.axis is persisted unnormalised, so a padded axis splits one
    axis into two (#504, in 8d5e0d6).

  • unmeasured_reason is never persisted, and its vocabulary is disjoint from
    UnmeasuredSubReason (#503, in 710ef78).

  • An absent dump column is a difference, not agreement (#360, in 1f5a73e).

  • Two mutants share the id M-R1 and two share M-R2, and nothing asserts
    that a mutant id is unique (#536, in af0788f).

  • The lint gate does not cover scripts/, and a syntax error reached a commit
    through the gap (#535, in bef52e1).

  • pre-commit run --all-files cannot pass on main, because ruff fails on
    prototypes/, which CI never lints (#483, in fe5b3c1).

  • test_mint_path_allowlist filters on the absolute path, so it is vacuous in
    a build worktree (#482, in 7284342).

  • Receipt pages render no subject_identity, so the 1.4.0 subject_model is
    invisible to a reader (#490, in eebaad8).

  • DC-16 selects the tracked set, not a filesystem walk (#471, #473, in
    b06272a).

  • The six UNLANDED mirror rows point at an open ticket and no longer pin its
    number (#521, in 658f769).

  • The G7 and G8 GitHub reads authenticate with GITHUB_TOKEN (#496, in
    cf250bd). Unauthenticated reads share a 60-per-hour budget per IP, which
    shared CI runners exhaust.

  • The dry-run claim is scoped to the subcommand that spends (#472, in
    8dab4d3).

  • The frozen class-2 reference is runnable, its copies are pinned, and the R40
    receipt is frozen (#458, in 6225c66).

  • numpy stays out of the runtime manifest (#161, #442, in 52ee261).

  • Two tests reported the environment as a code failure. One now skips with the
    privilege named, and the other scans where the build actually runs (#484, in
    db3a688).

  • The design snapshot describes the banner main draws, not the window it
    retired (#481, in 466e629), and no longer declares the paper surface that
    #308 retired (#485, in 8a4d064).

  • README "Measuring for real" snippet now shows what run ablation --execute
    actually requires
    (#460). --execute also needs --ratification,
    --task-family, and --estimand; the ratification preflight refuses before
    any model call and before the API-key check, which the prior wording did not
    say.

  • CHANGELOG 0.3.0 entry for #424 corrected. The ticket did not stay open on
    the negative control; it closed 2026-09-04 once the control was re-seeded to
    satisfy the #403 ruling's qualified-fixture precondition (#432). The 0.3.0
    section otherwise unchanged.

Internal

The bulk of this range is a rebuild of the empirical-Bayes method-of-moments
peel in src/skill_harness/aggregation, 65 commits that change fit.py by 904
lines. The rebuild carries one externally visible consequence, the rename
recorded under Changed. The work is covered by a frozen pre-registration
(b477983), a superseding v2 frozen on mechanism class 2 (e11e4e2,
397e840), a confirmatory run that returned REJECTED at R=1000 on the
committed root (13f0fbb) and NOT_REJECTED on a fresh root (feb318d), and
mutation receipts re-run against the code that ships (36dcc22, 4348275).

Stage 1A and Stage 2 regime simulators for #621/#685, the v3a Null
qualification screen, comparative collection screens, ADR 0002, and the
On-Irreducibility decline records change research findings and internal
records, not a PyPI-observable surface. Dependabot bumps of pinned dev and CI
actions throughout the range follow the same rule.