Skip to content

v0.2.0a0

Pre-release
Pre-release

Choose a tag to compare

@MrBinnacle MrBinnacle released this 20 Jul 19:13
· 662 commits to main since this release
6d92bd9

Added

  • v0.2 subject layer (skill_harness.subject, optional [inspect] extra):
    HarnessPin.capture() — the subject-harness configuration captured from the live
    environment as an admissibility field (refuses version="auto"), and
    build_paired_tasks() — the Full-vs-Null contrast as two Inspect tasks identical
    except for the skill under test (inspect_swe.claude_code(skills=…)), with
    file_contains / command_succeeds outcome oracles resolved against the agent's
    pinned cwd. Supply-chain review: docs/supply-chain/inspect-audit-2026-07-09.md.
  • skill-harness skill audit <SKILL.md> — fully offline preflight (no API key, no DB,
    no cost): structural lint against Anthropic's published authoring spec plus an
    evaluability report stating which axes a paid run could mechanically measure and
    which claims would return UNMEASURED. --strict exits 1 on warnings for CI.

Fixed

  • Installed copies of v0.1 were broken (worked only from a source checkout);
    caught by a fresh-venv install test:
    • hedge_index loaded its wordlist from tests/… (not shipped in the wheel) —
      now package data at src/skill_harness/oracles/tier1/fixtures/.
    • SQL migrations resolved from the repo root — now package data at
      src/skill_harness/storage/migrations_sql/; installed copies can bootstrap DBs.
    • tiktoken and statsmodels were imported at runtime but declared only in dev
      extras — now runtime dependencies.
  • examples/README.md still claimed the extractor had no OpenRouter fallback
    (stale since b5b9fe6).
  • Repaired all references left dangling by the 2026-07-08 privacy scrub
    (COUNCIL_FINDINGS.md, root CLAUDE.md) across CONTRIBUTING, CODEOWNERS, PR/issue
    templates, CHANGELOG, PRD, PLAN, release notes, and the case study.
  • Hostile-review hardening pass on the unreleased v0.2 subject, aggregation,
    and Tier-1 oracle layers: 39 confirmed findings fixed across the
    ablation / aggregation / calibration / extractor / CLI / storage paths —
    direction-aware ablation wins, BH-FDR gate consumption, scorer-crash
    inadmissibility, resume cost attribution, sentinel-verdict exclusion from
    statistics, and single-source pricing/tokenizer. The four remaining
    test-quality findings are tracked, not silently dropped.
  • Citation oracle no longer arm-differentially deflates hyperlink references:
    lowercase prose is no longer miscounted as a flag emission (flag matching is
    now case-sensitive), and an external http(s) markdown-link target is
    preserved as a citation rather than stripped away with its link text.

Changed

  • README rewritten for its actual audience (Claude Code users with a skills folder):
    value proposition first, two-command keyless quickstart, honest pre-alpha status,
    fact-checked comparison table (skill-eval-harness, promptfoo, Inspect).
  • PRD.md, PLAN.md, RELEASE-NOTES-v0.1.md moved under docs/.
  • docs/findings/v0.2-reaim-gate.md amended after the 2026-07-09 competitive sweep:
    adds "Harness pin" and "Differentiation vs field" pre-registration fields and a
    correction block recording that the original lock predated the sweep.

Security

  • Untrusted skill names and model output are escaped with rich.markup.escape
    and asserted NUL/control-free before terminal rendering, so bracketed text
    cannot inject Rich markup into CLI output.
  • CI workflow actions are pinned by commit SHA, and structural enforcement bans
    (e.g. raw sqlite3.connect) are wired as tests rather than left as convention.