Skip to content

v0.65.2

Latest

Choose a tag to compare

@github-actions github-actions released this 21 Oct 23:18
· 1 commit to main since this release

[0.65.2] - 2026-10-22

Cost phases 2 and 3: hm check --reuse, compact hm check, golden JSON, SPEC defaults instead of
Open Questions. Tier-1 cost ratio 14.31 -> 11.36.

Cost phase 3 (SPEC-hm-cost-p3): tier-1 transcript ratio 12.21 -> 11.36, short of the pre-registered
10.0 bar (recorded; spend per hm run -5%). A reused gate saves test wall time, not tokens: the remaining
cost is context carried across stages.

Added

  • hm check --reuse reprints the last recorded pass when nothing it depends on changed (options,
    HEAD, gate list, harness config, tree snapshot, harness-maker version). The record is
    $(git rev-parse --git-path hm/check-verdict.json), written only after an exit-0 run with no unreadable
    file and an unmoved HEAD; a failing run removes it. /hm:wrapup Step 1 uses it.
  • docs/evals/history/<slug>.jsonl freezes a task's counted eval rows when a later task replaces
    docs/evals/results.jsonl.

Changed

  • /hm:spec settles spec-validator critiques as defaults under Refinement Decisions; Open Questions
    holds only what the DRI owns (WHY/WHAT/scope/trade-off, IRR and oracle critiques), so approval is usually a
    stamp.
  • /hm:execute runs hm check once, unconditionally, after writing the REVIEW (the per-round rerun is
    gone), and chains independent shell steps with &&.

Cost phase 2 (SPEC-hm-cost-p2): tier-1 carry ratio 14.31 -> 12.21.

Added (phase 2)

  • hm spec_machine export-golden writes specs/SPEC-<slug>.golden.json; wrapup lands it
    (--optional).
  • approval in hm spec_machine check.

Changed (phase 2)

  • hm check prints compact output (JSON line plus failing gates); --full restores the old output.
  • Stage template rules: keep the PLAN short, edit documents in place, record IRRs with the
    verbatim categories, and fall back to a stdlib seeded-random property test when Hypothesis is absent
    (a missing Hypothesis no longer counts as a new external dependency that voids approval).

Boundary tests (advisory)

Layer 1 of PLAN-test-fidelity-gap. Pure parser checks; advisory only.

Status: ✅ PASS

============================= test session starts ==============================
platform linux -- Python 3.12.15, pytest-9.0.3, pluggy-1.6.0
rootdir: /home/runner/work/harness-maker/harness-maker
configfile: pyproject.toml
plugins: asyncio-1.3.0, anyio-4.14.2, respx-0.23.1, hypothesis-6.155.7, xdist-3.8.0
asyncio: mode=Mode.STRICT, debug=False, asyncio_default_fixture_loop_scope=None, asyncio_default_test_loop_scope=function
collected 38 items

tests/integration/test_boundary_bench_resume.py s                        [  2%]
tests/integration/test_boundary_codex_toml.py ............               [ 34%]
tests/integration/test_boundary_harness_yaml.py ........                 [ 55%]
tests/integration/test_boundary_hooks_json.py ....                       [ 65%]
tests/integration/test_boundary_meta.py ......                           [ 81%]
tests/integration/test_boundary_settings_json.py .......                 [100%]

======================== 37 passed, 1 skipped in 3.78s =========================