v3.4.1

And make sure to update the README, because the README doesn't seem updated, even though we're on a new version and there's way more to the tool.

v3.4.1

Post-release patch for v3.4.0. Bug fixes and doc corrections only — no new public API, no breaking changes.

Fixes came from a CodeRabbit review of #61 after v3.4.0 shipped. Test portability failures on Windows CI were caught in #62.

---

run_eval.py — control query not counted in shared call budget

_check_infra used a private CallBudget(1) that the caller never saw. With --max-calls 15 you got 16 actual model calls.

# before
calls = len(eval_set) * runs_per_query * (max_retries + 1)
...
from scripts.call_budget import CallBudget as _CB
if not _check_infra(..., _CB(1), max_retries):

# after
calls = 1 + len(eval_set) * runs_per_query * (max_retries + 1)
...
if not _check_infra(..., budget, 0):

bsc.py eval pre-check updated to match: (1 + len(cases) * runs * (retries + 1)) * len(models).

New behavioral test added: test_run_eval_reports_infrastructure_failed_when_control_query_not_triggered — asserts infrastructure_failed: true and zero eval calls when the control query does not trigger.

---

bsc.py — hard-coded .git/hooks broken in worktrees and submodules

doctor() and _install_hooks() both used ROOT / ".git" / "hooks". In a git worktree the hooks directory lives at .git/worktrees/<name>/; in a submodule it lives inside the parent repo. Both cases silently broke hook install and the doctor check.

# before — two hard-coded call sites in doctor() and _install_hooks()
hook_dest = ROOT / ".git" / "hooks" / "pre-commit"
git_hooks = ROOT / ".git" / "hooks"

# after — shared resolver, fallback when git unavailable
def _hooks_dir():
    hooks = subprocess.check_output(
        ["git", "rev-parse", "--git-path", "hooks"], cwd=ROOT, text=True, ...
    ).strip()
    return (ROOT / hooks).resolve()

The hooks subcommand was also incorrectly picking up the path positional added to other subcommands, requiring a skill path it does not need:

# before
elif name not in ("doctor", "compare"):
    p.add_argument("path", type=Path)

# after
elif name not in ("doctor", "compare", "hooks"):
    p.add_argument("path", type=Path)

---

review_gate.py — wrong summary line in --release mode

In --release mode a review-skipped finding is promoted to an error. The if chain checked skipped_finding first, so the promoted error still printed "skipped — no review record" even though errors was non-empty.

# before
if skipped_finding:
    print("Review gate: skipped — no review record")
elif errors:
    print(f"Review gate: failed — {len(errors)} error(s)")

# after
if errors:
    print(f"Review gate: failed — {len(errors)} error(s)")
elif skipped_finding:
    print("Review gate: skipped — no review record")

Exit-code logic was already correct; only the printed summary was wrong.

---

validate.py — scoring exception swallowed, gate returned exit 0

# before — prints to stderr and continues; validate.py exits 0 with a broken score
except Exception as exc:
    print(f"Score: (error: {exc})", file=sys.stderr)

# after — appended as an error Finding; gate exits 1
except Exception as exc:
    findings.append(Finding(
        severity="error", rule="score-error", message=f"Scoring failed: {exc}",
    ))

---

.github/workflows/test.yml — canary used wrong invocation; review_gate escape let errors through

Canary was calling python -m scripts.run_eval from working-directory: skills/skill-creator with a hardcoded model version ID and ANTHROPIC_API_KEY only. Three problems: python -m scripts.run_eval bypasses bsc.py's sys.path setup; the hardcoded model ID claude-haiku-5-5 will rot; CLAUDE_CODE_OAUTH_TOKEN was missing so Claude Code's OAuth auth path failed silently.

# before
- name: Run trigger canary
  working-directory: skills/skill-creator
  env:
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
  run: |
    python -m scripts.run_eval \
      --eval-set tests/should_trigger.yaml \
      --skill-path . \
      --model claude-haiku-5-5 \
      --max-calls 15

# after — runs from repo root via bsc.py, tier alias, OAuth token
- name: Run trigger canary
  env:
    CLAUDE_CODE_OAUTH_TOKEN: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }}
  run: |
    python bsc.py eval "$canary_dir" --live \
      --model haiku \
      --max-calls 15

review_gate escape: the step had || [ $? -eq 2 ], which passes CI whenever the exit code is 2 — but also silently eats errors. For lint and static_analysis this softening is intentional (warnings should not block PRs). For review_gate it is wrong: any non-zero exit is a hard failure. Removed. The step is also now conditioned on non-PR runs — PR branches may have a pending review in progress; main and scheduled runs must not.

# before — escape swallows errors on review_gate too
python -m scripts.review_gate . || [ $? -eq 2 ]

# after — hard fail; PR runs skip the step entirely
- name: Enforce review gate on main and scheduled runs
  if: github.event_name != 'pull_request'
  run: python -m scripts.review_gate .

---

CLAUDE.md — stale description of what CI runs

The file said CI runs validate.py (the unified gate) on every push. CI actually runs quick_validate, lint, static_analysis, and review_gate as separate steps — validate.py's score threshold and description-confidence checks are intentionally excluded from CI. Also corrected the pre-commit hook description (it also runs lint and review_gate, not only quick_validate).

# before
CI runs it on every push; the pre-commit hook runs `quick_validate` (faster, structural only).

# after
CI runs `quick_validate`, `lint`, `static_analysis`, and `review_gate` as separate checks;
it does not run the unified gate, its score threshold, or description-confidence checks.
The CI review gate blocks main pushes and scheduled runs; PR development can proceed
with a pending review. The pre-commit hook runs `quick_validate`, lint, and the review
gate, plus pyright when available.

---

README.md — bsc.py hooks install missing from Quick start; --model undocumented

The hooks install subcommand added in v3.4.0 was not in the Quick start. The old step 2 jumped straight to "Create or improve a skill" with no mention of the hook. New step 2, now in the README:

2. Install the pre-commit hook (optional but recommended):

python bsc.py hooks install

Runs quick_validate on every commit to catch structural errors before they reach CI.

Steps 2–5 renumbered to 3–6.

bsc.py eval --model / --models (also added in v3.4.0) were also absent. The eval section ended at --live with no indication that model selection was possible. Added immediately after the --live block, now in the README:

Pass --model haiku (or --models haiku,sonnet) to run against specific model tiers.

---

tests/test_readme.py — code block count off by one after README change

# before
assert len(_code_blocks(quick_start)) == 4, (
    "Install, validate, preview and live evaluation need code blocks"
)

# after
assert len(_code_blocks(quick_start)) == 5, (
    "Install, hooks, validate, preview and live evaluation need code blocks"
)

---

tests/test_bsc_hooks.py — execute-bit and submodule assertions fail on Windows CI

stat().st_mode & 0o111 is always 0 on Windows — chmod with execute bits is a no-op on NTFS. git submodule add with local file:// paths exits 128 on the Windows GitHub Actions runner.

# before — unconditional, fails on Windows
assert hook.stat().st_mode & 0o111

elif layout == "submodule":
    parent = tmp_path / "parent"
    ...

# after
if sys.platform != "win32":
    assert hook.stat().st_mode & 0o111

elif layout == "submodule":
    if sys.platform == "win32":
        pytest.skip("git submodule add with local paths is unreliable on Windows CI")
    ...

---

PRs: #61 · #62 · #63