Skip to content

v0.4.1 — Codex review fixes (eval / MCP / lockfile)

Choose a tag to compare

View all tags
@JRS1986 JRS1986 released this 18 May 20:12
· 108 commits to main since this release
v0.4.1
989de91
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Patch release. Three Codex-review findings, all real bugs, fixed in one focused PR.

Fixed

eval `test_command_detected` false positive on empty repos

The check previously trusted the absence of the linter's `missing-build-test-commands` finding as proof. But that finding only fires when context files exist, so an empty repo got the contradictory pair "no agent-context files exist" + "recognizable test commands mentioned."

The check now scans agent-context files directly for any of the recognized verifier tokens, with word-boundary-aware matching so short tokens like `ci` no longer match inside `precise` or `decision`.

MCP `server-not-approved` toothless under default policy

The generated default policy declares `unapproved_servers: "deny"` with an empty `approved_servers` list. But `_check_server` short-circuited the unapproved check on empty allowlist, silently allowing every detected server under the default policy.

The check now tracks the policy posture: `deny` → severity `error`; `requires_approval` → severity `warning`; permissive default stays informational. Without a policy file, the scanner has no opinion (unchanged).

`uv.lock` stale

`uv.lock` recorded `coding-scaffold v0.1.0` even though `pyproject.toml` had moved to 0.4.0 across the v0.2.0, v0.3.0, and v0.4.0 cuts. Re-locked so `uv sync` no longer produces an accidental local diff on first checkout.

Verification

  • `uv run ruff check` clean.
  • `uv run pytest -q` → 310 passed (was 304, +6 regression tests).

Known dogfooding gap

The Codex review noted that the scaffold's own repo scores low on `coding-scaffold eval run`. True. Addressing that requires running `setup run` on this codebase and committing the generated artifacts, which deserves its own PR. Tracked as future work, not in this patch.

Backwards compatibility

Backwards compatible with v0.4.0. Two minor behavior changes worth knowing:

  • Repos with a default-policy file but no `approved_servers` entries will now see `server-not-approved` (severity `error`) for every detected MCP server. This is intentional — the previous behavior contradicted the policy's stated `deny` default.
  • `eval run` against an empty repo now correctly fails the `test_command_detected` check (it previously passed by accident).

Full notes: CHANGELOG.md.

Assets 2
Loading