v0.7.0 — Context Efficacy Proving Harness (--prove)
Turns the scanner from a context linter into a context proving harness: a
report-only Efficacy Score (0–100) that proves what a repo's AI artifacts actually
do, with reproducible per-artifact evidence. It never changes the Proficiency Score
or maturity level.
What it proves
- Commands — documented CLI commands resolve on
PATH; under--prove-exec, a fixed<cmd> --helpprobe of the allowlisted, PATH-resolved binary runs (never the documented args). - Hooks — validate-only (wiring + contained script existence); hooks are never executed.
- Context budget (deterministic) — always-on token footprint (instruction files + skill frontmatter), % of a reference window, efficiency factor. Quantifies the conciseness/bloat work.
Usage
measure-ai-proficiency --prove # resolve-only (runs no repo code)
measure-ai-proficiency --prove-exec # also probes allowlisted commands in a sandbox (local only)Security (passed an adversarial harden + spec audit)
--prove runs no repo code; --prove-exec is opt-in, local-only, hard-blocked on remote/GitHub scans. argv-list only (never a shell); allowlist of build/test/lint nouns (no shells/interpreters; repo config can only narrow it); HOME isolated to a throwaway temp dir (no credential reads); symlink containment; per-call timeout + output cap; report fields sanitized; the CI comment renders the report as inert fenced text. MCP prove_efficacy is resolve-only. Full threat model in docs/EFFICACY.md.
Surfaces
CLI (--prove/--prove-exec/--context-window) in all four reporters + JSON/CSV; MCP prove_efficacy tool; a PR CI workflow that comments the efficacy result.
Quality
152 tests passing; EVAL-014/015/016 guard the engine + two security invariants.