Priorities 3, 4 & 7: MCP/skill governance, agent permissions, readiness benchmark#58
Merged
Merged
Conversation
…orities 3, 4, 7)
Priority 4 - Machine-readable permissions
src/coding_scaffold/permissions.py:
permissions write writes .coding-scaffold/agent-permissions.json
with filesystem read/write/deny patterns, shell
allowed + requires_approval lists, network=
disabled_by_default, mcp defaults. Lightly
project-aware (Python -> pytest/ruff,
Node -> npm test, etc.). Idempotent; --force
to regenerate.
Priority 3 - MCP governance
src/coding_scaffold/mcp.py:
mcp policy init writes .coding-scaffold/mcp-policy.json
mcp scan inspects opencode.json + .claude/settings.json;
parses servers; flags remote/unpinned/risky-
launcher/broad-fs-arg/unapproved/denied/
review-required-capability findings
mcp snapshot writes .coding-scaffold/mcp-snapshot.json with
server fingerprints
mcp diff compares scan vs snapshot; exits 1 on drift
No commands executed. No network. Pure config parsing.
Priority 3 - Reviewable skill packs
src/coding_scaffold/skills.py:
skills new <name> scaffolds .coding-scaffold/skills/<name>/ with
SKILL.md, manifest.json, README.md, scripts/,
tests/
skills lint flags broad-usage, hidden-instruction (error),
undeclared-capability, missing sections,
missing manifest fields, invalid risk_level,
placeholder owner, checksum drift
skills approve <n> writes CHECKSUM (sha256 of SKILL.md ||
manifest.json) so future drift surfaces in lint
skills export <n> bundles the skill as tar.gz
Priority 7 - Readiness benchmark
src/coding_scaffold/eval_harness.py:
eval init writes .coding-scaffold/eval-config.json with toggles
eval run runs deterministic checks for build/test/lint signals,
agent instructions, policy, denied files, PR template,
MCP policy (when MCP detected), session trace location,
context lint clean, context budget acceptable
eval report prints the most recent report (--cached for last run)
Reuses lint_context and inspect_context_budget; no model-intelligence
scoring; no shell execution.
CLI
12 new commands wired through src/coding_scaffold/cli.py with --json
output on every one. Existing commands unchanged.
Tests
61 new tests across 4 test files (test_permissions.py, test_mcp.py,
test_skills.py, test_eval_harness.py). 266 total pass, ruff clean.
Docs
- New "Machine-Readable Permissions Artifact", "MCP Governance", and
"Skill Pack Governance" sections in Security.md.
- New "Readiness Benchmark" section in Team-Rollout.md.
- CHANGELOG [Unreleased] covers all 12 new commands.
Zero new dependencies, zero LLM calls, zero network calls. Existing
behavior unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements priorities 3, 4, and 7 from the maintainer's brief. Priorities 5 (worktree) and 6 (memory) remain explicitly out of scope.
Priority 4 — Machine-readable permissions
`coding-scaffold permissions write` generates `.coding-scaffold/agent-permissions.json` with the canonical shape from the brief (filesystem read/write/deny, shell allowed + requires_approval, network=disabled_by_default, mcp policy defaults). Project-aware: Python repos get `pytest`/`ruff` in the allowed list, Node gets `npm test`, etc. Idempotent — `--force` to regenerate.
Priority 3 — MCP governance
No commands are executed. No network calls. Pure JSON config parsing.
Priority 3 — Reviewable skill packs
Priority 7 — Readiness benchmark
Checks (deterministic, no shell, no LLM): build/test/lint signals, agent instructions exist, policy pack present, non-empty deny list, PR template present, MCP policy present when MCP is detected, session-trace location, context lint clean, context budget under limit.
The benchmark scores readiness — it does not measure model intelligence. The point is to confirm the repo gives the agent enough to work with.
Architecture & constraints honored
Tests
Total: 266 passing (was 205, +61). Ruff clean.
Docs
Known limitations
Out of scope (per the brief)