Add a bounded, machine-enforced self-improvement gate to update-knowledge - #9
Merged
Conversation
…edge The routine can now fold a fix to its OWN skill file into the PR it opens (instead of only ever suggesting it) - but only through a fenced, default-deny gate designed to be safe for a self-modifying autonomous loop. Design was pressure-tested with a fan-out of 5 independent design lenses, each adversarially critiqued, then synthesized. The decisive finding: a self-graded prose checklist is not a gate - it's the primer's own "Friendly Fire" weakness aimed at the gate itself. So the safe design is narrow + harness-enforced: - New SKILL.md step 7 "Triage improvements to this routine" (Open-a-PR -> 8). Two lanes: suggest (default) vs. apply-into-the-PR. Apply never means auto-merge; a human still reviews and merges every PR. - Auto-apply envelope is tiny: one non-semantic, repo-internal fix per pass (stale cross-ref / broken path / typo), <=10 lines, one SKILL.md only, as a single self-edit: commit. Everything semantic, web-sourced (the injection surface), or touching guardrails/verification/frontmatter/the-gate/permissions stays a human suggestion. - .github/workflows/self-edit-guard.yml + self-edit-guard.py (the repo's first CI) mechanically enforce scope, protected regions (sentinel-fenced), size, and the per-pass cap - so those checks are real, not self-graded. Tested against a valid case plus scope/protected/size/cap violations before shipping. - Applied self-edits surface in a leading "Routine self-improvements" PR section; runbook reviewer checklist and a CLAUDE.md convention added. Open questions surfaced for the human in the PR: branch-prefix mismatch vs. the Routine push restriction; making self-edit-guard a required check; and whether update-knowledge's own hard stops are mechanical rather than prose. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nsk6ryTqYCAuE6dRW8ZnHJ
Folds in two of the three follow-ups flagged on the self-improvement PR: - Branch prefix: step 8 now opens the PR from a claude/knowledge-update-* branch (was knowledge/update-*), keeping it inside the Routine's default claude/-only push restriction so the "never touches main" floor holds without disabling that restriction. Matches the repo's earlier history. - Hard stops made explicit: the skill's "Guardrails for this skill" section now maps each of the three stops to a real backstop for a scheduled one-shot (iteration = one trigger/pass + no self-reinvoke + the native CLAUDE_CODE_MAX_SUBAGENTS_PER_SESSION cap; budget = subscription/daily-run cap, real only with metered overage off; stall = N/A / no-op exit). The runbook gains a "Budget & caps" section and env-var rows with concrete values, and the branch-prefix caveat is rewritten as reconciled. The third follow-up (make self-edit-guard a required check; set the Routine env vars / keep overage off) is settings-only and noted for the human. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Nsk6ryTqYCAuE6dRW8ZnHJ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Routine self-improvements
(This new PR section is itself part of the feature — every future update-knowledge PR leads with it.)
self-edit:.What this adds
The
update-knowledgeroutine can now fold a fix to its own skill file into the PR it already opens — instead of only ever suggesting it — satisfying your ask: "either suggested for a human to confirm, or if the change is safe and passes a suitability validation step, implemented without human confirmation."Because this is a self-modifying autonomous loop, I didn't hand-wave the "suitability validation": I ran a fan-out of 5 independent design lenses, adversarially critiqued each, and synthesized. Decisive finding across all five: a self-graded prose checklist is not a gate — it's the primer's own §5A "Friendly Fire" weakness aimed at the gate itself. So the design is narrow + harness-enforced, mirroring this repo's #1 principle ("a cost alert is not a ceiling; enforcement lives in the harness"), applied to self-editing.
How it works
SKILL.mdstep 7, "Triage improvements to this routine" (Open-a-PR → step 8). Two lanes: suggest (default) vs. apply-into-the-PR. "Apply" never means auto-merge — a human still reviews and merges; the edit takes effect only after merge.SKILL.mdonly, ≤10 lines, singleself-edit:commit, default-deny. Anything semantic, web-sourced (the injection surface), or touching guardrails / verification / the-gate / permissions stays a human suggestion..github/workflows/self-edit-guard.yml+self-edit-guard.py(the repo's first CI) fail any PR whoseself-edit:commit breaks scope, touches a sentinel-protected region, or exceeds the size/cap. Tested against a valid case + 4 violation types; passed live on this PR.## Routine self-improvements; runbook reviewer checklist + a CLAUDE.md convention added.Follow-ups folded into this PR (the three open questions)
claude/knowledge-update-YYYY-MM-DDbranch (wasknowledge/update-*), so it stays inside the Routine's defaultclaude/-only push restriction — the "never touches main" floor holds without disabling that restriction. Matches the repo's earlier history too. Runbook caveat rewritten as reconciled.CLAUDE_CODE_MAX_SUBAGENTS_PER_SESSION; budget = subscription/daily-run cap (real only if metered overage is off); stall = N/A / no-op exit. The runbook gained a "Budget & caps" section + env-var rows with concrete values.Still needs you (settings-only — can't be done in a PR)
self-edit-guarda required check. GitHub → Settings → Branches → add a rule formain→ Require status checks to pass before merging → selectguard. (No MCP/API tool for branch protection is available to me, so this one's manual — ~30s.) Safe to require: the workflow runs on every PR to main, so the check always reports.CLAUDE_CODE_MAX_SUBAGENTS_PER_SESSION=12, and keep metered overage OFF (or set a spend alert) so the subscription limit is a true hard budget stop. Details inrunbooks/staying-current.md→ "Budget & caps."Not merging — per repo convention, a human reviews and merges.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01Nsk6ryTqYCAuE6dRW8ZnHJ