Skip to content

Retire pre-rebuild skill names from targets/ and lint against their return - #78

Merged
MaxGhenis merged 3 commits into
mainfrom
claude/determined-cartwright-22fef4
Aug 2, 2026
Merged

Retire pre-rebuild skill names from targets/ and lint against their return#78
MaxGhenis merged 3 commits into
mainfrom
claude/determined-cartwright-22fef4

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

Summary

The July catalog rebuild (#61) renamed every skill to its short directory name (skills/<name>/), but the dashboard agent family, the app multizone validator, several commands, and the multizone advisory hook still told agents to load 13 -skill-suffixed names that no longer exist. An agent following those instructions gets a failed Skill load and proceeds without the domain knowledge the file promised it.

Mapping

Each stale name was mapped only after verifying the current skill actually carries the cited content:

Pre-rebuild name Current skill Coverage checked
policyengine-interactive-tools-skill policyengine-tools multizone integration, data patterns A/B/C, embedding, path-mounted vs root-served
policyengine-frontend-builder-spec-skill policyengine-tools "Required stack" (Next App Router, Tailwind v4, ui-kit, bun)
policyengine-recharts-skill policyengine-tools (+ policyengine-design for tokens) "Charts" section: ResponsiveContainer, var(--chart-N), niceTicks
policyengine-vercel-deployment-skill policyengine-tools "Deployment gotchas"
policyengine-modal-deployment-skill policyengine-tools "Pattern C: Modal gateway + worker + polling"
policyengine-simulation-mechanics-skill policyengine population Simulation, economic_impact_analysis
policyengine-parameter-patterns-skill policyengine "Reform dictionaries and parameter paths" replaces the dead "section 6.5" citation
policyengine-us/uk/design/standards/app-skill short names direct renames
policyengine-api-v2-skill policyengine-api v1 endpoint surface + v2 alpha status

Where several old skills merged into one, load lists are deduplicated rather than left with repeated entries. Section-name citations were updated too ("Multi-zone integration (preferred)" → "Multizone integration"; "Canonical zone config — static export" → static-export zones under the same section). analyze-policy and generate-content plugin paths move off the retired category-directory layout (skills/domain-knowledge/...-skill/skills/<name>/). The integrator's household-payload comment now points at policyengine-api (which documents the v1 payload shape) and conditionally loads it; the planner's pattern table follows the skill's current A/B/C lettering with CSV folded into Pattern A. The writing skill's example PR body used two -skill names and was modernized.

Anti-rot

tests/test_no_stale_references.py gains three defenses so this class of rot is caught in CI instead of by an agent at runtime:

  1. A FORBIDDEN entry banning [a-z0-9-]+-skill\b (the pre-rebuild naming convention) across skills/, targets/, docs/, bundles/, presets/ — escapable per line with <!-- stale-ok -->.
  2. test_skill_invocations_resolve — every `Skill: <name>` instruction must name a directory in skills/ (plugin-qualified names skipped).
  3. test_referenced_skills_paths_exist — every skills/... path referenced in the scanned trees (including bundle manifests) must resolve on disk, expanding the {country} template; a negative lookbehind keeps policyengine-skills/... URLs from false-matching.

Verification

  • uv run pytest --ignore=tests/test_skill_examples.py: 71 passed
  • Mutation check: a planted Skill: policyengine-us-skill + dead path line trips all three tests, each with a file:line message
  • Wrapper smoke build (scripts/build_claude_wrapper.py) succeeds; grep of the output finds zero -skill tokens and the dashboard agents present
  • snap-bbce-repeal and uk-spring-statement-2026, kept as precomputed-CSV examples, verified to exist via gh repo view

🤖 Generated with Claude Code

…eturn

The July catalog rebuild (#61) renamed every skill to its short directory
name, but the dashboard agent family, three app/deploy commands, and the
multizone hook still instructed agents to load 13 -skill-suffixed names
that no longer exist (policyengine-interactive-tools-skill,
policyengine-frontend-builder-spec-skill, policyengine-parameter-patterns-skill,
...). An agent following those instructions gets a failed Skill load and
no domain knowledge.

Map every reference to the current skill that verifiably carries the cited
content, deduplicating load lists where several old skills merged into one:

- interactive-tools / frontend-builder-spec / recharts / vercel-deployment /
  modal-deployment -> policyengine-tools (stack spec, multizone, charts,
  Vercel + Modal deployment)
- simulation-mechanics / parameter-patterns -> policyengine (population
  Simulation; "Reform dictionaries and parameter paths" replaces the dead
  "section 6.5" citation)
- us/uk/design/standards/app -skill names -> their short equivalents;
  api-v2-skill -> policyengine-api
- Old section names updated ("Multi-zone integration (preferred)" ->
  "Multizone integration"); analyze-policy and generate-content plugin
  paths moved off the retired category-directory layout; the integrator's
  household-payload pointer now cites policyengine-api and loads it for
  the policyengine-api data pattern; planner's pattern table matches the
  skill's A/B/C letters (CSV folded into A)
- The writing skill's example PR body used two -skill names; modernized

Make the rot detectable so this cannot silently regress:

- Ban [a-z0-9-]+-skill\b in the anti-rot lint (the pre-rebuild naming
  convention can only reappear deliberately via <!-- stale-ok -->)
- test_skill_invocations_resolve: every `Skill: <name>` instruction must
  name a directory in skills/
- test_referenced_skills_paths_exist: every skills/... path referenced in
  scanned trees (including bundle manifests) must resolve, expanding the
  {country} template

Verified: full suite minus test_skill_examples passes (71); a planted
stale reference trips all three checks; the wrapper smoke build succeeds
with no -skill tokens in its output.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis force-pushed the claude/determined-cartwright-22fef4 branch from 22c3e9d to df05275 Compare August 2, 2026 17:30
MaxGhenis and others added 2 commits August 2, 2026 13:53
…nti-rot guards

Review findings (cross-family audit): the /analyze-policy skill list expanded
{country} with the flag value ca to nonexistent skills/policyengine-ca/ — spell
the us/uk/ca -> directory mapping explicitly; backend-builder and
deploy-dashboard keyed replacement-skill loads and the Modal deploy step on
custom-backend / api-v2-alpha, values the planner never emits (its vocabulary
is precomputed | precomputed-csv | policyengine-api | custom-modal); the -skill
ban is now case-insensitive, Skill: invocations with trailing junk are flagged
instead of prefix-parsed, and unknown skills/{...} templates fail instead of
being silently skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Re-verification caught one remaining behavior-gating custom-backend: the
dashboard-planner fallback suggested a pattern value outside its own emitted
vocabulary. targets/ now has zero custom-backend / api-v2-alpha occurrences.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merging under the cross-family dual-review protocol (Claude Fable 5 + GPT-5.6-sol). Sol's audit found two blockers in the original head — the --country ca flag expanding to nonexistent skills/policyengine-ca/, and replacement-skill/Modal-deploy conditions keyed on custom-backend/api-v2-alpha, values the planner never emits — plus guard-hardening notes. Fixes applied (2e44a2b), and sol's re-verify caught one remaining behavior-gating custom-backend in the planner's no-variables fallback, fixed verbatim (a8f6f2b); targets/ now greps clean of both stale values. Guards mutation-tested (planted typo-invocation, unknown template, and case-variant -Skill all trip; clean tree 31/31). Also rebased over #77/#79's microsim-guard work — union in test_no_stale_references.py, full suite 99/99.

@MaxGhenis
MaxGhenis merged commit 4cfc21b into main Aug 2, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant