Skill Sunset: a local, read-only audit for stale AGENTS.md rules

[ooocooc]

The problem

My AGENTS.md and CLAUDE.md files kept growing, but I had no safe way to tell which instructions were still useful and which were stale, duplicated, or too broad.

So I built Skill Sunset, a local, read-only audit for agent instruction files. The important constraint is: TEST is not RETIRE. A suspicious rule is not removed just because a heuristic dislikes it.

npx skill-sunset@latest audit --codex --open

It scans AGENTS.md, CLAUDE.md, and generic SKILL.md files, then produces an evidence-backed report with conservative verdicts such as MERGE, UPDATE, DEMOTE, RETIRE, and TEST. It does not call an AI API, modify the files it scans, or delete anything.

A reproducible example

The repository includes a public fixture used by the current CLI and tests. A fresh run today produced:

• 5 files scanned
• 11 findings
• 71/100 health score
• 0 symlinks, oversized files, or read errors skipped

The findings keep verified duplicates and broken references separate from behavior hypotheses that still need an experiment.

There is also one external integration signal: HOL Guard v3.0.18 added command-safety coverage for the installed skill-sunset audit surface after PR #2611 was merged.

Repository: https://github.com/ooocooc/open-skill-sunset

I am the project author. Parts of this write-up were drafted with AI assistance and then checked against the current CLI output and repository evidence.

I would especially value technical counterexamples: which verdict, edge case, or instruction pattern does the tool classify incorrectly? Redacted examples are welcome.

[Hahaknight]

Strong framing on TEST ≠ RETIRE — that distinction is where most cleanup tooling goes wrong.

Reading this from the skill-authoring side: staleness is half of instruction-file health. The other half I keep hitting is trigger drift — a rule can be perfectly current but effectively dead, because its description no longer matches how users actually phrase the request, so the agent never loads it. A file can pass a freshness audit and still contribute zero. A future check worth considering: per-rule 'last actually invoked' heuristics where the data exists.

(Context: I maintain a pack of workflow SKILL.md files — https://github.com/Hahaknight/claude-skills-pro — and description-as-trigger-surface is the #1 reason a well-written skill silently does nothing.)

[ooocooc]

Thanks — this was useful. I’ve turned it into a local activation checklist in the generated report bundle. It tests two paraphrases that should load a Skill and one nearby request that should not. I’m keeping the evidence boundary strict: no conversation history or AI calls, and if a runtime exposes no reliable load event the result stays UNKNOWN—not “unused” and never RETIRE evidence. A load event only proves the Skill was found, not that every rule was followed. Your point helped separate instruction freshness from whether the runtime actually opened the Skill.

[Hahaknight]

.