changelog-discipline

Changelogs your users can trust — 8 enforceable rules, benchmarked. Every published "changelog generator" skill we surveyed makes commits pretty; none makes the result verifiable. LLM-written changelogs have documented failure modes: invented entries, hallucinated version numbers and dates, breaking changes buried or dropped, friendly rewrites that drift from what the code does. This skill is the discipline layer that blocks each one — while still writing for users, not for the git log.

Built and bench-tested by SkillProof, the tested Claude skills directory.

What it does differently

We surveyed 101 changelog/release-notes skills in our 16k-skill dataset plus the standalone tooling (git-cliff, release-please, conventional-changelog) before writing this one. The two halves of a good changelog live in different products: honesty (per-line commit traceability) exists only in mechanical generators, and readability (user-impact language, clean grouping) exists only in LLM skills with zero verification rules. No surveyed skill or tool enforces both. This one does:

1. Derived from git, never from memory. The range is resolved and read (git log, diffs for vague subjects) before a single entry is written. No repo access → no changelog.
2. Every line traces to a real commit or PR. A trace map is built first; an entry that can't point at a commit doesn't ship, and cited (#123) / (abc1234) references must exist in the actual history.
3. Breaking changes are hunted, never dropped. Not just BREAKING CHANGE: footers — removed APIs, renamed flags, changed defaults found in the diff. They go first, marked BREAKING, with a one-line migration note.

Plus: nothing-invented-and-coverage-accounted (every commit mapped or consciously excluded), strict Keep a Changelog grouping, user-impact language that never widens a claim beyond the diff, versions/dates read from real tags (or Unreleased — never minted), and a mandatory self-audit pass before delivery. Output contract: the changelog only.

Install

git clone https://github.com/Skillproofdev/changelog-discipline ~/.claude/skills/changelog-discipline
# restart Claude Code — triggers on "write a changelog", "release notes", "what changed between X and Y"

One command — the repo IS the skill.

Benchmark (measured, not estimated)

Four real OSS repos with hand-curated changelogs as ground truth (Django, Tailwind CSS, FastAPI, curl). For each, one released tag range; two agents get the identical prompt and the repo — the only difference is whether the agent reads this SKILL.md first. Scored on commit-coverage %, fabricated-entry count (mechanically verified links/hashes/dates), breaking-change recall against the human changelog's breaking sections, mechanical format compliance, and blind readability preference.

Range (tags)	Arm	Key-fact cov.	Traceable commit cov.	Fabricated entries	Breaking recall	Breaking first?	Format (0–6)	Blind pref.
FastAPI 0.116.2→0.117.0	base	9/9	12/18 (67%)	0	n/a¹	n/a	4	—
	skill	9/9	9/18 (50%)	0	n/a¹	n/a	6	✅
curl 8.14.1→8.15.0	base	12/12	22/278 (8%)	0	1/1	yes	3	✅
	skill	12/12	59/278 (21%)	0	1/1	yes	6	—
Tailwind v4.0.0→v4.1.0	base	16/17	0/164 (0%)	0	4/7	no	3	tie
	skill	16/17	143/164 (87%)	0	3/7	yes	5	tie
Django 5.2→6.0	base	32/38	23/404 (6%)	0	27/33	no	3	✅
	skill	33/38	234/404 (58%)	0	29/33	yes	6	—

^{¹ FastAPI 0.117.0's curated notes have no breaking section, so recall is n/a. Instead it's a zero-breaking trap: any entry presented as breaking = a fabrication. Both arms passed (0).}

Summary. Across 4 ranges × 2 arms, scored mechanically wherever possible against the human changelogs, with negative results published in full (full verdict, method):

• Fabrication — the headline metric — was 0 for every cell, a tie. Every #-citation in all 8 outputs (up to 195 in one skill run) resolves to a real reference in the range; no invented versions, dates, or PRs; all spot-checked prose (incl. Django's 11 CVEs) is commit-backed. On this corpus the base agent was already honest, so the skill's guarantee held but wasn't differentiated. We report the tie rather than bury it.
• The skill clearly wins where discipline shows: format (30/36 vs 13/36 — base uses non-Keep-a-Changelog headings, drops the ISO date, and adds a notes epilogue), breaking-change placement (skill puts breaking items first and marks them BREAKING in 4/4 ranges; base does in 0/4), and traceability (every skill line maps to a commit; base's prose covers changes it cannot trace back — the gap this skill exists to close).
• The skill does not win everything, and we say so: base is preferred on pure readability for the two large, noisy repos (curl, Django), edges tailwind breaking recall by phrasing a deprecation base happened to catch, and even beats the skill on FastAPI traceable coverage — because the skill correctly dropped three internal commits that base listed. Key-fact coverage is near-identical; both miss the same facts that never surface in a commit subject (Django's DB-backend-API breakers, Tailwind's node_modules default).

^{Blind preference used 1 rater (not 3) this run and is under-powered. Per-run token cost was not captured; the skill arm additionally pays to read SKILL.md.}

When it triggers

"Write a changelog", "release notes for v2.3", "update CHANGELOG.md", "what changed between 1.4 and 2.0", release-notes-from-commits/tags/PRs requests. Explicitly excluded: writing unrelated to git history (blog posts, marketing copy, docs prose) and commit-message authoring.

Why trust this

We test other people's skills for a living (public methodology). Our own skills get the same treatment — measured benchmarks, negative results included.

More skills from us: skillproof-skills index · Free tools for skill authors: SKILL.md validator, token calculator, Rules⇄SKILL.md converter.

License

MIT — use it, fork it, ship it.