What happened
Release 0.8.6 was cut, its PR opened, and CI failed. The failing test was test_real_changelog_unreleased_section_is_parseable in tests/unit/test_check_changelog.py, added in #390. It asserted that the repository's own CHANGELOG.md always has content under ## [Unreleased].
That assertion is false on exactly one class of branch: a release branch. cut-release.yml moves every bullet out of ## [Unreleased] into a new ## [X.Y.Z] - <date> heading, so immediately after a cut the section is legitimately empty. The test therefore failed on release/v0.8.6, and would have failed on every future release branch.
Fixed in #401. The release was recut and shipped as 0.8.6.
Why no test caught it
Nothing in the suite ever exercises the repository in its post-cut state. Every unit test runs against the working tree as committed on a feature branch, where [Unreleased] always has content because that is what the changelog guard from #390 requires. The one state where the invariant does not hold is the one state we never construct.
This is not specific to the changelog test. Any assertion about repository-level files that a release cut mutates has the same blind spot: CHANGELOG.md, the version heading structure, and anything a future release step rewrites.
Suggested fix
Add a test that applies the cut transformation in memory and asserts the result still parses. The transformation is small and already lives in cut-release.yml around the marker = "## [Unreleased]" logic, so it can be replicated in a fixture without invoking the workflow:
- Read the real
CHANGELOG.md.
- Insert a
## [X.Y.Z] - <date> heading directly beneath ## [Unreleased] and move the bullets under it, matching what the workflow does.
- Assert
parse_changelog / unreleased_has_content still return a well-formed result on the transformed text.
That covers the failure mode directly and costs one test.
Worth considering as a follow-up: the cut logic currently lives as inline Python inside cut-release.yml. Extracting it into scripts/ would let a test call the real implementation instead of a copy, and would remove the risk of the fixture drifting from the workflow.
Related
What happened
Release 0.8.6 was cut, its PR opened, and CI failed. The failing test was
test_real_changelog_unreleased_section_is_parseableintests/unit/test_check_changelog.py, added in #390. It asserted that the repository's ownCHANGELOG.mdalways has content under## [Unreleased].That assertion is false on exactly one class of branch: a release branch.
cut-release.ymlmoves every bullet out of## [Unreleased]into a new## [X.Y.Z] - <date>heading, so immediately after a cut the section is legitimately empty. The test therefore failed onrelease/v0.8.6, and would have failed on every future release branch.Fixed in #401. The release was recut and shipped as 0.8.6.
Why no test caught it
Nothing in the suite ever exercises the repository in its post-cut state. Every unit test runs against the working tree as committed on a feature branch, where
[Unreleased]always has content because that is what the changelog guard from #390 requires. The one state where the invariant does not hold is the one state we never construct.This is not specific to the changelog test. Any assertion about repository-level files that a release cut mutates has the same blind spot:
CHANGELOG.md, the version heading structure, and anything a future release step rewrites.Suggested fix
Add a test that applies the cut transformation in memory and asserts the result still parses. The transformation is small and already lives in
cut-release.ymlaround themarker = "## [Unreleased]"logic, so it can be replicated in a fixture without invoking the workflow:CHANGELOG.md.## [X.Y.Z] - <date>heading directly beneath## [Unreleased]and move the bullets under it, matching what the workflow does.parse_changelog/unreleased_has_contentstill return a well-formed result on the transformed text.That covers the failure mode directly and costs one test.
Worth considering as a follow-up: the cut logic currently lives as inline Python inside
cut-release.yml. Extracting it intoscripts/would let a test call the real implementation instead of a copy, and would remove the risk of the fixture drifting from the workflow.Related
develop, which is the reason a red release branch is possible at all