Add changelog workflow based on 'news fragments'

[SnoopJ]

Requested Feature

The current workflow for maintaining Sopel's NEWS file involves a lot of human labor, a lot of which is associated with writing changelog text that goes with each changeset.

I propose to introduce a workflow for maintaining the NEWS file using a "fragments" workflow like the one used by CPython.

The basic idea is that each PR includes a "fragment" file that lists entries to be added to the changelog, and at release time, these fragments are automatically gathered into a new changelog entry.

I am personally a fan of the tool scriv for its simplicity and flexibility, and intend to submit a PR based upon it, but there are other tools that implement a similar fragments-based workflow, e.g. towncrier and setuptools-changelog.

Problems Solved

This approach spreads the cost of maintaining the changelog across all changesets, and encourages PR authors to summarize their own work in a human-readable fashion.

This workflow also allows changesets to list changes in multiple sections of the changelog. For example, a PR may include changes that affect both the core details of Sopel and public-facing API. This is technically possible with the current manual authoring of the changelog, but it seems that the most common realization is that we choose one category or another for each such changeset.

It is also easy to check using CI that a changeset includes a fragment file. Most workflows I have seen allow bypassing this requirement using a specific NO NEWS REQUIRED label or similar.

Alternatives

• Continue to manually collate and edit the changelog.
  • Requires human effort at release time, proportional to the size of a release
• Have changesets edit the changelog directly.
  • Creates a confusing historical record of changes by modifying the changelog before a release happens.
• Generate the changelog automatically from commit messages (using e.g. git-chglog)
  • Requires more human attention to commit message format than status quo, has lacking granularity (i.e. a commit that includes multiple kinds of changes), and shoves more metadata into commit messages.

Notes

No response

[Exirel]

I like the idea of having a "unreleased" section at the top of the changelog that everyone can add stuff to when creating a PR. That's the way I've been working on stuff when I don't want to add any more work or tools in a solution.

[SnoopJ]

I realize that what I'm proposing may make more sense if I show the actual commands that are run during PR preparation and changelog creation. Here's what those look like from a terminal POV:

PR author workflow

$ cd ~/repos/sopel
$ git checkout -b feature/amazing-feature
$ vim sopel/bot.py  # edit file, add amazing feature
$ scriv create --edit  # create a new fragment file, open in $EDITOR; user adds their changelog info
Creating changelog.d/20231126_183333_snoopj_amazing_feature.rst
$ git add sopel/bot.py
$ git add changelog.d/20231126_183333_snoopj_amazing_feature.rst
$ git commit -m "bot: add amazing feature"

Release workflow

$ cd ~/repos/sopel
$ scriv collect --version v1.2.3
$ git add NEWS
$ git rm changelog.d/*  # these steps could be replaced by adding --add to the collect command
$ git commit -m "NEWS: update changelog for v1.2.3"

[SnoopJ]

I like the idea of having a "unreleased" section at the top of the changelog that everyone can add stuff to when creating a PR. That's the way I've been working on stuff when I don't want to add any more work or tools in a solution.

I admire the simplicity of this solution, but I feel like the granularity of fragments is worth the marginal additional complexity. It's slightly harder to do a partial revert of a changeset that touches the unreleased changelog, for instance, and of course there is the possibility of merge conflicts.

Still, I think it would be an improvement over writing the changelog all at once! 😅

[SnoopJ]

As I think about this some more, one of the things that a fragments-based workflow would make harder is the association of changes with individual PRs.

The best way I can see to get association of change notes with PRs is to create the PR first, then write the changelog entry referring to it, and add that to the PR. This would mean either separate commits for the changelogs (messy) or effectively a mandatory rebase/squash to fold that reference juggling into the other commits in a PR's history.

I think I can see a way that the tool could automagically add references to specific commits to the changelog, but I'm not sure how attached we are to the concept of referring to a PR specifically, or about the process I described above. I can at least see how we'd sanity check the process above (i.e. by checking that each top-level bullet point has a PR reference), but this merits some further consideration.

[dgw]

This has been at the back of my mind since it was opened, and I finally have something meaningful to say about it.

I don't think PR authors should be responsible for writing changelog entries, and I especially don't want to force people to install and learn yet another tool (scriv or otherwise). If it's hard or annoying to maintain the PR reference when using such a tool, that's another strike against it for me. I also don't love metadata about the changes being mixed in with the changes themselves, whether the fragment is added in the same or a separate commit from the actual code change.

As I see it, the "unreleased" changelog that @Exirel suggested should be maintained in a long-running branch, and updated when patches land. #1790 demonstrates that that's how it was done for 7.0.0. Why an unreleased-news-8.x branch didn't happen for 8.0.0, I couldn't say. It should have. 🤷‍♂️ (Looking back on #1790 makes me wince a little because now I don't think all 126 of those commits belong in the mainline history, but "spilt milk" etc.)

The downside is that this workflow doesn't have an obvious way to make CI verify that a changelog entry happens. A cron-style GHA workflow could check the NEWS file on a specific branch against merged PRs from the current milestone and "fail" if it's missing any, on a weekly/daily basis… but only because I haven't thought of any better ideas.

[SnoopJ]

The downside is that this workflow doesn't have an obvious way to make CI verify that a changelog entry happens. A cron-style GHA workflow could check the NEWS file on a specific branch against merged PRs from the current milestone and "fail" if it's missing any, on a weekly/daily basis… but only because I haven't thought of any better ideas.

I'm inclined to say that automation is really hard to come by when the changelog entry and the changeset are separated from each other and may not be worth the effort.

As I see it, the "unreleased" changelog that @Exirel suggested should be maintained in a long-running branch, and updated when patches land.

Why a separate branch? If this was being tracked on master, perhaps in a separate file for the 'unreleased' notes, it could be part of the changeset when the PR author is willing to add it. If notes aren't added at changeset time, it's "just" someone else's problem (possibly that same author, but later on), which isn't so far from the status quo. All of that makes sense to me and I think is an improvement on the current workflow, except the part where there are two branches.

At any rate, it sounds like an uphill climb for a fragments-based workflow, I will probably close this issue out once discussion of the alternatives peters out.

[dgw]

As I see it, the "unreleased" changelog that @Exirel suggested should be maintained in a long-running branch, and updated when patches land.

Why a separate branch? If this was being tracked on master, perhaps in a separate file for the 'unreleased' notes, it could be part of the changeset when the PR author is willing to add it. If notes aren't added at changeset time, it's "just" someone else's problem (possibly that same author, but later on), which isn't so far from the status quo. All of that makes sense to me and I think is an improvement on the current workflow, except the part where there are two branches.

What annoyed late-2023-me (and still annoys present-day-me) about how 7.0.0 happened is the "commit vomit" that ended up getting merged into master.

If a non-trivial PR includes change notes, they'll probably be in a separate commit.

If a PR doesn't include change notes, "someone else's problem" means a maintainer amending that PR to include notes prior to merging, or adding another commit with the notes to master after merging.

The long-running separate branch keeps all the changelog drafting work out of mainline, and can be cleanly squashed down (as I just did earlier with #2600) when it's time to prepare for release. It also means that only users who can merge PRs need to know the style guide for writing NEWS entries, and that multiple PRs building on each other toward a larger feature or refactor can be condensed more organically into a single logical changelog entry.^

There are very few cases in which I'd advocate for a long-lived branch, but collecting changelog entries during development is one of them.* I still think the primary fault for 8.0 is not the workflow itself, but the fact that it wasn't adhered to for whatever reason.

---

^ — Some fragment-based tools might support that last one, to be fair; I haven't looked into it. But our human-authored changelogs include a lot of "We did a thing [PR1, PR2, PR3]"–style entries that were done in stages to make code review more manageable, but don't need more than the single line from a downstream user or plugin dev's perspective.

* — Release maintenance is another, for backporting certain bugfixes to the current stable release while we work on the next one—something Sopel's already done for years.