Automate Changelog Generation

[bhgomes]

This is part of an effort to remove pain points in the merging process, especially as we are growing the team and have a very long CI pipeline that needs to run as smoothly as possible.

To reduce the number of merge conflicts (and so the number of CI-reruns) that occur while merging PRs we should automate the entire changelog generation process. We add a script and a GitHub action to generate/verify the changelog:

Constraints

1. The script may not need commit privileges to the manta repo ( read only access token )
2. We will in the future enforce release-xX.Y.Z-something

Ideas

• One binary, with create and verify mode - alternative:
• One binary, create mode only
  • when creating, you run the script and commit the change to branch
  • when verifying, CI runs the script => new changelog is generated => diff output.md Changelog.md => if diff returns empty, return success

Generate Changelog Diff

1. Find the previous tag.
2. Find all merged PRs since that tag (PR number, PR title, labels)
3. Bucket by L- label, if no label exit
4. New changelog section for the current tag
5. Generate changelog entry per (number, title) pair in its bucket
6. Generate a new changelog footer.

Steps

0. Come up with a sane and "future-proof" changelog layout and labelling system for PRs
1. e.g. https://keepachangelog.com/en/1.0.0
2. Create locally executable tool that creates the modified changelog for us ( may have dependencies to git and gh )
3. know what the next release is going to be - parse the version from the release-vX.Y.Z-something branchname. If the branch name doesn't parse, return failure
4. (Optional) Check that the version number is strictly increasing
5. skip changelog generation and verification if the release PR has a L-nochangelog ????
6. how to get the commits since last release
7. how to fetch the labels
8. how to write those commits to the new Changelog
9. Create verification utility
10. Have a github workflow - may be an action, doesn't have to - that executes the binary from step 1 => store output.md
11. as part of that same workflow - compare output.md with changelog.md, return success if identical
12. Make sure this workflow triggers on release branches only
13. ( Maybe adapt for extra sections in the release notes on GH )

FUTURE:

1. Support withdrawing releases:
   2. check old sections of the changelog
   3. pull their release PR
   4. check if it has L-withdrawn or something like that
   5. rewrite the old section to add [ YANKED ]

Verify Changelog Diff

1. If the branch name is release-vX.Y.Z continue otherwise skip with return value 0
2. Run the generation script and compare equal to the current PR

TODO

• Add L- labels add label requirement to PR merge rules
• Add a gh actions labelling check workflow
  • A PR has been labelled with 1 or more L- labels. It is required to pass for every PR.
  • If branch name starts with release- it must be labelled L-release AND
  • If labelled L-release, branch name must conform to release-vX.Y.Z<-arbitrary_string>. The part in <> is optional, must be lead with a - character and may not contain whitespace
• Update Changelog to conform to https://keepachangelog.com/en/1.0.0 (see the https://github.com/manta-network/manta-rs CHANGELOG.md file) but without the "Unreleased" section
• Add the generation script (can use git and gh as dependencies). May be a shell script or a small Rust CLI program.
• Add the verification action
• Change publish_draft_release to ensure it's being run Add gh action release branch name to Release PR merge rules (has to be release-vX.Y.Z)

[ghzlatarev]

Can also fix the build-changelog job in the publish-draft-release workflow https://github.com/Manta-Network/Manta/blob/manta/.github/workflows/publish_draft_releases.yml#L574