Automate changelog updates

[MisRob]

Current workflow

1. When opening a PR, developers are guided by the PR template to fill in the changelog section in the PR description
2. Some time before the PR is merged, the changelog section from the PR description needs to be manually pasted to CHANGELOG. md. Otherwise, the changelog check action will fail on the PR and it can't be merged.

Expected automated workflow

1. (same) When opening a PR, developers are guided by the PR template to fill in the changelog section
2. Before merging a PR, the changelog check action checks that the changelog section is present in the PR template
3. After merging a PR, the CHANGELOG.md in the main branch gets automatically updated with the changelog section from the just merged PR's decription

This will also mean that developers won't need to fill the PR number and link anymore since that could too be done by means of automation during updating CHANGELOG.md.

Benefits

• Prevents conflicts on PRs
• Easier PR review and merge process

Tasks

Beta Give feedback

1. Update the changelog check action to check for presence of the changelog item in the PR description (instead of currently checking for the presence of the CHANGELOG.md file in the PR files)
2. Remove the PR number and link from the PR template (so that PR authors don't need to fill it anymore)
3. Create a new action that will copy the changelog item from the PR description and pastes it to the CHANGELOG.md after the PR is merged. It pastes the changelog item above all other changelog items and it adds the PR number and the link. If a PR is open by dependabot, update CHANGELOG.md with a new item that has the PR title in the "Description", "Dev Depency upgrade" in "Products impact", and "-" in all remaining fields.

Guidance

• For contributors who don't have admin permissions on our repositories, I recommend using their own test repository during the development process.
• @rtibbles posted some tips motivated by another action but may apply here as well:

...I strongly suspect that our attempts to update a PR from an event triggered on a PR will continue to fail, due to the scoped permissions of the github token in the particular events that we are responding to. This is the same reason that for PR builds on Kolibri, I had to execute on completion of the workflow, rather than in the main build workflow (triggered by the PR): https://github.com/learningequality/kolibri/blob/release-v0.16.x/.github/workflows/pr_build_comment.yml#L4

Noting that depending on which events we're triggering on, this might affect automatic changelog updating also.

[KshitijThareja]

Hi @MisRob. I understand that the team is yet to make decisions with regards to this issue, but it seems pretty interesting to me and I'd like to work on it if and whenever it is up for contributions. Please let me know if I can help with this one. Thanks :)

[rtibbles]

I think we could make a special case for the dependabot PRs, and directly template the update to the CHANGELOG in the case of that specific user.

[MisRob]

@rtibbles What would we do if a developer needed to have some input into it? (e.g. marking a breaking change) Would they need to open another PR to update that changelog item? I am not sure how often that will happen though.

Do you think that having an action add a comment on PRs wouldn't be helpful in this case? Originally, I suggested it to resolve the situation when people remove the changelog section from the PR description (happens) but then realized it could be applied to dependabot ones too. Reading your comment, for dependabot we could prefill the comment much more than for regular PRs.

[MisRob]

Thanks @KshitijThareja, that'd be lovely! I will let you know after we figure out details. Contributors sometimes have trouble with GH actions due to permissions, but you could play around with it in your own test repository.

[MisRob]

@rtibbles One disadvantage of using an automated comment is that contributors probably won't realize they need to fill it in. But well, each approach has its own issues, so not sure if it's such a big deal. Did you have any ideas how to deal with the deleted changelog section from the PR description?

[rtibbles]

For dependabot, we can still manually edit the changelog after the fact, if it is breaking (on the PR itself).

I think we could make the action still error if the information is not filled in?

[MisRob]

For dependabot, we can still manually edit the changelog after the fact, if it is breaking (on the PR itself).

Oh wait, from the previous discussions, I thought that CHANGELOG.md should be updated after we merge the PR, because of the conflicts. So I assumed that and tried to be careful about making sure everything's in place before we merge. I am not sure if I understood it well now.

[rtibbles]

Oh true - that would make it easier in general. Right, the dependabot would be a bit of an edge case, and we'd have to follow up after merge if it needed editing.

We would probably want to fail if someone hadn't filled out the template in the PR though, as we want that filled in to make the automatic update.

[MisRob]

Right, the dependabot would be a bit of an edge case, and we'd have to follow up after merge if it needed editing.

If we had an action that would post a comment to the PR with the changelog section (rather than having it in the PR description), than even on dependabot PRs we could still adjust it before merge. After the merge, an another action would take it and paste to the CHANGELOG.md.

[MisRob]

And external contributors couldn't remove this comment from regular PRs. So it'd solve missing changelog section. However, I'm not sure if they could edit it, actually... So perhaps it's a bad idea.

[MisRob]

We would probably want to fail if someone hadn't filled out the template in the PR though, as we want that filled in to make the automatic update.

By this you meant that we would update the current changelog check action to check for the presence of the changelog item in the PR description (rather than checking on CHANGELOG.md)?

[rtibbles]

Yes.

[MisRob]

Okay, I will update the issue. I realized that the comment idea wouldn't work due to permissions for majority of PRs. Hopefully we won't have braking changes on dependabot PRs too often. And the template check will prevent from missing changelog item. So sounds like a path forward. Thanks for discussing.

[MisRob]

Updated! @rtibbles if you'd like to read through the final version, I'd appreciate it.

[MisRob]

@KshitijThareja After @rtibbles confirms, I think you'd be welcome to give it a try. Please let us know if it's all clear. GH actions development can be tricky, I heard :) As mentioned, my recommendation would be to try it on your own test repository to prevent from permissions troubles.

[MisRob]

Alright, we agreed this is ready to go. @KshitijThareja I updated the issue a lot. If you're interested, let us know and I can assign you. Thank you!

[KshitijThareja]

Sure @MisRob, I'm interested in working on this issue.

[MisRob]

Thanks @KshitijThareja, I will assign you then.

[KshitijThareja]

Hi @MisRob!
Just an update on this issue. I've completed most of the work. I've updated the the changelog check action to check for presence of the changelog item in the PR description. I am almost done with changelog update workflow and currently testing it in my own repository.
I expect to make a PR this week, if all goes well.

[MisRob]

Hi @KshitijThareja, thanks, that's wonderful

[KshitijThareja]

Hi @MisRob
I've linked a PR for this issue. I'd love to hear your feedback on the changes introduced.
Thanks : )