Should there be dedicated release notes?

[alx9r]

I've been tracking a few breaking changes between Pester 3 and 4 that really ought to be highlighted in for other users in release notes for Pester 4. Here is a sampling:

• Get-Command SomeMockedFunction returns less complete object in Pester 4 #810
• [Pester 4.0.3] Call depth overflow on $xmlObject | Should be $xmlObject #785
• $array.Name | Should ... behaves differently between Pester 3 and 4
• Summary of scope-related differences for mocks between Pester 3 and 4 #812

It looks like Pester doesn't really have release notes. The closest thing seems to be CHANGELOG.md, but that's a rather different focus from release notes.

It seems like Pester ought to have explicit release notes. It's a customary way of giving users an idea of what to expect when upgrading versions. Doing so seems especially important for the Pester 3 to 4 transition as I expect there has never been so many deployed Pester tests when a Pester version with breaking changes was released.

I don't have much of an opinion about how the release notes should be published other than that I think they should exist. FWIW I've just been using GitHub's releases page for this project and have been pretty happy with it.

[nohwnd]

Yeah you are right, and it's planned to sum up the changes to release notes. Putting the notes directly to the release instead of putting it to wiki is a good idea.

[alx9r]

It seems like (the same) release notes should appear in (at least) the following places:

• PowerShell Gallery
• Get-Module Pester -ListAvailable | % ReleaseNotes
• GitHub Releases

If there's a PowerShellGet way for users to access release notes for a module without having to install it first, release notes should probably be published through that channel as well.

[nohwnd]

@alx9r I understand your concerns, but at least for the gallery users it should not be that abrupt, because we are pushing Pester there as stable version for a while, and it has tons of downloads.

Adding release notes to the module manifest is already reported, but not done becasue no-one took time to figure out which formats are needed for each publish (github release, module manifest, nuget, choco, psgallery), and then automated that.

I would suggest making the release notes part of annotated git tag (which means I will have to manually craft the release notes, but I guess github will be able to pick it up), and then extracting the info from there to the manifest, and nuspec (or how nuget defines release notes :)

Anyways this seems to be quite a bit of work, so for the current release I would just manually add release notes to the release on github + put it on wiki and link to it from readme, and call it a day. Fine with you? :)

[alx9r]

@nohwnd I see no problem with that. The only thing I feel strongly about is that, beginning with 4.0, there be at least one way to discover release notes that is both obvious and low-cost for users. I think your proposal to publish to the github releases page and link to it from README.md achieves that.

I'd be a bit concerned about the additional maintenance involved with manually publishing the full text of the release notes to the wiki. The "known issues" section is oftentimes populated only some time after release, which means there is the matter of updating the release notes through each channel to consider. Is there a reason to do that rather than just linking to the releases page from the wiki?

FWIW, here is my manual release checklist I have been using for the PowerShell projects in my github:

Release

• review changes since prior release
• decide on new version number according to semver guidelines
• prepare release notes
• add items to changelog.md
• update date in changelog.md
• update version in changelog.md
• update version in module manifest
• commit and push branch
• create pull request
• confirm CI tests pass
• merge pull request
• confirm CI tests pass
• draft new release on github
• publish release
• confirm that documentation link(s) refers to correct release
• delete disused branches from github

Bump Version

• create branch bump-to-version-x.y.(z+1)
• update version in module manifest
• add new version number to changelog.md
• commit and push branch
• confirm CI tests pass
• merge pull request
• confirm CI tests pass

I can see from my records that much of the work I've been spending on releases has been the "review changes since prior release" and "prepare release notes" steps (range is 5 to 45 minutes). Running the remainder of the checklist has averaged around 40 minutes of wall clock time, but I can see that only about half of that was work time with the other half spent waiting for CI to complete. In my case automation could currently potentially save about 20 minutes of work time per release. It would be interesting to hear from you what that number is for Pester. It could be that between our projects it's worth the time to figure out automation for much more of the release process.

[nohwnd]

@alx9r if you are referring to "put it on wiki" then what I meant is that we will put some longer-than-usual info about what has changed between v3 and v4 on wiki, similar to this. And keep that there with note that subsequent releases will have their release notes on the releases page. I don't expect to be writing two pages of text for every release. :)

I think release time for pester is max 10 minutes (+ build time) updating the changelog from commit history, updating versions in files and running tests. When I release I create a commit just for the release notes and tag so I am pretty sure it will build, which removes the need for extra branch. The different places where version needs to be updated are all validated by tests. Then I just tag it, push and CI releases it for me. So if we automate this part I can update changelog and then run release script that will parse out the version and relase notes, put release notes in tag, and than I just push.

[it-praktyk]

Using GithubReleaseManager to automate GitHub releases

[alx9r]

@nohwnd This is a follow-up from your comment on #687. If you want I could draft a set of release notes for 4.0.x according to the philosophy I mentioned.

[nohwnd]

@alx9r That would be great. There should not be any more things added to the v4 milestone unless they are critical.

[alx9r]

@nohwnd

...we will put some longer-than-usual info about what has changed between v3 and v4 on wiki, similar to this...

I think such a long-form document is probably the right idea. Does that exist yet?

[alx9r]

It seems like we've reach consensus that there should be dedicated release notes. Discussion of what and where should continue in #853.