x/pkgsite: warn if a module regularly breaks semver

[mvdan]

On the right side of the module page there are checkmarks like "stable version" and "redistributable license". I think we should also add a checkmark along the lines of "follows semver". That is, that minor or bugfix versions don't repeatedly make clearly backwards-incompatible changes, like removing entire functions or packages.

Here's an example of such an obvious breakage: https://pkg.go.dev/google.golang.org/grpc/naming

That page does show a This package is not in the latest version of its module warning, but one would usually not find such a warning until after experiencing breakage when updating the module and finding build failures due to the now-missing package.

In other words, if I'm looking at what modules to add as dependencies and I looked at https://pkg.go.dev/google.golang.org/grpc, I'd think twice before adding that module if I saw an alert saying one of the following:

• Last breaking change on date (or the version)
• N v1 releases contain breaking changes (or N% of stable v1 releases)
• N packages have had breaking changes since v1.0.0 (or N% of total packages)

Please realise that I'm only using grpc as an example since it's the one I've struggled with the most recently. The feature would equally apply to any other v1+ module which uses stable semver tags, but which actively breaks the semver compatibility rules.

cc @bcmills @heschik @julieqiu @leitzler

[bcmills]

I would be inclined to only report breaking changes found in the latest patch release of each minor version.

(A breaking change corrected in a subsequent patch release is arguably “just a bug” — it should count toward “frequency of buggy releases” but not “frequency of intentional breaking changes”.)

[mvdan]

That sounds reasonable to me. If a later bugfix/patch release entirely fixed the breakage, I think that should normally make the breakage not matter for go get -u.

only report breaking changes found in the latest patch release

I assume you mean breaking relative to the first release in its major version, not relative to the immediately preceding release. That is, if v1.0.1 removed a package present in v1.0.0, and v1.0.2 is still missing the package, v1.0.2 still contains breaking changes even though it did not direcly remove a package itself.

[bcmills]

I assume you mean breaking relative to the first release in its major version, not relative to the immediately preceding release.

Yes. Specifically, breaking relative to the latest patch release of any preceding minor version.

(If a module author “cheats” by backporting the breaking change to a patch release of every preceding minor version... I guess that's a grey area of SemVer? It's essentially saying “the fact that this ever worked at all is a severe bug”, which is analogous to the “security” exception in the Go 1 compatibility policy.)