x/pkgsite: too many users are filing package removal requests

[mvdan]

Say I mistakenly pushed a git tag on a Go module of mine, and now I can see it on pkg.go.dev. I only know the basics about Go modules, so I am not yet familiar with retractions. From the home page of pkg.go.dev, I find https://pkg.go.dev/about#removing-a-package:

If you would like to hide versions of a module on pkg.go.dev, as well as from the go command, you should retract them. Retracting a module version involves adding a retract directive to your go.mod file and publishing a new version. See the Go blog post New module changes in Go 1.16 and the modules reference for details.

If you cannot retract your module, you can file a request for the pkgsite team to remove your package.

Presumably, this should lead me to publish a module retraction, not to file a request. However, plenty of requests are filed regularly. I've always had the suspicion that the majority of those should be fixed with retractions instead, so I engaged with one of them. And indeed I think this particular user wants to use a retraction; they want to entirely take back a v1.0.0 tag from their users, not simply hide the documentation.

Assuming that all of these users are filing issues by following the pkg.go.dev docs as shown above, I think the docs are confusing. They should better clarify that:

1. A retraction is almost always what the user should do. In fact, I struggle to imagine what the site means by "If you cannot retract your module". If we have an edge case in mind, I would call it out explicitly. If not, I would remove the alternative entirely.

2. Removing a package from pkg.go.dev only hides the documentation. The Go module and version will still work for go get pkg@latest requests, which I think is a pretty common concern amongst users. For example, the particular issue above is about having pushed a v1.0.0 tag when the module is meant to be a v0. The wording should be clearer.

   It's true that the link points to an issue template which explains this in an HTML comment, but by that point the user has already made the choice to not retract. Plus, can we trust the user to read the entire template carefully before filling the form? :)

3. How to retract the latest version in a clean way. https://go.dev/ref/mod does say:

   To retract a version, a module author should add a retract directive to go.mod, then publish a new version containing that directive.
   A version containing retractions may retract itself.

   That is, to retract a v1.0.0, it is possible to publish a v1.0.1 which retracts both versions. The user can follow the links to the blog post or docs, but they have to do quite a lot of reading to find the few important sentences.

   In fact, I would remove the link to the blog post, because the link to the modules reference already overlaps with its information. Plus, the example shown in the blog post is rather simple - it does not cover how to retract the latest version, for example.

--

I propose new wording. Lengthier, but hopefully clearer.

If you would like to hide versions of a module on pkg.go.dev, as well as from the go command, you should retract them. Retracting a module version involves adding a retract directive to your go.mod file and publishing a new version. See the modules reference for details.

Note that it is possible to retract the latest version of a module; the link above includes an example. Also note that published versions cannot be reused or modified, and this includes retracted versions.

If you cannot retract your module (TODO: why?), you can file a request for the pkgsite team to hide your package documentation. Note that the package will continue to be available via go get or go install unless its module is retracted.

Note the TODO per the point 1 above.

Also note that I explicitly call out that retracted versions cannot be used again. I think this is a common misunderstanding: inexperienced Go users think that they can remove or retract a Go module or version, to then "start fresh" as if that package or version was never published. That is usually not true. Perhaps this should also be called out in https://go.dev/ref/mod#go-mod-file-retract.

cc @bcmills @seankhliao @julieqiu

[hyangah]

We've seen the case where the repository is already gone so there is no place to publish version for retraction. (e.g. github id change, domain name change, ...) Other than that I don't see the case.

Good idea. Please send a PR.

In the second paragraph, I'd suggest explicitly call out the fact that removal from pkgsite doesn't remove the versions or modules from the module proxy.

Note that it is possible to retract the latest version of a module; the link above includes an example. Also note that published versions cannot be reused or modified, and this includes retracted versions.

[neild]

Perhaps there should be separate section headings for "Removing a package" and "Removing a version of a package", with the latter giving the exact steps to take in the common case of recovering from an accidentally pushed version tag.

[mvdan]

We've seen the case where the repository is already gone so there is no place to publish version for retraction.

I see, thanks. I'll amend the text.

In the second paragraph

Did you mean the third paragraph, where we talk about go get continuing to work?

Perhaps there should be separate section headings for "Removing a package" and "Removing a version of a package"

Personally, I wouldn't do that split. The difference between what a user sees on pkg.go.dev and what they get via go get is already confusing enough. I think we should just aim to solve the common problem that keeps coming up again and again: I published a module or version by mistake and I want to take that back.

If anything, we could call the two sections "removing a package's documentation" and "removing versions of a package", but I think the vast majority of people will want to do more than just hiding the documentation pages.

[firelizzard18]

Maybe it's worth mentioning that you should immediately delete the tag from the remote if you realize you pushed the wrong tag, and realize that very soon after. If you catch it before anyone downloads it, you can actually delete it.

[seankhliao]

At some point I wrote a blog post trying to enumerate the common cases. I've since learned about shorter starting pseudoversions and that retractions work even without matching module names (??).

I get the feeling that the only removal requests that need manual action should be when repos have been removed.

[gopherbot]

Change https://go.dev/cl/443341 mentions this issue: static/frontend: improve package removal documentation

[gopherbot]

Change https://go.dev/cl/443400 mentions this issue: _content/ref/mod.md: add more retraction examples