cmd/go: [modules + integration] go mod index, reindex a goproxy directory

[nim-nim]

This report is part of a series, filled at the request of @mdempsky, focused at making Go modules integrator-friendly.

Please do not close or mark it as duplicate before making sure you’ve read and understood the general context. A lot of work went into identifying problems points precisely.

Needed feature

Go needs an official go mod index command that processes a goproxy directory and updates its indexes (list files).

Constrains

• the destination should be any directory path the user specifies
• the feature should also be exposed as a function in the official Go API
• it is probably not useful or desirable to be able to operate on a specific goproxy branch …/<module>/@v/, if reindexing is fast enough. Operating on specific branches would be less robust in presence of operator or tooling branch selection mistakes.

Motivation

A goproxy baseline directory may be assembled from the output of several CI/CD jobs. That does not require teaching the CI/CD system deep Go knowledge, just having it pick up the file lists produced by go mod pack (issue #31302).

However, if several of those jobs produce modules for different minor versions of the same module, their lists will collide and conflict. Therefore, the CI/CD system needs a command to regenerate them.

As rewriteVersionList shows, updating indexes is not completely trivial, and requires more Go module knowledge than the typical CI/CD system has.

[agnivade]

@bcmills @jayconrod