cmd/go: [modules + integration] go mod foreign, list the foreign content of a module

[nim-nim]

This report is part of a series, filled at the request of @mdempsky, focused at making Go modules integrator-friendly.

Please do not close or mark it as duplicate before making sure you’ve read and understood the general context. A lot of work went into identifying problems points precisely.

Needed feature

Go needs an official go mod foreign command that lists the foreign content of a packed or unpacked Go module.

Constrains

• foreign content is anything the Go compiler has no use for in the Go module
  • everything except the Go module source files
• while CompiledGoFiles and SysoFiles can be used by the compiler, they are opaque blobs that can not be checked by code analysis tools, and present a risk of hostile code injection; they should be listed too (with warnings)
• the Go project should decide if XTestGoFiles are part of the module code or something foreign that happens to be located in the module tree
• the command should take a mod module descriptor filsesystem path as input.
• the command needs to operate both on:
  • unpacked module trees (with go.mod located at the tree root) and
  • packed modules (with the module descriptor filename matching the zip payload filename)
• the output should be machine-readable
• the feature should also be exposed as a function in the official Go API, returning a go object (map, list, struct…)

Motivation

Existing Go projects have been created before Go module existed, when the Go world was a vast GOPATH soup directly accessible on the filesystem, without clear delimitations.

Converting Go projects to modules by plunking a mod file in the project root, and zipping everything below this root, works as a first approximation. It does not help projects and integrators to triage things that made sense when everything was a single filesystem, and make no sense now that they’re segregated in a module zip file.

While the Go project should not be responsible nor care about anything, except Go source code located within Go modules, it should help projects, by listing other module files, so they can be managed by tools separate from the Go compiler.