Run tests/build with go work off to catch more problems

[codyoss]

          > As a side note to @eliben @jba I don't understand why go build did not catch this... Does it have to do with the module workspace or am I missing something?

The root repo has a go.work file, which will mask such issues. This is the reason why having a go.work file committed is considered dangerous; when it's committed, it's worth at least running the tests with GOWORK=off to catch such issues.

You can try to run all tests in this repo with GOWORK=off in CI - this may uncover surprises :)

Originally posted by @eliben in #9168 (comment)

[eliben]

It's not documented officially at the moment, AFAIK. The best discussion I'm aware of happens on golang/go#53502 - discussing the various use cases people have of committing go.work and how to mitigate issues.

I'll try to get some topic experts' opinions on this

[bcmills]

Testing with GOWORK=off would catch some problems, but not all of them. Notably:

• Tests that try to write to their working directory may succeed locally but fail when run from the module cache (which is normally read-only) (see cmd/go: document that tests should not write to their working directory golang/go#28386).

• Tests that try to access data in subdirectories located in a different module may fail to locate it. (x/tools/gopls/test/debug: test fails when run in the module cache golang/go#59841 and https://go.dev/cl/196258 are two such examples.)

• Tests that run go list in their own working directory to examine dependencies will see either the dependencies of their own module or the dependencies of a workspace, depending on whether the user has indicated a workspace using the GOWORK environment variable.

Ideally, a project like google-cloud-go that is developing multiple modules together for release versioning should test in at least two configurations: one with the working directory in the module and workspace, and one with the module as seen from outside (compare golang/go#34352).

[bcmills]

Unfortunately, we don't yet have a good story for how to structure the “seen from outside” tests for pre-commit testing. (golang/go#28835 proposes one such mechanism, but it's not clear whether there is a better alternative.)

One option for post-commit testing is to create an external module, use go get to slot in the module to be tested, and then run go test $MOD_DIR/... on the original module directory, to run the tests as seen by the external module. (For pre-commit testing, you would also need to use go mod edit -replace to slot in the module, since go get can't fetch it otherwise.)

That might look something like: https://go.dev/play/p/3GuBSk1lP0p.