cmd/go: work sync and mod vendor don't use other main modules in workspace

[pantsmann]

What version of Go are you using (go version)?

$ go version 
go version go1.18.4 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/{me}/.cache/go-build"
GOENV="/home/{me}/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/{me}/gocode/pkg/mod"
GONOPROXY="github.com/{me}"
GONOSUMDB="github.com/{me}"
GOOS="linux"
GOPATH="/home/{me}/gocode"
GOPRIVATE="github.com/{me}"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.18.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/{me}/gocode/src/a/go.mod"
GOWORK="/home/{me}/gocode/src/a/go.work"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1715927059=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I'm using a go.work file that looks something like this:

go 1.18
use (
     /home/{me}/gocode/src/a
     /home/{me}/gocode/src/b
)

Module a requires module b. I make changes to both a and b and everything works hunky-dory locally. However, my development process includes deploying the code to a google appengine instance on the cloud. I have not gotten this to work yet.

What did you expect to see?

I expected to be able to deploy the same code that is building and running locally to my test GAE instance.

What did you see instead?

Instead, I end up deploying the modified a with the old b which fails to build.

What I have tried

1. running go mod vendor
   • This of course doesn't work because it ignores the go.work file and pulls b from the modules cache based on the old version specified in a's go.mod
   • I would expect this to consult the go.work file or have a command line option to tell it to use the code specified in go.work
2. running go work sync
   • This doesn't work because there is no version tag for the new b code
   • I expected it to update a's go.mod file with a psuedo-version for the git revision in /home/{me}/gocode/src/b
3. added a replace directive in the go.work file replace github.com/{me}/b => /home/{me}/gocode/src/b
   • throws error complaining that there is no version on the left side
4. changed replace directive to be replace github.com/{me}/b v0.0.3 => /home/{me}/gocode/src/b
   • no error, but go work sync doesn't do anything to a's go.mod or add any other files to a.
5. tagged b with v0.0.3-test, remove replace from go.work, run go work sync
   • expected go work sync to find new tagged version, but it didn't
   • no changes to a
6. added new replace directive to go.work replace github.com/{me}/b v0.0.3 => github.com/{me}/b v0.0.3-test
   • this adds go.work.sum file to /home/{me}/gocode/src/a,
   • still can't deploy because go mod vendor seems to ignore this file.
   • I assume if I didn't use a vendor folder and tagged all my private repos and then added replace directives in go.work for each of them then deployed maybe that would work. Unfortunately, workspaces have already lost most of their anticipated benefits by item 3. If I did all the rest of this it would be even more inconvenient than putting those replace directives in the individual go.mod files since with go.work I can't deploy local copies of the code.

Conclusions

I can hear someone say workspaces are only supposed to make local development easier. Ridiculous.

go work sync is worthless. It only updates indirect dependencies in the modules in the workspace, not for the actual modules I'm working on. I would expect the use directives to cause go work sync to either add a pseudo-version or a new tagged version to the go.mod files. It does neither. A replace directive also does neither and doesn't even do as much as replace directives do in the go.mod files.

go mod vendor understands replace directives in the go.mod files that point to local folders. It does not understand any kind of replace directives in the go.work file. It also doesn't understand go.work.sum files. We've been using go mod vendor to avoid all the replace directives for our many private repos, but these shortcomings negate that to large degree.

Even if I do all my testing locally this still leaves me unable to test deployment without tagging new versions and this is still too early to be deciding on what should be added to the next version.

I hope I'm missing something because I was really excited about workspaces and they have been working wonderfully for local development. They seem worthless, though with so much potential, when it comes to anything else. I really hope this can be fixed.

[mknyszek]

CC @matloob

[matloob]

Hi I'm not as familiar with the deploy process for app engine with modules; I've only used it in the old GOPATH mode. What method do you use to deploy with App Engine? Does it use a single module when building?

I'm not sure if this will be useful to you, but we added support for workspace vendoring. You can now create a workspace-specific vendor directory using go work vendor, and that vendor directory will be respected when in the workspace. The reason that I'm not sure it will be useful is that if app engine doesn't upload the full workspace, it won't be used in a build.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)