Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: '"bzr": executable file not found in $PATH' when fetching github.com/influxdata/platform #29790

Closed
stevenh opened this issue Jan 17, 2019 · 16 comments

Comments

Projects
None yet
4 participants
@stevenh
Copy link
Contributor

commented Jan 17, 2019

What version of Go are you using (go version)?

go version go1.11.4 freebsd/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
GOARCH="amd64"
GOBIN="/data/go/bin"
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOOS="freebsd"
GOPATH="/data/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/freebsd_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/data/go/src/github.com/multiplay/go/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build442059268=/tmp/go-build -gno-record-gcc-switches"

What did you do?

GO111MODULE=on go get -u=patch

or

GO111MODULE=on go mod vendor

What did you expect to see?

I expected the commands to succeed.

What did you see instead?

go: launchpad.net/gocheck@v0.0.0-20140225173054-000000000087: bzr branch --use-existing-dir https://launchpad.net/~niemeyer/gocheck/trunk . in /data/go/pkg/mod/cache/vcs/f46ce2ae80d31f9b0a29099baa203e3b6d269dace4e5357a2cf74bd109e13339: exec: "bzr": executable file not found in $PATH
go: labix.org/v2/mgo@v0.0.0-20140701140051-000000000287: bzr branch --use-existing-dir https://launchpad.net/mgo/v2 . in /data/go/pkg/mod/cache/vcs/ca61c737a32b1e09a0919e15375f9c2b6aa09860cc097f1333b3c3d29e040ea8: exec: "bzr": executable file not found in $PATH
go: error loading module requirements

The cause of this is we have a dependency on github.com/influxdata/platform which has a dependency in its go.mod file of launchpad.net/gocheck and labix.org/v2/mgo however neither packages are reference by github.com/influxdata/platform

This means that both go get and go mod vendor when run with module support incorrectly assume that intermediate go.mod files are valid.

Even worse is a matching replace in the local go.mod doesn't apply and hence can not be used to override the 3rd party dependency.

This means that the failure of a 3rd party to update their go.mod can prevents any dependent projects from downloading / updating.

@stevenh

This comment has been minimized.

Copy link
Contributor Author

commented Jan 17, 2019

Ok its even worse than I first through, having downloaded github.com/influxdata/platform when I run any module related commands it fails due to the inability to get the dependencies even when I'm trying to get it to update the dependencies.

go-dev:platform> go mod vendor
go: launchpad.net/gocheck@v0.0.0-20140225173054-000000000087: bzr branch --use-existing-dir https://launchpad.net/~niemeyer/gocheck/trunk . in /data/go/pkg/mod/cache/vcs/f46ce2ae80d31f9b0a29099baa203e3b6d269dace4e5357a2cf74bd109e13339: exec: "bzr": executable file not found in $PATH
go: labix.org/v2/mgo@v0.0.0-20140701140051-000000000287: bzr branch --use-existing-dir https://launchpad.net/mgo/v2 . in /data/go/pkg/mod/cache/vcs/ca61c737a32b1e09a0919e15375f9c2b6aa09860cc097f1333b3c3d29e040ea8: exec: "bzr": executable file not found in $PATH
go: error loading module requirements

go-dev:platform> go mod tidy
go: launchpad.net/gocheck@v0.0.0-20140225173054-000000000087: bzr branch --use-existing-dir https://launchpad.net/~niemeyer/gocheck/trunk . in /data/go/pkg/mod/cache/vcs/f46ce2ae80d31f9b0a29099baa203e3b6d269dace4e5357a2cf74bd109e13339: exec: "bzr": executable file not found in $PATH
go: labix.org/v2/mgo@v0.0.0-20140701140051-000000000287: bzr branch --use-existing-dir https://launchpad.net/mgo/v2 . in /data/go/pkg/mod/cache/vcs/ca61c737a32b1e09a0919e15375f9c2b6aa09860cc097f1333b3c3d29e040ea8: exec: "bzr": executable file not found in $PATH
go: error loading module requirements

If I edit the go.mod manually to remove the unneeded dependencies go mod tidy and go mod vendor then work, so it seems to indicate that go mod is incapable of fixing broken dependencies automatically.

@mark-rushakoff

This comment has been minimized.

Copy link
Contributor

commented Jan 17, 2019

We're discussing separately in influxdata/influxdb#11222, but the dependencies are valid albeit very indirect:

go mod why -m launchpad.net/gocheck labix.org/v2/mgo
go: finding github.com/SermoDigital/jose/jwt latest
go: finding github.com/SermoDigital/jose/jws latest
# launchpad.net/gocheck
github.com/influxdata/platform/nats
github.com/nats-io/nats-streaming-server/server
github.com/hashicorp/go-msgpack/codec
github.com/hashicorp/go-msgpack/codec.test
labix.org/v2/mgo/bson
labix.org/v2/mgo/bson.test
launchpad.net/gocheck

# labix.org/v2/mgo
github.com/influxdata/platform/nats
github.com/nats-io/nats-streaming-server/server
github.com/hashicorp/go-msgpack/codec
github.com/hashicorp/go-msgpack/codec.test
labix.org/v2/mgo/bson
@dpinela

This comment has been minimized.

Copy link
Contributor

commented Jan 18, 2019

@stevenh Since exec: "bzr": executable file not found in $PATH appears in your error messages, it seems that the actual problem is that the dependencies are hosted on Bazaar repos, but you don't have Bazaar installed.

The issue with the go tool trying to fetch replaced dependencies is #27859, which is already fixed at tip and also in the 1.12 beta.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 18, 2019

When filing an issue, please title the issue according to the actual symptom you observed, not a hypothesis as to its cause.

The go tool does assume that all of the go.mod files in the module graph are well-formed, but that seems irrelevant here: the problem as reported in the error message is that your PATH does not include a bzr executable.

From influxdata/influxdb#11222 (comment), it appears that your issue has been resolved. Is there anything remaining to do here?

@bcmills bcmills changed the title cmd/go: go get / mod incorrectly assumes intermediate go.mod files are valid cmd/go: '"bzr": executable file not found in $PATH' when fetching github.com/influxdata/platform Jan 18, 2019

@stevenh

This comment has been minimized.

Copy link
Contributor Author

commented Jan 19, 2019

Sorry about the title was trying to be helpful and didn’t think downloads fail when brz is missing correctly conveyed this actual problem, as that's an environment setup issue not a tooling issues which was what this was raised to convey.

You are right in this case brz isn’t in the path, we knew that, but it could have been for any number of reasons e.g repo down, moved etc however that doesn’t change the fact its trying to download things it doesn’t need at it assumes all dependent repos have correct go.mod files.

Even when the influx devs kindly fixed the issue, I would expect the standard flow i.e. to just re-run the go get to work, but unfortunately it still failed. I ended up have to manually update our go.mod file as the tools seem to insist on downloading the current listed versions before it processes any updates.

According to feedback from the influx devs the dependency was for tests only which raises another potential issue that there doesn’t seem to be any way in go.mod to flag a dependency as a test only dependency?

Back to the main issue of dependency go.mod files we all know that the world isn’t perfect so issues with dependencies will happen so there needs to be a way for top level projects to recover from this; which sounds like has been by fix for #27859, we'll need to test.

In this case we were lucky, influx was a direct dependency it could have been many levels deep so we could have been stuck waiting for many repos update before we do anything, which seems to be a serious problem with go.mod

The hope this makes sense?

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 22, 2019

Even when the influx devs kindly fixed the issue, I would expect the standard flow i.e. to just re-run the go get to work, but unfortunately it still failed.

In general go get makes a minimal change to the module graph, but if the module graph can't be loaded then it can't determine what the minimal change is. (Fixes for #26454 and #26474 might make it more tolerant to errors of this sort, particularly if the get targets are strict upgrades.)

@stevenh

This comment has been minimized.

Copy link
Contributor Author

commented Jan 22, 2019

Thanks @bcmills will re-test this flow when 1.12 is released, which seems like it fixing a lot of issues in this area, and will update the thread based on that.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 22, 2019

According to feedback from the influx devs the dependency was for tests only which raises another potential issue that there doesn’t seem to be any way in go.mod to flag a dependency as a test only dependency?

A test dependency is a dependency. We intend for go mod tidy to leave the go.mod file in a state such that go test all remains reproducible (see #27920 (comment)).

That said, your dependency on launchpad.net/gocheck may be a bug: it seems to be for a test of a package that is not itself in the transitive import graph, which (as far as I am aware) isn't required for go test all.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 22, 2019

a package that is not itself in the transitive import graph, which (as far as I am aware) isn't required for go test all.

I was mistaken: the all target currently includes transitive dependencies-of-tests-of-dependencies, so at least they're consistent.

@stevenh

This comment has been minimized.

Copy link
Contributor Author

commented Jan 22, 2019

So just to confirm my understanding; are you saying that that building any project which is go.mod enabled will have to download the dependencies for all tests of dependencies?

e.g.
mypackage -> dependency1 -> depenency2 (dependency of only tests for dependency1)

If so this sounds like projects are going to suffer from dependency bloat quickly and could even break if said dependencies aren't publicly available.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 22, 2019

Depends on what you mean by “download the dependencies”.

You will need to download the go.mod files for all of the modules in the module graph, which includes modules that provide dependencies for tests of dependencies. However, you will not need to download the source code for those modules unless you actually execute a build from those sources (for example, using go test all).

@stevenh

This comment has been minimized.

Copy link
Contributor Author

commented Jan 22, 2019

Thanks for clarifying, so it sounds like size bloat wont be a direct issue then, but this could still easily lead to issues when test only dependencies aren't available publicly or there's some other issue downloading go.mod files for those dependencies.

Do you think it would make sense to make said failures a warning and not fatal error, which they appear to be now?

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 24, 2019

Do you think it would make sense to make said failures a warning and not fatal error, which they appear to be now?

No, they cannot be warnings. In general we do not know how the missing modules might impact the selected versions of other modules in the build graph, so if we just drop those modules we might (more-or-less silently) end up with a different configuration from what we expected for the given version.

Reproducibility is a critical feature of modules, so we shouldn't degrade to a non-reproducible configuration without explicit user intervention.

@stevenh

This comment has been minimized.

Copy link
Contributor Author

commented Jan 24, 2019

Do you see any solution to the specific problem of test only dependency causing failures such as this?

Pretty much every other package manager has the concept of separate runtime and build / test dependencies. Which when translated to golang, as a development language, would mean separate build and test dependencies; without which there is a real risk of 3rd party dependencies resulting in failure as demonstrated by what we saw here.

Other areas of the golang tool chain clearly know the difference between normal code and test code so it’s seems reasonable to continue this.

With this is mind I believe we need a clear method of mitigating such an issue if the tool is to be successful and not cause frustration that would prevent adoption.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 24, 2019

The solution currently on the roadmap is to provide a module proxy that can cache and serve go.mod files, so most users will be able to fetch those directly from the proxy (via HTTPS) rather than installing more esoteric tools (like bzr) or relying on origin servers that may or may not be reliable.

We're certainly keeping an eye on issues surrounding test dependencies, but at the moment we do not plan to separate test (or tool) dependencies from other general module dependencies.

@bcmills bcmills removed the WaitingForInfo label Jan 24, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Jan 29, 2019

As far as I can tell there are no remaining direct actions to take for this issue.

We can revisit the story around transitive dependencies once we see how the use of module proxies shakes out.

@bcmills bcmills closed this Jan 29, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.