staticcheck tried to download a broken URL

[alevchuk]

Host operating system: output of uname -a

Linux base 4.19.25-v7+ #1205 SMP Mon Feb 25 18:19:20 GMT 2019 armv7l GNU/Linux

node_exporter version: output of node_exporter --version

latest checkout of node_exporter 19e5bb6abdfbc69e9f2039485842eb3c0cf87d08

node_exporter command line flags

can't build

Are you running node_exporter in Docker?

no

What did you do that produced an error?

make

What did you expect to see?

make should succeed

What did you see instead?

prometheus@base:~/gocode/src/github.com/prometheus/node_exporter $ make
>> checking code style
mkdir -p /home/prometheus/gocode/bin
curl -s -L https://github.com/dominikh/go-tools/releases/download/2019.1/staticcheck_linux_arm > /home/prometheus/gocode/bin/staticcheck
>> running staticcheck

seems to be there is no ARM for 2019.1 static check https://github.com/dominikh/go-tools/releases/tag/2019.1

[SuperQ]

Oops. I guess we need to have an optional build process, or convince @dominikh to add arm to the build pipeline.

/cc @simonpasquier This is probably an issue for upstream Prometheus as well. I'll give the beta "transfer issue" button a try. 😁

[dominikh]

I'd be happy to add ARM targets. Should we build for ARMv5 (which is supported by v6 and v7 platforms, too), or have one version per possible GOARM? You'd have to change the format of the URL in the latter case.

[SuperQ]

@dominikh This users is on armv7, but we build quite a lot of platforms for Prometheus. You might want to look at promu, which we use for crossbuild publishing.

[SuperQ]

@dominikh What do you think about adding a CircleCI build/publish to go-tools?

[alevchuk]

Hey thanks for quick plan for the fix!

I was gonna say "pulling in binaries of the internet during make makes is hard to review what's being built" yet possibly the opposite is true. Not sure.

Github release builds seem reviewable. E.g. https://github.com/dominikh/go-tools/releases/tag/2019.1 yet there seems to be no view of the commits since last review.

[dominikh]

@dominikh What do you think about adding a CircleCI build/publish to go-tools?

I want people to run fixed releases, not master (unless they want to find issues in staticcheck), and I think I can build the odd release manually just fine ;)

I was gonna say "pulling in binaries of the internet during make makes is hard to review what's being built" yet possibly the opposite is true. Not sure.

There could be any code in the binary, it doesn't have to match at all what the tag says, so that requires trust. Then again, nobody reads the code of the tools they go get, so the effect is the same.

[alevchuk]

Still seems to be broken.

Re-ran make with the latest checkouts:
prometheus/node_exporter@19e5bb6
8155cc4
prometheus/promu@cdcd435

prometheus@base:~/gocode/src/github.com/prometheus/node_exporter $ make
>> checking code style
mkdir -p /home/prometheus/gocode/bin
curl -s -L https://github.com/dominikh/go-tools/releases/download/2019.1/staticcheck_linux_arm > /home/prometheus/gocode/bin/staticcheck
>> running staticcheck
chmod +x /home/prometheus/gocode/bin/staticcheck
GO111MODULE=on go list -e -compiled -test=true -export=false -deps=true -find=false -tags= -- ./... > /dev/null
go: downloading github.com/sirupsen/logrus v1.3.0
go: downloading github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90
go: downloading github.com/soundcloud/go-runit v0.0.0-20150630195641-06ad41a06c4a
go: downloading golang.org/x/net v0.0.0-20190206173232-65e2d4e15006
go: downloading github.com/golang/protobuf v1.2.0
go: downloading github.com/godbus/dbus v0.0.0-20181101234600-2ff6f7ffd60f
go: downloading github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf
go: verifying github.com/soundcloud/go-runit@v0.0.0-20150630195641-06ad41a06c4a: checksum mismatch
        downloaded: h1:TGsOnmXp0mo82KbjaDcsTibGxWIdZNXbKJB18gFn1RM=
        go.sum:     h1:os5OBNhwOwybXZMNLqT96XqtjdTtwRFw2w08uluvNeI=
Makefile.common:141: recipe for target 'common-staticcheck' failed
make: *** [common-staticcheck] Error 1


prometheus@base:~/gocode/src/github.com/prometheus/node_exporter $ cat /home/prometheus/gocode/bin/staticcheck
Not Found

[simonpasquier]

@alevchuk the change to Makefile.common has to be propagated to the various repositories but it should work for github.com/prometheus/prometheus using the master branch. Can you check again for this one?

[alevchuk]

@simonpasquier i'll check prometheus as well (will need to try before and after your fix, because it never saw a break in prometheus). The delay between git push and pull is usually sub-second. Can you tell me what introduces the propagation delay here?

[alevchuk]

I found a workaround for my node_exporter build problem.

prometheus@base:~/gocode/src/github.com/prometheus/node_exporter $ cp  ../prometheus/Makefile.common Makefile.common

checking code style make: *** No rule to make target 'check_license', needed by 'common-all'. Stop.

and remove "check_license" and "unused" from common-all line in Makefile.common

- common-all: precheck style check_license staticcheck unused build test
+ common-all: precheck style staticcheck build test

btw, I tried building prometheus, as requested, yet dropped it because it required "bzr" which pulls in a ton of dependencies

To be honest this build was stressful. Yet, I appreciate the fast root-cause.

[SuperQ]

Yes, I think someone introduced a dependency for one package that uses bzr rather than git to pull source.

[simonpasquier]

Re bzr being needed to build Prometheus, it is commented in the Makefile:

prometheus/Makefile

Lines 22 to 24 in 10ec255

	# Go modules needs the bzr binary because of the dependency on launchpad.net/gocheck.
	$(eval $(call PRECHECK_COMMAND_template,bzr))
	PRECHECK_OPTIONS_bzr = version

[simonpasquier]

The delay between git push and pull is usually sub-second. Can you tell me what introduces the propagation delay here?

To be precise, the canonical version of Makefile.common is maintained on https://github.com/prometheus/prometheus then someone has to open PRs on the other repositories to sync the local version with the canonical one. It is a bit tedious but we haven't spent more time to streamline the process yet...

[SuperQ]

@simonpasquier It seems like launchpad.net/gocheck is abandoned. There is now http://labix.org/gocheck located at gopkg.in/check.v1.

[simonpasquier]

Unfortunately we don't control this dependency as it is pulled indirectly from github.com/hashicorp/consul...

simon@simon-laptop:~/workspace/src/github.com/prometheus/prometheus ✔ $ GO111MODULE=on go mod why  launchpad.net/gocheck                                                                     
go: finding google.golang.org/grpc/transport latest
# launchpad.net/gocheck
github.com/prometheus/prometheus/discovery/consul
github.com/hashicorp/consul/api
github.com/hashicorp/consul/api.test
github.com/hashicorp/serf/serf
github.com/hashicorp/go-msgpack/codec
github.com/hashicorp/go-msgpack/codec.test
labix.org/v2/mgo/bson
labix.org/v2/mgo/bson.test
launchpad.net/gocheck

[SuperQ]

Yea, I was looking at that stack of dependencies. It looks like the latest version of github.com/hashicorp/go-msgpack/codec has eliminated the test that depends on labix.org/v2/mgo/bson, which eliminates the need for launchpad.net/gocheck.

[simonpasquier]

Yes, sending a PR right away.