Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: duplicate symbols when including two main packages in -coverpkg in module mode #30374

Closed
drakonka opened this issue Feb 24, 2019 · 25 comments

Comments

Projects
None yet
@drakonka
Copy link

commented Feb 24, 2019

Note that I am not sure if this is a bug or expected behaviour, only that this does not happen in the latest stable 1.11 release and does on 1.12rc1.

What version of Go are you using (go version)?

$ go version
go version go1.12rc1 linux/amd64

Does this issue reproduce with the latest release?

Not the latest stable release (1.11.5)

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/lazer/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/lazer/go"
GOPROXY=""
GORACE=""
GOROOT="/home/lazer/sdk/go1.12rc1"
GOTMPDIR=""
GOTOOLDIR="/home/lazer/sdk/go1.12rc1/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/lazer/Dev/gosnaillife/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build721704492=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I ran the following command:

go1.12rc1 test -race -v -coverpkg gitlab.com/drakonka/gosnaillife/cmd/snaillifecli,gitlab.com/drakonka/gosnaillife/cmd/snaillifesrv, -coverprofile allcoverage-rc.out ./...

Both of the above are main packages producing two binaries.

What did you expect to see?

I expected the tests to run and pass, with coverage output, as they do on Go 1.11.5

What did you see instead?

The following error:

2019/02/24 11:39:04 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gitlab.com/drakonka/gosnaillife/cmd/snaillifecli and $WORK/b293/_pkg_.a(_go_.o)

I run Go RC and Go 1.11 side by side as part of my CI, which is how I noticed this issue. I am able to reproduce it locally as well. Full logs for the successful run of Go 1.11 and the failure run of Go 1.12rc1 can be found here:

@mvdan

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

I found the same regression just a few days ago, but hadn't filed an issue yet. This is impeding my daily development, and I'd consider it a regression introduced in 1.12.

The 1.12 release will be out any day now though, so it's probably more realistic to target a backport to 1.12.1. /cc @bcmills @ianlancetaylor @hyangah since this very likely relates to the recent modinfo changes in the cycle.

@mvdan mvdan added this to the Go1.12 milestone Feb 25, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

I'd consider it a regression introduced in 1.12.

This is almost certainly a bad interaction between #23910, //go:linkname, and the fix for #27584. Note that the former dates to at least Go 1.10, and can be a problem for any program that happens to use //go:linkname in package main (although those are hopefully rare).

Before the fix for #27584, the error (and redefinition) still existed, but because the variable was initialized in an init function, instead of a compile-time error one of the values of debug.modinfo would be overwritten at run-time.

@bcmills

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

Given that we do not currently merge coverage profiles from multiple processes (#28235), I'm not sure why you would want to include two different main packages in the same -coverpkg argument (except, perhaps, to simplify the patterns passed to -coverpkg).

The workaround for now is to include at most one main package in the -coverpkg arguments, or to run coverage in GOPATH mode.

@bcmills

This comment has been minimized.

Copy link
Member

commented Feb 25, 2019

@bcmills bcmills changed the title Duplicate symbol error when running go test with two main packages in -coverpkg cmd/go: duplicate symbols when including two main packages in -coverpkg in module mode Feb 25, 2019

@jayconrod jayconrod self-assigned this Feb 25, 2019

@andybons andybons modified the milestones: Go1.12, Go1.13 Feb 25, 2019

@schoentoon

This comment has been minimized.

Copy link

commented Feb 27, 2019

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

@drakonka

This comment has been minimized.

Copy link
Author

commented Feb 27, 2019

Seems very similar to the issues I had when Golang 1.11 came out #23910. I guess I'm just disabling -coverpkg on CI for now.

Depending on the details of your setup that seems like it might be overkill when you can just remove the conflicting main packages from -coverpkg and still get coverage reporting, right?

@markus-wa

This comment has been minimized.

Copy link
Contributor

commented Feb 28, 2019

Same issue on my end https://travis-ci.org/markus-wa/demoinfocs-golang/jobs/499992529

I see the milestone is 1.13, does that mean it won't be fixed in 1.12.x?

@mvdan

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@bcmills I didn't know this was silently broken in 1.11; thanks for pointing this out. I think temporarily working around the issue for now is acceptable, but perhaps we should provide a simple example or one-liner that people could switch to.

@bcmills

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

@mvdan , the one-liner is GO111MODULE=off. (There is a reason I wasn't comfortable turning it on-by-default in 1.12, and bugs like this one are that reason: they're unusual but not esoteric.)

@mvdan

This comment has been minimized.

Copy link
Member

commented Feb 28, 2019

Thanks for the tip! If the bug(s) can be fixed in time for 1.13, then the timing with GO111MODULE=on would be ideal :)

@markus-wa

This comment has been minimized.

Copy link
Contributor

commented Mar 1, 2019

the one-liner is GO111MODULE=off

@bcmills but with that set to off, go, understandably, isn't able to find dependencies. - .travis.yml

Am I missing something obvious here?

I'm not sure why you would want to include two different main packages in the same -coverpkg argument

In my case I have a couple of examples for my lib and each of them has it's own main, then I just do go test -coverpkg=./... ./...

@gopherbot

This comment has been minimized.

Copy link

commented Mar 1, 2019

Change https://golang.org/cl/164877 mentions this issue: cmd/go: avoid link errors when -coverpkg covers main packages

@sjansen

This comment has been minimized.

Copy link

commented Mar 1, 2019

I was bitten by the same problem: multiple main packages in subdirectories below examples/.

If you're slightly insane, here's a work around:

go test -race \
    -coverpkg .,$(
        ls -1d */ \
        | sed -E \
            -e :a \
            -e '$!N ; s|\n|,| ; ta' \
            -e 's#dist/|docs/|examples/|scripts/##g' \
            -e 's/,,/,/g ; s/^,// ; s/,$//' \
            -e 's|/|/...|g'
    ) \
    -coverprofile=dist/coverage.txt \
    -tags integration ./...
@markus-wa

This comment has been minimized.

Copy link
Contributor

commented Mar 2, 2019

@sjansen You can also use go list instead, if that's more sane.

go test -v -coverprofile=coverage.out -coverpkg=$(go list ./... | grep -v 'examples/' | awk -vORS=, '{ print $1 }' | sed 's/,$/\n/')

I'm still a bit bummed to lose coverage on the examples though 😄.

@drakonka

This comment has been minimized.

Copy link
Author

commented Mar 2, 2019

I've been doing it like this:

CVPKG=$(go list ./... | grep -v -e mocks -e snaillifecli -e snaillifesrv | tr '\n' ',') 

go test -race -v -coverpkg $CVPKG -coverprofile allcoverage-stable.out ./... 

peterzen added a commit to peterzen/goresolver that referenced this issue Mar 3, 2019

@bcmills

This comment has been minimized.

Copy link
Member

commented Mar 8, 2019

@gopherbot, please backport to 1.12. This is a significant regression from GOPATH mode and the fix is straightforward.

@gopherbot

This comment has been minimized.

Copy link

commented Mar 8, 2019

Backport issue(s) opened: #30684 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@gopherbot gopherbot closed this in 10156b6 Mar 8, 2019

@jba

This comment has been minimized.

Copy link
Contributor

commented Mar 18, 2019

I can still reproduce this in 1.12.1:

> docker run -it golang:1.12.1
# cd ..
# git clone https://github.com/google/go-cloud
# cd go-cloud
# export GO111MODULE=on
# go test -coverpkg=./... ./...
...
2019/03/18 12:52:58 duplicate symbol runtime/debug.modinfo (types 30 and 30) in gocloud.dev/samples/blobdownload and $WORK/b466/_pkg_.a(_go_.o)  

(Note: I had to run the go test line several times because I would get occasional "Could not resolve host: github.com" errors. Not sure whether that's specific to my setup.)

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented Mar 18, 2019

Reproduced. Trying to narrow it down.

It's possible this is a new issue. The _go_.o part of the message is suspicious. I don't think that file should be getting linked.

@jba

This comment has been minimized.

Copy link
Contributor

commented Mar 18, 2019

It might be useful to know that this repo has nested modules.

@gopherbot

This comment has been minimized.

Copy link

commented Mar 18, 2019

Change https://golang.org/cl/168200 mentions this issue: cmd/go: avoid link error when -coverpkg covers main packages (more)

@gopherbot gopherbot closed this in d34548e Mar 19, 2019

@jayconrod

This comment has been minimized.

Copy link
Contributor

commented Mar 19, 2019

@gopherbot Please backport to 1.12. This is a more complete fix for an issue that was supposed to be fixed in 1.12.1.

@dmitshur

This comment has been minimized.

Copy link
Member

commented Mar 21, 2019

For posterity, the second backport issue was created manually: #30937 (for 1.12).

@imsodin

This comment has been minimized.

Copy link

commented May 17, 2019

I am encountering errors on go1.12.5 just like previously reported.

$ go version
go version go1.12.5 linux/amd64
go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/simon/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/media/ext4_data/Coding/go"
GOPROXY=""
GORACE=""
GOROOT="/media/ext4_data/Linux/source/go"
GOTMPDIR=""
GOTOOLDIR="/media/ext4_data/Linux/source/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/simon/tmp/syncthing/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build248314634=/tmp/go-build -gno-record-gcc-switches"

Running all tests with -coverpkg on all packages

git clone https://github.com/syncthing/syncthing.git
cd syncthing
go test -short -coverprofile coverage.out -coverpkg ./cmd/...,./lib/... ./cmd/... ./lib/...

results in errors like previously reported:

2019/05/15 16:02:50 duplicate symbol go.constinfo.main (types 45 and 45) in github.com/syncthing/syncthing/cmd/strelaypoolsrv and /home/simon/.cache/go-build/87/87afda35dba0b8de44e2f9d449eda7ce4caef7ed781612883b4e457eab1ee3d5-d(_go_.o)

The same also happens with just ./.... I cleaned out ~/.cache/go-build for good measure, didn't change anything.

To make sure it isn't some weird local problem of mine I also ran it on Syncthing's CI and reproduced the issue there as well: https://build.syncthing.net/viewLog.html?buildId=39242&buildTypeId=Syncthing_Coverage&tab=buildLog

@bcmills

This comment has been minimized.

Copy link
Member

commented May 20, 2019

@imsodin, please file a separate issue — there may be a similar bug involving the -coverprofile flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.