Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: crypto/elliptic build error under -linkshared mode #58966

Closed
quaresmajose opened this issue Mar 10, 2023 · 11 comments
Closed

cmd/compile: crypto/elliptic build error under -linkshared mode #58966

quaresmajose opened this issue Mar 10, 2023 · 11 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@quaresmajose
Copy link

quaresmajose commented Mar 10, 2023

What version of Go are you using (go version)?

$ go version
go version go1.20.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

This issue can be reproduced inside the official golang:1.20-alpine docker container and it works fine on the previous golang:1.19-alpine

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/build/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3925415831=/tmp/go-build -gno-record-gcc-switches"

What did you do?

This error has been discovered in Yocto/OpenEmbedded that builds the go shared std library and this issue also affect
other go projects [1], the workaround to build successfully since go 1.20 is not using the -linkshared

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310

Steps to reproduce:

# fetch some affected source
git clone https://github.com/foundriesio/fioconfig && cd fioconfig

# go-1.19 is good
cat > Dockerfile.shared << EOF
FROM golang:1.19-alpine
RUN mkdir /build
WORKDIR /build
RUN apk add --update --no-cache g++
COPY . .
RUN go mod download
RUN go install -buildmode=shared -linkshared std
RUN go build -linkshared -tags vpn -o ./fioconfig main.go
EOF
docker build . -f Dockerfile.shared

# go-1.20 is broken
cat > Dockerfile.shared << EOF
FROM golang:1.20-alpine
RUN mkdir /build
WORKDIR /build
RUN apk add --update --no-cache g++
COPY . .
RUN go env
RUN go mod download
RUN go install -buildmode=shared -linkshared std
RUN go build -linkshared -tags vpn -o ./fioconfig main.go
EOF
docker build . -f Dockerfile.shared

What did you expect to see?

Build my go linkshared application without any errors like on previous version 1.19.

What did you see instead?

Linking errors

# command-line-arguments
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Add
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Add
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).CombinedMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).CombinedMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Double
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Double
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).IsOnCurve
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).IsOnCurve
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Params
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Params
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarBaseMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarBaseMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).ScalarMult
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Unmarshal
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Unmarshal
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).UnmarshalCompressed
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).UnmarshalCompressed
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).normalizeScalar
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).normalizeScalar
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).pointFromAffine
/usr/local/go/pkg/tool/linux_amd64/link: too many errors
The command '/bin/sh -c go build -linkshared -tags vpn -o ./fioconfig main.go' returned a non-zero code: 1
@quaresmajose
Copy link
Author

quaresmajose commented Mar 10, 2023

It looks like the crypto/elliptic is now no more than a compatibility wrapper and the new crypto/ecdh don't implement the P-224 curves [2].

[2] https://words.filippo.io/dispatches/go-1-20-cryptography

@FiloSottile is the author of the blog post so he maybe know better what going on.

@cagedmantis cagedmantis added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 10, 2023
@cagedmantis cagedmantis added this to the Backlog milestone Mar 10, 2023
@cagedmantis
Copy link
Contributor

cc @rolandshoemaker @golang/security

@mdempsky
Copy link
Contributor

mdempsky commented Mar 14, 2023

Standalone repro:

$ cat x.go
package main

import "crypto/elliptic"

var curve elliptic.Curve

func main() {
	switch curve {
	case elliptic.P224():
	}
}
$ go install -linkshared -buildmode=shared std
$ go build -linkshared x.go
# command-line-arguments
type:*crypto/elliptic.nistCurve[*crypto/internal/nistec.P224Point]: unreachable sym in relocation: crypto/elliptic.(*nistCurve[*crypto/internal/nistec.P224Point]).Add
...

I think this is a compiler or linker issue, not an issue with crypto/elliptic itself.

I expect the issue has to do with instantiating generic types across DSO boundaries (due to inlining).

/cc @golang/compiler

@mdempsky
Copy link
Contributor

@cherrymui gave this explanation:

Okay, I think I understand it a little bit. We just don't support a type symbol referencing a method symbol across DSO boundary. That reference, R_METHODOFF, is relative to the section start. But we cannot do it across the DSO boundary, because we can't reference the section in a different DSO. Also, the runtime code resolves the section-relative offset to the same runtime module that contains the type symbol.

So, either the type symbol and the method symbol are both in the shared object, or both in the main executable. But not cross.

@mdempsky mdempsky changed the title crypto/elliptic: linkshared regression in crypto/internal/nistec.P224Point since version 1.20 cmd/compile: crypto/elliptic build error under -linkshared mode Mar 14, 2023
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Mar 14, 2023
@cuonglm
Copy link
Member

cuonglm commented Mar 15, 2023

FYI, this is started failing since https://go-review.googlesource.com/c/go/+/443535 (with 18baca6 and de6abd7 applied for fixing go install -linkshared -buildmode=shared std)

@quaresmajose
Copy link
Author

The described steps to reproduce this issue uses GOARCH="amd64" but please note that it can be reproduced on other architectures too. At last I can reproduce the issue on arm and arm64 so maybe this is architecture independent.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/476735 mentions this issue: cmd/link: improve error message and debugging

@cuonglm
Copy link
Member

cuonglm commented Mar 17, 2023

@cherrymui gave this explanation:

Okay, I think I understand it a little bit. We just don't support a type symbol referencing a method symbol across DSO boundary. That reference, R_METHODOFF, is relative to the section start. But we cannot do it across the DSO boundary, because we can't reference the section in a different DSO. Also, the runtime code resolves the section-relative offset to the same runtime module that contains the type symbol.
So, either the type symbol and the method symbol are both in the shared object, or both in the main executable. But not cross.

@mdempsky maybe when building with -linkshared, we should always apply InlineCalls even the instantiated function was compiled in imported package?

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/477375 mentions this issue: cmd/compile: re-compile instantiated generic functions in linkshared mode

gopherbot pushed a commit that referenced this issue Mar 21, 2023
Correct an error message to missing section, not unreachable
symbol.

Also, under -v >= 2, dump symbol info on error for debugging.

Updates #58966.

Change-Id: I0f832c517d64f4b672b313a8b9be2d028744f945
Reviewed-on: https://go-review.googlesource.com/c/go/+/476735
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
quaresmajose added a commit to quaresmajose/openembedded-core that referenced this issue Mar 22, 2023
Since go 1.20 some packages fails to link [1][2]
and only build staticaly with GO_LINKSHARED = ""

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310
[2] golang/go#58966

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
kraj pushed a commit to YoeDistro/poky that referenced this issue Mar 23, 2023
Since go 1.20 some packages fails to link [1][2]
and only build staticaly with GO_LINKSHARED = ""

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310
[2] golang/go#58966

(From OE-Core rev: afb3fc1701988dfcd79191343b49f4db0966236a)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
kraj pushed a commit to YoeDistro/poky that referenced this issue Mar 23, 2023
Since go 1.20 some packages fails to link [1][2]
and only build staticaly with GO_LINKSHARED = ""

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310
[2] golang/go#58966

(From OE-Core rev: d6a010a9d62fc1ae75935c0d0664b8353eede756)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
kraj pushed a commit to YoeDistro/poky that referenced this issue Mar 23, 2023
Since go 1.20 some packages fails to link [1][2]
and only build staticaly with GO_LINKSHARED = ""

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310
[2] golang/go#58966

(From OE-Core rev: d6a010a9d62fc1ae75935c0d0664b8353eede756)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
rpurdie pushed a commit to yoctoproject/poky that referenced this issue Mar 23, 2023
Since go 1.20 some packages fails to link [1][2]
and only build staticaly with GO_LINKSHARED = ""

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310
[2] golang/go#58966

(From OE-Core rev: eb40d6c2ef72814336365208722e2dc96322e07d)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
halstead pushed a commit to openembedded/openembedded-core that referenced this issue Mar 23, 2023
Since go 1.20 some packages fails to link [1][2]
and only build staticaly with GO_LINKSHARED = ""

[1] https://lists.yoctoproject.org/g/meta-virtualization/topic/97182310
[2] golang/go#58966

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
@cuonglm
Copy link
Member

cuonglm commented Mar 25, 2023

@gopherbot please backport this issue to go1.20

@gopherbot
Copy link
Contributor

Backport issue(s) opened: #59236 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

quaresmajose added a commit to quaresmajose/meta-virtualization that referenced this issue Mar 27, 2023
This reverts commit d2a630c.

The linkshared is fixed upstream [1] and backported in oe-core [2]

[1] golang/go#58966
[2] https://git.yoctoproject.org/poky/commit/?id=d5c79ca6a77af1a04992fff4300333e02d94d84d

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
quaresmajose added a commit to quaresmajose/meta-virtualization that referenced this issue Mar 27, 2023
The linkshared is fixed upstream [1] and backported in oe-core [2]

[1] golang/go#58966
[2] https://git.yoctoproject.org/poky/commit/?id=d5c79ca6a77af1a04992fff4300333e02d94d84d

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
@dmitshur dmitshur modified the milestones: Backlog, Go1.21 Mar 29, 2023
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Mar 29, 2023
@golang golang locked and limited conversation to collaborators Mar 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

7 participants