Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cmd/link: issues with Apple's new linker in Xcode 15 beta [1.20 backport] #62597

Closed
gopherbot opened this issue Sep 12, 2023 · 13 comments
Closed

cmd/link: issues with Apple's new linker in Xcode 15 beta [1.20 backport] #62597

gopherbot opened this issue Sep 12, 2023 · 13 comments
Labels
CherryPickApproved Used during the release process for point releases compiler/runtime Issues related to the Go compiler and/or runtime.
Milestone
Go1.20.9

Comments

@gopherbot
Copy link
Contributor

@cherrymui requested issue #61229 to be considered for backport to the next 1.20 minor release.

@gopherbot please backport this to Go 1.20 and 1.21 releases. I'll fill in the details in the corresponding issues. Thanks.
@gopherbot gopherbot added the CherryPickCandidate Used during the release process for point releases label Sep 12, 2023
@gopherbot gopherbot mentioned this issue Sep 12, 2023
Closed
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Sep 12, 2023
@gopherbot gopherbot added this to the Go1.20.9 milestone Sep 12, 2023
@cherrymui
Copy link
Member

cherrymui commented Sep 12, 2023 •
edited
Loading

Xcode 15 (to be released some time soon) will include Apple's new linker, enabled by default. There are a number of issues with running Go 1.20 with the new Apple linker, from linker warnings to program fail to run. There are a number of fixes/workarounds included in Go 1.21 and tip, but I think they are too many to backport. So for 1.20, I propose we force to use the old Apple linker. I'm pretty sure the old Apple linker will be around for long enough time to cover the lifetime of Go 1.20.

We also want to backport CL https://go.dev/cl/527415, to disable DWARF by default for c-shared mode (otherwise the new Apple linker doesn't accept the c-shared object). We need this because the user may link the c-shared object with C code without going through the Go toolchain, so can't force old linker for this.

We also want to backport CL https://go.dev/cl/508696, which suppresses a warning. Apparently, although the old linker doesn't emit the warning and we pass a flag to force the old linker, it still emits the warning, probably because the new linker parses flags before exec'ing the old linker.

Specifically, CLs to be included in Go 1.20 branch:

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/527798 mentions this issue: [release-branch.go1.20] cmd/link: suppress -bind_at_load deprecation warning for ld-prime

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/527818 mentions this issue: [release-branch.go1.20] cmd/link: force old Apple linker in plugin mode

@gopherbot
Copy link
Contributor Author

Change https://go.dev/cl/527819 mentions this issue: [release-branch.go1.20] cmd/link: disable DWARF by default in c-shared mode on darwin

@joedian joedian added the CherryPickApproved Used during the release process for point releases label Sep 20, 2023
@gopherbot gopherbot removed the CherryPickCandidate Used during the release process for point releases label Sep 20, 2023
gopherbot pushed a commit that referenced this issue Sep 21, 2023
@cherrymui
[release-branch.go1.20] cmd/link: force old Apple linker in plugin mode
071f03a 
There are some bugs in Apple's new linker that probably will not
be fixed when Xcode 15 is released (some time soon). We fix/work
around them but it is too much to backport them all. Force old
Apple linker to work around.

Updates #61229.
For #62597.

Change-Id: Ia5941918e882b22b4dbc41c74764d19d413d0b56
Reviewed-on: https://go-review.googlesource.com/c/go/+/527818
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
gopherbot pushed a commit that referenced this issue Sep 21, 2023
@cherrymui
[release-branch.go1.20] cmd/link: disable DWARF by default in c-share…
08c544d 
…d mode on darwin

[This is a (manual) backport of CL 527415 to Go 1.20.]

Currently, linking a Go c-shared object with C code using Apple's
new linker, it fails with

% cc a.c go.so
ld: segment '__DWARF' filesize exceeds vmsize in 'go.so'

Apple's new linker has more checks for unmapped segments. It is
very hard to make it accept a Mach-O shared object with an
additional DWARF segment.

We may want to stop combinding DWARF into the shared object (see
also #62577). For now, disable DWARF by default in c-shared mode
on darwin.

Updates #61229.
For #62597.

Change-Id: I313349f71296d6d7025db28469593825ce9f1866
Reviewed-on: https://go-review.googlesource.com/c/go/+/527819
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
gopherbot pushed a commit that referenced this issue Sep 21, 2023
@cherrymui
[release-branch.go1.20] cmd/link: suppress -bind_at_load deprecation …
83dce45 
…warning for ld-prime

ld-prime emits a deprecation warning for -bind_at_load. The flag
is needed for plugins to not deadlock (#38824) when linking with
older darwin linker. It is supposedly not needed with newer linker
when chained fixups are used. For now, we always pass it, and
suppress the warning.

Updates #61229.
For #62597.

Change-Id: I4b8a6f864a460c40dc38adbb533f664f7fd5343c
Reviewed-on: https://go-review.googlesource.com/c/go/+/508696
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
(cherry picked from commit 040dbf9)
Reviewed-on: https://go-review.googlesource.com/c/go/+/527798
@gopherbot
Copy link
Contributor Author

Closed by merging 071f03a to release-branch.go1.20.

@gopherbot
Copy link
Contributor Author

Closed by merging 08c544d to release-branch.go1.20.

@gopherbot
Copy link
Contributor Author

Closed by merging 83dce45 to release-branch.go1.20.

@lespea
Copy link

lespea commented Sep 22, 2023

This is a major issue for us... do you think it's going to be a while before a patch release with this fix?

@cherrymui
Copy link
Member

The next Go 1.20.9 release will likely be available next week.

In the mean time, you can build Go from source using the release-branch.go1.20 or release-branch.go1.21 branch (which has the same content as the next minor release), or from tip. Or you can work around this by passing command line flag -ldflags=-extldflags=-Wl,-ld_classic or environment variable CGO_LDFLAGS=-Wl,-ld_classic to choose the old Apple linker.

@578559967
Copy link

-Wl,-force_load flag need to be ignored when check ld_classic is supported in linkerFlagSupported.

@cherrymui
Copy link
Member

Where is -Wl,-force_load from? I don't think the go toolchain pass it. If you pass it on the command line, usually we don't filter user-passed flags. And as you already pass flags, you probably can just pass -ld_classic or -ld_new yourself.

@578559967
Copy link

Where is -Wl,-force_load from? I don't think the go toolchain pass it. If you pass it on the command line, usually we don't filter user-passed flags. And as you already pass flags, you probably can just pass -ld_classic or -ld_new yourself.

We pass -Wl,-force_load myclib.a -lotherlibA -lotherlibB in CGO_LDFLAGS to link some C libs. But linkerFlagSupported ignored -lotherlibA -lotherlibB and keeps the -Wl,-force_load myclib.a and fails the test due to some undefined symbol errors instead of -ld_classic not supported by the linker.

As the -ld_classic is only supported in the newer linker, I need to pass this flag on demand. I don't want to do the repeated job as the compiler already did.

@cherrymui
Copy link
Member

So you suggest adding -Wl,-force_load to trimLinkerArgv https://cs.opensource.google/go/go/+/master:src/cmd/link/internal/ld/lib.go;l=2082 ? That seems reasonable.

On the other hand, we don't test -ld_classic on tip, and for most cases in Go 1.21.4. I don't think we want to backport a change to trimLinkerArgv. Try updating to Go 1.21.4.

This was referenced Jun 12, 2024
Closed
Closed
@gabyhelp gabyhelp mentioned this issue Jun 26, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
CherryPickApproved Used during the release process for point releases compiler/runtime Issues related to the Go compiler and/or runtime.
Projects
None yet
Milestone
Go1.20.9
Development

No branches or pull requests
6 participants
@lespea @578559967 @gopherbot @cherrymui @joedian and others