Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

os: opening/locating/copying system libraries is broken in macOS Big Sur beta #39832

Open
DiegoMagdaleno opened this issue Jun 24, 2020 · 4 comments
Milestone

Comments

@DiegoMagdaleno
Copy link

@DiegoMagdaleno DiegoMagdaleno commented Jun 24, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/me/Library/Caches/go-build"
GOENV="/Users/me/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/me/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/77/cbrny7lx71vbbgw8jt0zt6x80000gn/T/go-build140751726=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

This recipe is very clear and I actually have an example with my own software:

  1. Use the default go (or any tbh) library to copy a library that is part of the macOS operating system for example /usr/lib/system.b.dylib.
  2. This will return a "No such file or directory".

There is a reason of why this happens, according to Apple, they changed how default system libraries work on macOS (example the C library).

Quote:

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen() the path, which will correctly check for the library in the cache. (62986286)

My project can be located here

This code checks otool to get linked libraries and then copies them (The code might be bad).

What did you expect to see?

  • I expect to get no errors and have the library copied.

What did you see instead?

  • I get an error "no such file or directory" due to this change Apple made.

Proposed solution

Perhaps an idea of mine will be having an array of the default libraries of macOS (such as libsystem.b.dylib) and check at runtime if the OS is macOS if it is, and the user wants to copy some of this libraries, the solution is to syscall dlopen() instead of open(), hope this issue can help anyone and if not I guess I will have to fix it on my side.

@randall77
Copy link
Contributor

@randall77 randall77 commented Jun 24, 2020

I'm a bit confused as to what this implies. Does the Go toolchain not work? Do Go binaries not work? What about shared libraries and plugins?
Or is it just that os.Open("/usr/lib/libSystem.dylib") doesn't work?
Does the same problem happen in C? If so, I don't think there's anything for us to fix - this sounds like intentional behavior on Apple's part. (But why? Weird.)

@randall77
Copy link
Contributor

@randall77 randall77 commented Jun 24, 2020

Note that this problem is listed as a "known issue" of the beta, not as an intended feature. Maybe it will be fixed.

@DiegoMagdaleno
Copy link
Author

@DiegoMagdaleno DiegoMagdaleno commented Jun 24, 2020

@randall77 Well what works is basically everything in Golang, however, yeah the os.Open("/usr/lib/libsystem.dylib"), is basically broken, I tested in C and yes the same happens unless I do dlopen(), that is Apple's new behavior, while the issue only affects systems libraries, means that without a workaround opening/copying default libraries on macOS will no longer work (in Go or other stuff that doesn't do a workaround)

@dmitshur dmitshur changed the title Opening/Locating/Copying System libraries is broken in macOS Big Sur. Opening/Locating/Copying System libraries is broken in macOS Big Sur beta Jun 24, 2020
@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 24, 2020

If Go programs generally work, but they fail in exactly the ways that C programs fail, then I don't think there is anything we should or even can change in the Go project.

@cagedmantis cagedmantis changed the title Opening/Locating/Copying System libraries is broken in macOS Big Sur beta os: opening/locating/copying system libraries is broken in macOS Big Sur beta Jun 24, 2020
@cagedmantis cagedmantis modified the milestones: Backlog, Go1.16 Jun 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.