cmd/link: too many open files on high object-count dependencies #29110
Labels
Comments
I see the problem, sending a CL. |
Change https://golang.org/cl/152757 mentions this issue: |
@gopherbot, please backport to Go 1.11. |
Backport issue(s) opened: #29112 (for 1.11). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases. |
Change https://golang.org/cl/152760 mentions this issue: |
gopherbot
pushed a commit
that referenced
this issue
Dec 5, 2018
…emporary directory Updates #29110 Fixes #29112 Change-Id: I077d1a9caa7f4545de1418cec718c4a37ac36ef8 Reviewed-on: https://go-review.googlesource.com/c/152757 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> (cherry picked from commit 35435b2) Reviewed-on: https://go-review.googlesource.com/c/152760
Can confirm this fix solved the issue. Thank you! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
I think this issue is a continuation of #21621, just in a later step when using a huge package, not when building it.
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
I've built a Go library consisting of more than 1024 Go files (wrapper for Tor):
github.com/ipsn/go-libtor
. I can successfully build this library into a package (note, only Linux and Android is supported for now).When using this package in another project however, Go's linker uses as many file descriptors as the number of object files the package has. Exceeding the 1024 object file limit, Go cannot link against this package any more on Linux without raising the ulimit.
What did you expect to see?
Successful compilation.
What did you see instead?
Link fails with
too many open files
.Reproduction
I figured it's easier to show-and-tell. First up, prepare an empty GOPATH so we don't mess with anything else:
Download and build the Tor wrapper, this takes about 3 minutes on my laptop (verbosity added so you can track the progress, there are 919 files (yes, this is not the case that exceeds 1024 because I didn't want to break the lib, but we'll adjust ulimit to repro nonetheless)):
Ok, lets create a tiny repro program that just imports the library we packaged up previously. Create a small package and dump the contents into it:
Fetch any missing dependencies and run it, you should see some logs from Tor:
Now lower the open file descriptors to below the library file count of 919 and try again:
The text was updated successfully, but these errors were encountered: