Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

os: TestOpenFileLimit fails on openbsd #51713

Closed
rsc opened this issue Mar 16, 2022 · 4 comments
Closed

os: TestOpenFileLimit fails on openbsd #51713

rsc opened this issue Mar 16, 2022 · 4 comments
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. OS-OpenBSD
Milestone
Go1.19

Comments

@rsc
Copy link
Contributor

rsc commented Mar 16, 2022

For #46279, we added os.TestOpenFileLimit, which checks that it can open 1200 files.
This test fails on openbsd, even after doing setrlimit(RLIMIT_NOFILE, lim) with lim.Cur raised to lim.Max.

Perhaps the builder runs with a hard limit < 1200.
Whatever the reason, OpenBSD is not a first-class port,
so the test will be skipped with a pointer to this issue,
and we will leave the issue for any OpenBSD maintainers
that want to take a look.
@rsc rsc added this to the Backlog milestone Mar 16, 2022
@rsc rsc added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Mar 16, 2022
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/393354 mentions this issue: os: raise open file rlimit at startup

gopherbot pushed a commit that referenced this issue Mar 17, 2022
@rsc
os: raise open file rlimit at startup
8427429 
Some systems set an artificially low soft limit on open file count,
for compatibility with code that uses select and its hard-coded
maximum file descriptor (limited by the size of fd_set).

Go does not use select, so it should not be subject to these limits.
On some systems the limit is 256, which is very easy to run into, even
in simple programs like gofmt when they parallelize walking a file tree.

After a long discussion on go.dev/issue/46279, we decided the best
approach was for Go to raise the limit unconditionally for itself, and
then leave old software to set the limit back as needed. Code that
really wants Go to leave the limit alone can set the hard limit, which
Go of course has no choice but to respect.

Take 2, after CL 392415 was rolled back for macOS and OpenBSD failures.
The macOS failures should be handled by the new call to sysctl("kern.maxfilesperproc"),
and the OpenBSD failures are handled by skipping the test (and filing #51713).

Fixes #46279.

Change-Id: I45c81b94590b447b483018a05ae980b8f02dc5de
Reviewed-on: https://go-review.googlesource.com/c/go/+/393354
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@mengzhuo
Copy link
Contributor

FYI: It still failed on openbsd/mips64

https://build.golang.org/log/8d2a91e75462acd10f0dc525f53e25004631e444

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/394094 mentions this issue: os: skip TestOpenFileLimit on openbsd/mips64

gopherbot pushed a commit that referenced this issue Mar 22, 2022
@ianlancetaylor @gopherbot
os: skip TestOpenFileLimit on openbsd/mips64
96567fb 
For #46279
For #51713

Change-Id: I444f309999bf5576449a46a9808b23cf6537e7dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/394094
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joel Sing <joel@sing.id.au>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/401594 mentions this issue: os: use a lower file count for TestOpenFileLimit on openbsd

@golang golang locked and limited conversation to collaborators Apr 23, 2023
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 2, 2023
@dmitshur dmitshur modified the milestones: Backlog, Go1.19 Nov 2, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge NeedsFix The path to resolution is known, but the work has not been done. OS-OpenBSD
Projects
None yet
Milestone
Go1.19
Development

No branches or pull requests
5 participants
@rsc @mengzhuo @dmitshur @bcmills @gopherbot