-
Notifications
You must be signed in to change notification settings - Fork 18k
os: don't waste extra threads for opening files #32618
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Change https://golang.org/cl/182397 mentions this issue: |
/cc @rsc @ianlancetaylor |
After the suggested change, |
Are you sure? fcntl(2) being effectively used in #32619 "provides for control over descriptors" according to the man page. |
@Al2Klimov The per-FD flags are accessed with F_GETFD/F_SETFD, the per-file flags are accessed with F_GETFL/F_SETFL. O_NONBLOCK is defined as a "file status flag" (not file descriptor flag), and used with F_SETFL. |
If |
@ianlancetaylor Unf actually that is pretty nasty. I mean, one could hope that in 2019 every process was already prepared to handle EAGAIN, but it really wouldn't surprise me if doing that still either made a bunch of things error out, or sent them into busy loops. |
Fun fact: a Go process reading stdin is the only process on my machine with a non-blocking stdin/stdout.
|
So my earlier argument was "some things will break", with a vague promise of breaking. Here's a concrete breakage if you automatically add O_NONBLOCK to all os.OpenFile calls: man 7 fifo
So, automatic O_NONBLOCK at open(2) time will break all fifo uses where the write side is opened first. |
Strange. With Create() instead of Open() I also got error=nil. |
I'd like to point out that O_NONBLOCK is an attribute of the file description (where description != descriptor), not the file. That is, two calls to open will produce two different file descriptions. A file description can have many file descriptors. For example, calling dup on a file descriptor will yield a new file descriptor that shares the first file descriptor's file description. Same goes for fork. The core issues explained by @tv42 still apply, but two processes each opening the same file cannot screw each other up, as they will not share the same file description. Of course, the file descriptors 0–2 still commonly share a single file description, but opening /dev/stdout in non-blocking mode should be safe. Refer to the following excerpt of open(2)
as well as the entire subsection |
@dominikh is correct, I misremembered the details. The traps only activate if you share an open fd (inherit it or pass it over unix domain socket). So by that logic, just Adding
Also, I'd like to clarify the issue subject: it should be "os: don't waste extra threads when opening fifos". This bug is only relevant if you're opening lots of FIFOs for reading concurrently. That's a really rare edge case scenario. The long wait in the syscall doesn't trigger on most kinds of files. FIFOs are a rare special case, and 1) leaving POSIX doesn't define what You could perhaps argue for automatic |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
mkfifo pipe
andGOMAXPROCS=1 lldb theprogramshownbelow
.What did you expect to see?
Not much more threads than one.
What did you see instead?
11 threads including a lot of hanging in the
open(2)
syscall.The text was updated successfully, but these errors were encountered: