-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: rust unit test "Text file busy" #996
Conversation
58dd004
to
d1e95d3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Optional change request. I can do a quick rereview if you do make the change.
d1e95d3
to
5b80f1c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I saw this flake after 500 runs, which is a lot better. If we decide that's too much at any point, we could use a mutex for any tests that are forking, or run any tests that fork in a single thread
@@ -59,6 +59,17 @@ mod tests { | |||
|
|||
use super::*; | |||
|
|||
/// OS error 26 is "Text file busy", | |||
/// which can happen when executing a script | |||
/// that is has been written to immediately before. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: the race is caused by threading + forking
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think its due to forking, at least not if this can be attributed to rust-lang/rust#114554.
Even if cargo test
runs by forking processes (which I'm not certain about either), each process will write to files in different tempdirs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
> This is due to the following sequence of events occuring:
>
> 1. The main thread opens the file descriptor for writing into `/tmp/executable.sh`.
>
> 2. The spawned thread calls `fork(2)`, inheriting the file descriptor that has write access to `/tmp/executable.sh`.
>
> 3. The main thread closes its copy of the file descriptor — but note that it is **not** closed in the child process.
>
> 4. The main thread then attempts to execute the file. However, because there exists a process with a file descriptor open that has write access to the file, it fails with ETXTBSY and the process panics.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cargo test
is threading, Command
is forking
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Command::spawn == fork() -> exec()
that fork, I see..
I think it's clear enough for now, for more detail there is the upstream issue linked as well
Co-authored-by: Matthew Kenigsberg <matthew@floxdev.com>
Resolve issues with rust unit tests in
models::container_builder::tests::*
.The container builder tests spuriously failed with
Text file busy
:Text file busy (26)
may occur when executing a scriptthat is has been written to immediately prior.
This is a known bug in rust (and allegedly other languages)
which is tracked in rust-lang/rust#114554.
We typically see this error in tests, where we write a new container builder script
and immediately execute it within the same process:
https://github.com/flox/flox/blob/main/cli/flox-rust-sdk/src/models/container_builder.rs#L76-L83
In production use, this should not be a problem as the script will be written
by a different process, i.e.
pkgdb
.