File::create in gcc-shim sometimes fails with multiple test threads

[spl]

On macOS 10.14.5 (Mojave) with rustc 1.35.0 and 1.36.0, if I do the following with cc-rs at d2ba46f:

$ export RUST_BACKTRACE=full
$ while cargo test; do sleep 0.00001; done

I eventually get the following:

cargo:warning=thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:997:5
cargo:warning=stack backtrace:
cargo:warning=   0:        0x10b4928c3 - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::hf2949dcf16da83bd
cargo:warning=   1:        0x10b48ed52 - std::sys_common::backtrace::_print::h4b87ed6ba0cd5304
cargo:warning=   2:        0x10b491986 - std::panicking::default_hook::{{closure}}::hde39fed432870543
cargo:warning=   3:        0x10b49172f - std::panicking::default_hook::hfce8820d41dbdcbf
cargo:warning=   4:        0x10b49203f - std::panicking::rust_panic_with_hook::hfcf2d0777bc6c409
cargo:warning=   5:        0x10b491b6c - std::panicking::continue_panic_fmt::h4d668741ea600293
cargo:warning=   6:        0x10b491a58 - rust_begin_unwind
cargo:warning=   7:        0x10b4a21a1 - core::panicking::panic_fmt::hdf67eedb88644e1e
cargo:warning=   8:        0x10b487a1e - core::result::unwrap_failed::h72e112a4d5b9862c
cargo:warning=   9:        0x10b487c2c - core::result::Result<T,E>::unwrap::hd3ca7ccf5659392e
cargo:warning=  10:        0x10b48500a - gcc_shim::main::h533f983c23ce4b8d
cargo:warning=  11:        0x10b485841 - std::rt::lang_start::{{closure}}::h02cd9a674d8a658f
cargo:warning=  12:        0x10b491a37 - std::panicking::try::do_call::h1252fc9a2ff235eb
cargo:warning=  13:        0x10b493a6e - __rust_maybe_catch_panic
cargo:warning=  14:        0x10b4923fd - std::rt::lang_start_internal::h4c054360e442146c
cargo:warning=  15:        0x10b485821 - std::rt::lang_start::h4f275a1c31c55923
cargo:warning=  16:        0x10b485191 - main

error occurred: Command "cc" "-O2" "-ffunction-sections" "-fdata-sections" "-m64" "-o" "/[...]/cc-rs/target/debug/gcc-test.m754lUmxjWLh/foo.o" "-c" "foo.c" with args "cc" did not execute successfully (status code exit code: 101).

I cannot reproduce the panic if I replace cargo test with cargo test -- --test-threads 1.

The panic comes arbitrarily from (a) one of the two uses of File::create in src/bin/gcc-shim.rs and (b) one of the stub executables (cc, ar, etc.)

[spl]

I'm not really sure what's causing this.

After looking around the std::fs API docs, on a whim, I tried replacing File::create(path) with OpenOptions::new().write(true).create_new(true).open(path), but the panic returned.

[spl]

Interesting note: If I replace File::create(path) with File::create(path).or_else(|_| File::create(path)), I don't get a panic after running it for many minutes.

[alexcrichton]

FWIW all of the tests were written to be run single-threaded, so there's probably a lot of things lurking here and there...

[spl]

I see. But this is still a strange one. Do you have any idea why it might be happening?

[alexcrichton]

This in specific sorry no, no idea why it's happening :(

[spl]

Can you reproduce it?

[alexcrichton]

Sorry I'm not currently trying since I'm busy elsewhere.

[spl]

No problem. I would appreciate it if you could try when you have time.

[spl]

For the record, the problem appears to be due to hard-linking (std::fs::hard_link) the gcc-shim executable, and it only happens on macOS out of all the targets tested on Azure. It's a strange problem since it doesn't happen every time, but at least it's often enough to make it easily reproducible. The simplest solution seems to be copying (std::fs::copy) instead of hard-linking it.

[spl]

On reflection, this could be an Apple File System (APFS) issue. I didn't try any other file systems.

[spl]

As a follow-up for posterity's sake, I could reproduce the issue on an HFS+ file system but not on an exFAT file system (using a mounted disk image created by Disk Utility).