rustc panics when it can't spawn a thread

[RithvikChuppala]

I wrote up some public and private strcuts, public methods, and traits in a module called structures.rs

In main.rs, I import structures.rs and write tests in its own separate module (within main.rs) using #[cfg(test)] and #[test] above each test method, which tests some public functions and structs from structures.rs

In both files, the fn main() only contains 1 println! statement.

Compiling both main.rs and structures.rs gives me a few warnings but no errors.

I try to run the unit tests using "cargo test" in the "src" folder of my crate but it doesn't act deterministically. Running "cargo test --verbose" errors with the message:

thread '<unnamed>' panicked at 'failed to spawn thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', /rustc/8d69840ab92ea7f4d323420088dd8c9775f180cd/src/libstd/thread/mod.rs:619:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.43.1 (8d69840ab 2020-05-04) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental

note: some of the compiler flags provided by cargo are hidden

thread 'rustc' panicked at 'src/librustc_codegen_ssa/back/write.rs:1754: panic during codegen/LLVM phase', src/librustc/util/bug.rs:37:26

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.43.1 (8d69840ab 2020-05-04) running on x86_64-unknown-linux-gnu

note: compiler flags: -C debuginfo=2 -C incremental

note: some of the compiler flags provided by cargo are hidden

error: could not compile `ExprCalc`.

Caused by:
  process didn't exit successfully: `rustc --crate-name ExprCalc --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C debuginfo=2 --test -C metadata=7d54c5c1fcb15878 -C extra-filename=-7d54c5c1fcb15878 --out-dir /home/rchuppala/ExpressionCalculator/ExprCalc/target/debug/deps -C incremental=/home/rchuppala/ExpressionCalculator/ExprCalc/target/debug/incremental -L dependency=/home/rchuppala/ExpressionCalculator/ExprCalc/target/debug/deps` (exit code: 101)

Other times, it works successfully and gives me the result of running the tests. I've done a variety of things, including reinstalling Rust and resourcing the path, and received continued undeterministic results (sometimes working, sometimes erring). Any advice?

rustc --version: rustc 1.43.1 (8d69840 2020-05-04)
system: linux CentOS

[jonas-schievink]

Please check that your system is configured correctly, see #69140

[RithvikChuppala]

Please check that your system is configured correctly, see #69140

Does this mean that the issue is not with the rust code or with rust itself, but rather the system I'm running rust on?

[jonas-schievink]

Yes

[jonas-schievink]

Well, or at least that's very likely. All the compiler does is spawn a couple of threads, which fails on your system.

[Elinvynia]

Have you tried the CARGO_BUILD_JOBS=1 workaround?

[RithvikChuppala]

Have you tried the CARGO_BUILD_JOBS=1 workaround?

Yes. It appears to reduce to occurrence of the issue, although not solving it entirely.

[RalfJung]

Also Cc #46345

[camelid]

@RithvikChuppala Are you using a lot of compute power when this happens?

This seems unlikely to be an issue that Rust can solve, and there's already a related issue, so closing for now. Reopen if you disagree.

[jyn514]

Re-opening this; it's possible libtest could retry if it can't spawn a thread instead of panicking. #46345 is not really related; libtest has to handle the error somehow, the standard library can't do that for it.

[camelid]

Seems like this line might be the cause, but not sure:

rust/library/test/src/lib.rs

Line 473 in 981346f

cfg.spawn(runtest).unwrap();

[camelid]

Perhaps instead of panicking if we can't spawn more threads, we print a message like "Unable to spawn thread" and process::exit(1)? That way we won't get an ICE and thus bug reports.

[camelid]

Actually it might make more sense to just run the tests synchronously in that case.

[camelid]

Put up a draft PR that may fix this: #78165.

[ghost]

Re-opening this; it's possible libtest could retry if it can't spawn a thread instead of panicking. #46345 is not really related; libtest has to handle the error somehow, the standard library can't do that for it.

I think this issue was originally about the rustc ICE rather than libtest?
The ICE happened when compiling tests. The libtest issue only happens when running tests.

[camelid]

I think this issue was originally about the rustc ICE rather than libtest?
The ICE happened when compiling tests. The libtest issue only happens when running tests.

As far as I understand, this issue is about a libtest ICE, not a rustc ICE. The ICE occurs when libtest is unable to spawn a thread to run a test. For example, if your system is at the thread limit, libtest would try to spawn a thread, fail, and crash.

[ghost]

The log clearly shows the ICE happened when compiling tests using rustc:

Caused by:
  process didn't exit successfully: `rustc --crate-name ExprCalc --edition=2018 src/main.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C debuginfo=2 --test -C metadata=7d54c5c1fcb15878 -C extra-filename=-7d54c5c1fcb15878 --out-dir /home/rchuppala/ExpressionCalculator/ExprCalc/target/debug/deps -C incremental=/home/rchuppala/ExpressionCalculator/ExprCalc/target/debug/incremental -L dependency=/home/rchuppala/ExpressionCalculator/ExprCalc/target/debug/deps` (exit code: 101)

ICE stands for "internal compiler error", which should not be reported by libtest in my case (I don't have any doctest), because the compiler never runs non-documentation tests (edit: I take back what I said before, looks like rustdoc can call libtest and may cause ICE). Cargo does this job instead, and if libtest panics, only the normal panicking message and error: test failed, to rerun pass '--lib' (or other similar flags) are printed.

The log I pasted on #46345 (comment) does not contain anything related to ICEs.

[camelid]

@jyn514 ^

[jyn514]

Sorry for the mis-diagnosis, I updated the issue to match what's going on.

[jplatte]

Put up a draft PR that may fix this: #78165.

This was superseded by #81546 which was then merged, that probably means this issue has been fixed?

[camelid]

At this point, I'm not really sure what the cause of this issue. I think in the end it's unrelated to the libtest issue. Probably this should just be closed since I don't think it's actionable. But let's see if anyone else wants to comment.

[Enselic]

Probably this should just be closed since I don't think it's actionable.

Triage: There were no objections, so let's close as duplicate to #46345