Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix intermittent test failure caused by race condition #140

Merged
merged 3 commits into from Apr 7, 2019

Conversation

cjpearce
Copy link
Contributor

@cjpearce cjpearce commented Apr 7, 2019

First public pull request 😬

There's an intermittent integration test failure when you use multiple test threads (at least for me on a mac). I narrowed it down to two tests each spawning a process using Command which then try to compile the same file at the same time. If the timing doesn't work out, they both try to compile, and then one process runs clean before the other can run the executable - causing a panic.

Screenshot 2019-04-07 at 19 54 55

You can prevent it from happening by running with a single thread (cargo test -- --test-threads=1), because the Command blocks. That's not a particularly good solution though because it's not something you can configure in Cargo.toml.

I considered making the affected tests just run serially, but it occurred to me that this could also happen if someone accidentally runs rustlings in watch mode in two terminals without realising it. I wound't consider this that unlikely given it's a tool for learning.

I fixed it by ensuring that the executables made from separate processes don't conflict by appending a process id to the output executable name. I also extracted the commands into a single file next to clean so that we don't have to repeat the generated file name everywhere and risk missing something.

@komaeda komaeda added A-source Area: CLI source C-bug Category: Bug labels Apr 7, 2019
@komaeda
Copy link
Contributor

komaeda commented Apr 7, 2019

Thanks for this! Do you want to edit the Travis build source (https://github.com/rust-lang/rustlings/blob/master/.travis.yml#L6) as well so that we can directly test if this works?

@cjpearce
Copy link
Contributor Author

cjpearce commented Apr 7, 2019

Yeah of course! I didn't think of checking the CI script. Thanks

@komaeda
Copy link
Contributor

komaeda commented Apr 7, 2019

Thanks!

@bors r+

@bors
Copy link
Contributor

bors commented Apr 7, 2019

📌 Commit 65cb09e has been approved by komaeda

@bors
Copy link
Contributor

bors commented Apr 7, 2019

⌛ Testing commit 65cb09e with merge ffb165c...

bors added a commit that referenced this pull request Apr 7, 2019
Fix intermittent test failure caused by race condition

First public pull request 😬

There's an intermittent integration test failure when you use multiple test threads (at least for me on a mac). I narrowed it down to two tests each spawning a process using `Command` which then try to compile the same file at the same time. If the timing doesn't work out, they both try to compile, and then one process runs `clean` before the other can run the executable - causing a panic.

![Screenshot 2019-04-07 at 19 54 55](https://user-images.githubusercontent.com/3453268/55688324-20520980-596f-11e9-8474-5215d61a4387.png)

You can prevent it from happening by running with a single thread (`cargo test -- --test-threads=1`), because the `Command` blocks. That's not a particularly good solution though because it's not something you can configure in `Cargo.toml`.

I considered making the affected tests just run serially, but it occurred to me that this could also happen if someone accidentally runs rustlings in watch mode in two terminals without realising it. I wound't consider this that unlikely given it's a tool for learning.

I fixed it by ensuring that the executables made from separate processes don't conflict by appending a process id to the output executable name. I also extracted the commands into a single file next to `clean` so that we don't have to repeat the generated file name everywhere and risk missing something.
@bors
Copy link
Contributor

bors commented Apr 7, 2019

☀️ Test successful - checks-travis
Approved by: komaeda
Pushing ffb165c to master...

@bors bors merged commit 65cb09e into rust-lang:master Apr 7, 2019
pedantic79 pushed a commit to pedantic79/rustlings that referenced this pull request Apr 11, 2020
…aeda

Fix intermittent test failure caused by race condition

First public pull request 😬

There's an intermittent integration test failure when you use multiple test threads (at least for me on a mac). I narrowed it down to two tests each spawning a process using `Command` which then try to compile the same file at the same time. If the timing doesn't work out, they both try to compile, and then one process runs `clean` before the other can run the executable - causing a panic.

![Screenshot 2019-04-07 at 19 54 55](https://user-images.githubusercontent.com/3453268/55688324-20520980-596f-11e9-8474-5215d61a4387.png)

You can prevent it from happening by running with a single thread (`cargo test -- --test-threads=1`), because the `Command` blocks. That's not a particularly good solution though because it's not something you can configure in `Cargo.toml`.

I considered making the affected tests just run serially, but it occurred to me that this could also happen if someone accidentally runs rustlings in watch mode in two terminals without realising it. I wound't consider this that unlikely given it's a tool for learning.

I fixed it by ensuring that the executables made from separate processes don't conflict by appending a process id to the output executable name. I also extracted the commands into a single file next to `clean` so that we don't have to repeat the generated file name everywhere and risk missing something.
ppp3 pushed a commit to ppp3/rustlings that referenced this pull request May 23, 2022
…aeda

Fix intermittent test failure caused by race condition

First public pull request 😬

There's an intermittent integration test failure when you use multiple test threads (at least for me on a mac). I narrowed it down to two tests each spawning a process using `Command` which then try to compile the same file at the same time. If the timing doesn't work out, they both try to compile, and then one process runs `clean` before the other can run the executable - causing a panic.

![Screenshot 2019-04-07 at 19 54 55](https://user-images.githubusercontent.com/3453268/55688324-20520980-596f-11e9-8474-5215d61a4387.png)

You can prevent it from happening by running with a single thread (`cargo test -- --test-threads=1`), because the `Command` blocks. That's not a particularly good solution though because it's not something you can configure in `Cargo.toml`.

I considered making the affected tests just run serially, but it occurred to me that this could also happen if someone accidentally runs rustlings in watch mode in two terminals without realising it. I wound't consider this that unlikely given it's a tool for learning.

I fixed it by ensuring that the executables made from separate processes don't conflict by appending a process id to the output executable name. I also extracted the commands into a single file next to `clean` so that we don't have to repeat the generated file name everywhere and risk missing something.
dmoore04 pushed a commit to dmoore04/rustlings that referenced this pull request Sep 11, 2022
…aeda

Fix intermittent test failure caused by race condition

First public pull request 😬

There's an intermittent integration test failure when you use multiple test threads (at least for me on a mac). I narrowed it down to two tests each spawning a process using `Command` which then try to compile the same file at the same time. If the timing doesn't work out, they both try to compile, and then one process runs `clean` before the other can run the executable - causing a panic.

![Screenshot 2019-04-07 at 19 54 55](https://user-images.githubusercontent.com/3453268/55688324-20520980-596f-11e9-8474-5215d61a4387.png)

You can prevent it from happening by running with a single thread (`cargo test -- --test-threads=1`), because the `Command` blocks. That's not a particularly good solution though because it's not something you can configure in `Cargo.toml`.

I considered making the affected tests just run serially, but it occurred to me that this could also happen if someone accidentally runs rustlings in watch mode in two terminals without realising it. I wound't consider this that unlikely given it's a tool for learning.

I fixed it by ensuring that the executables made from separate processes don't conflict by appending a process id to the output executable name. I also extracted the commands into a single file next to `clean` so that we don't have to repeat the generated file name everywhere and risk missing something.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-source Area: CLI source C-bug Category: Bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants