Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swap std::sync::mpsc channel with crossbeam_channel #7844

Merged
merged 1 commit into from Jan 30, 2020

Conversation

@spastorino
Copy link
Member

spastorino commented Jan 28, 2020

Hoping it closes #7840

r? @Mark-Simulacrum

Switching this hoping it closes #7840
@rust-highfive

This comment has been minimized.

Copy link

rust-highfive commented Jan 28, 2020

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @Mark-Simulacrum (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Jan 28, 2020

r? @ehuss

@ehuss

This comment has been minimized.

Copy link
Contributor

ehuss commented Jan 29, 2020

I don't think we can make this change. When combined with #7838, it causes a very dramatic performance hit. Fresh builds can be many times slower (9 times slower with 500 units).

@Mark-Simulacrum

This comment has been minimized.

Copy link
Member

Mark-Simulacrum commented Jan 29, 2020

That's quite unexpected from my perspective at least -- have we done any profiling to suggest why that would be the case? Generally crossbeam-channel suggests in docs and such that it's faster/better all around than std's impl, but I guess that's not the case?

@ehuss

This comment has been minimized.

Copy link
Contributor

ehuss commented Jan 29, 2020

I might try digging into it more later. It looks like it is having a strange interaction with the jobserver helper thread. I can measure dropping the helper thread takes over 1s (but not always). Helper::join is a little funky in the way it uses signals, but it is not clear to me what's going on.

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Jan 29, 2020

I don't believe we can switch to crossbeam's channels since they do not have a blocking send function which #7838 will require, so this'll likely just require more churn otherwise. I was working on #7845 yesterday but didn't get a chance to polish it until this morning, but that's the route I think we should take (just write our own simple queue).

I'll comment over there about the performance though

@ehuss

This comment has been minimized.

Copy link
Contributor

ehuss commented Jan 29, 2020

The issue I was seeing is dropping the job server can take up to 1 second, see alexcrichton/jobserver-rs#23. I'm guessing that the change to crossbeam is just altering the behavior slightly enough for it to trigger that.

I don't believe we can switch to crossbeam's channels since they do not have a blocking send

I'm not sure what you mean by this. Crossbeam has a bounded channel, and Sender::send should block if the queue is full.

@alexcrichton

This comment has been minimized.

Copy link
Member

alexcrichton commented Jan 30, 2020

Ok the jobserver issue should be fixed now, sorry about that and thanks for helping to diagnose it @ehuss!

Also apologies, I was looking at the documentation for crossbeam-queue rather than crossbeam-channel, and yeah looks like crossbeam-channel has what we want.

Here's my proposed plan of action (which I'll go ahead and enact at the bottom)

  • Let's merge this PR. Super simple, fixes a known bug.
  • Independently we can figure out #7838
  • If necessary we can reconsider #7845 if needed by #7838

@bors: r+

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 30, 2020

📌 Commit 20ddff8 has been approved by alexcrichton

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 30, 2020

⌛️ Testing commit 20ddff8 with merge cc42c0b...

bors added a commit that referenced this pull request Jan 30, 2020
Swap std::sync::mpsc channel with crossbeam_channel

Hoping it closes #7840

r? @Mark-Simulacrum
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 30, 2020

💡 This pull request was already approved, no need to approve it again.

  • This pull request is currently being tested. If there's no response from the continuous integration service, you may use retry to trigger a build again.
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 30, 2020

📌 Commit 20ddff8 has been approved by alexcrichton

@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 30, 2020

⌛️ Testing commit 20ddff8 with merge 4fbd644...

bors added a commit that referenced this pull request Jan 30, 2020
Swap std::sync::mpsc channel with crossbeam_channel

Hoping it closes #7840

r? @Mark-Simulacrum
@bors

This comment has been minimized.

Copy link
Contributor

bors commented Jan 30, 2020

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing 4fbd644 to master...

@bors bors merged commit 20ddff8 into rust-lang:master Jan 30, 2020
11 checks passed
11 checks passed
homu Test successful
Details
rust-lang.cargo Build #20200128.3 succeeded
Details
rust-lang.cargo (Linux beta) Linux beta succeeded
Details
rust-lang.cargo (Linux nightly) Linux nightly succeeded
Details
rust-lang.cargo (Linux stable) Linux stable succeeded
Details
rust-lang.cargo (Windows x86_64-msvc) Windows x86_64-msvc succeeded
Details
rust-lang.cargo (build_std) build_std succeeded
Details
rust-lang.cargo (docs) docs succeeded
Details
rust-lang.cargo (macOS) macOS succeeded
Details
rust-lang.cargo (resolver) resolver succeeded
Details
rust-lang.cargo (rustfmt) rustfmt succeeded
Details
bors added a commit that referenced this pull request Jan 31, 2020
[beta] Revert scalable jobserver.

This reverts #7731, #7829, and #7836.

This should prevent #7840 on beta. I feel more comfortable reverting this than merging #7844, since the impact is still unknown.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

6 participants
You can’t perform that action at this time.