Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync::mpsc: synchronize receiver disconnect with initialization #110089

Merged
merged 1 commit into from Apr 13, 2023

Conversation

petrosagg
Copy link
Contributor

Receiver disconnection relies on the incorrect assumption that head.index != tail.index implies that the channel is initialized (i.e head.block and tail.block point to allocated blocks). However, it can happen that head.index != tail.index and head.block == null at the same time which leads to a segfault when a channel is dropped in that state.

This can happen because initialization is performed in two steps. First, the tail block is allocated and the tail.block is set. If that is successful head.block is set to the same pointer. Importantly, initialization is skipped if tail.block is not null.

Therefore we can have the following situation:

  1. Thread A starts to send the first value of the channel, observes that tail.block is null and begins initialization. It sets tail.block to point to a newly allocated block and then gets preempted. head.block is still null at this point.
  2. Thread B starts to send the second value of the channel, observes that tail.block is not null and proceeds with writing its value in the allocated tail block and sets tail.index to 1.
  3. Thread B drops the receiver of the channel which observes that head.index != tail.index (0 and 1 respectively), therefore there must be messages to drop. It starts traversing the linked list from head.block which is still a null pointer, leading to a segfault.

This PR fixes this problem by waiting for initialization to complete when head.index != tail.index and the head.block is still null. A similar check exists in start_recv for similar reasons.

Fixes #110001

@rustbot
Copy link
Collaborator

rustbot commented Apr 8, 2023

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @m-ou-se (or someone else) soon.

Please see the contribution instructions for more information. Namely, in order to ensure the minimum review times lag, PR authors and assigned reviewers should ensure that the review label (S-waiting-on-review and S-waiting-on-author) stays updated, invoking these commands when appropriate:

  • @rustbot author: the review is finished, PR author should check the comments and take action accordingly
  • @rustbot review: the author is ready for a review, this PR will be queued again in the reviewer's queue

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Apr 8, 2023
@rustbot
Copy link
Collaborator

rustbot commented Apr 8, 2023

Hey! It looks like you've submitted a new PR for the library teams!

If this PR contains changes to any rust-lang/rust public library APIs then please comment with @rustbot label +T-libs-api -T-libs to tag it appropriately. If this PR contains changes to any unstable APIs please edit the PR description to add a link to the relevant API Change Proposal or create one if you haven't already. If you're unsure where your change falls no worries, just leave it as is and the reviewer will take a look and make a decision to forward on if necessary.

Examples of T-libs-api changes:

  • Stabilizing library features
  • Introducing insta-stable changes such as new implementations of existing stable traits on existing stable types
  • Introducing new or changing existing unstable library APIs (excluding permanently unstable features / features without a tracking issue)
  • Changing public documentation in ways that create new stability guarantees
  • Changing observable runtime behavior of library APIs

@rust-log-analyzer

This comment has been minimized.

Receiver disconnection relies on the incorrect assumption that
`head.index != tail.index` implies that the channel is initialized (i.e
`head.block` and `tail.block` point to allocated blocks). However, it
can happen that `head.index != tail.index` and `head.block == null` at
the same time which leads to a segfault when a channel is dropped in
that state.

This can happen because initialization is performed in two steps. First,
the tail block is allocated and the `tail.block` is set. If that is
successful `head.block` is set to the same pointer. Importantly,
initialization is skipped if `tail.block` is not null.

Therefore we can have the following situation:

1. Thread A starts to send the first value of the channel, observes that
   `tail.block` is null and begins initialization. It sets `tail.block`
   to point to a newly allocated block and then gets preempted.
   `head.block` is still null at this point.
2. Thread B starts to send the second value of the channel, observes
   that `tail.block` *is not* null and proceeds with writing its value
   in the allocated tail block and sets `tail.index` to 1.
3. Thread B drops the receiver of the channel which observes that
   `head.index != tail.index` (0 and 1 respectively), therefore there
   must be messages to drop. It starts traversing the linked list from
   `head.block` which is still a null pointer, leading to a segfault.

This PR fixes this problem by waiting for initialization to complete
when `head.index != tail.index` and the `head.block` is still null. A
similar check exists in `start_recv` for similar reasons.

Fixes rust-lang#110001

Signed-off-by: Petros Angelatos <petrosagg@gmail.com>
@ibraheemdev
Copy link
Member

Thanks, I suspected this was the issue. @teskje can you verify your example no longer segfaults with this patch applied?

@teskje
Copy link

teskje commented Apr 9, 2023

I tried to reproduce the segfault with this diff applied for ~1 hour and wasn't able to!

@the8472
Copy link
Member

the8472 commented Apr 11, 2023

upstream fix: crossbeam-rs/crossbeam#972

@the8472
Copy link
Member

the8472 commented Apr 11, 2023

Since the cause has been identified is it possible to write a test that triggers the flaw reliably?

@ibraheemdev
Copy link
Member

Not really, it would require preemption at a specific point within send(). Possibly with a cfg(miri) yield() but I'm not sure that's something we want to do?

@joshtriplett joshtriplett added the I-libs-nominated The issue / PR has been nominated for discussion during a libs team meeting. label Apr 12, 2023
@m-ou-se m-ou-se removed the I-libs-nominated The issue / PR has been nominated for discussion during a libs team meeting. label Apr 12, 2023
@m-ou-se
Copy link
Member

m-ou-se commented Apr 12, 2023

@bors r+

@bors
Copy link
Contributor

bors commented Apr 12, 2023

📌 Commit f0d487d has been approved by m-ou-se

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 12, 2023
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 12, 2023
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#109810 (Replace rustdoc-ui/{c,z}-help tests with a stable run-make test )
 - rust-lang#110035 (fix: ensure bad `#[test]` invocs retain correct AST)
 - rust-lang#110089 (sync::mpsc: synchronize receiver disconnect with initialization)
 - rust-lang#110103 (Report overflows gracefully with new solver)
 - rust-lang#110122 (Fix x check --stage 1 when download-ci-llvm=false)
 - rust-lang#110133 (Do not use ImplDerivedObligationCause for inherent impl method error reporting)
 - rust-lang#110135 (Revert "Don't recover lifetimes/labels containing emojis as character literals")
 - rust-lang#110235 (Fix `--extend-css` option)

Failed merges:

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit f56a265 into rust-lang:master Apr 13, 2023
11 checks passed
@rustbot rustbot added this to the 1.70.0 milestone Apr 13, 2023
@klensy klensy mentioned this pull request May 26, 2023
bors added a commit to rust-lang-ci/rust that referenced this pull request May 30, 2023
…lacrum

deps: bump crates

Updates few deps:

drops a lot of cxx* crates:
```console
$ cargo update -p iana-time-zone-haiku
    Updating crates.io index
    Updating cc v1.0.77 -> v1.0.79
    Removing codespan-reporting v0.11.1
    Removing cxx v1.0.94
    Removing cxx-build v1.0.94
    Removing cxxbridge-flags v1.0.94
    Removing cxxbridge-macro v1.0.94
    Updating iana-time-zone-haiku v0.1.1 -> v0.1.2
    Removing link-cplusplus v1.0.8
    Removing scratch v1.0.5
```
cc: https://github.com/rust-lang/cc-rs/releases/tag/1.0.78, https://github.com/rust-lang/cc-rs/releases/tag/1.0.79
iana-time-zone-haiku: https://github.com/strawlab/iana-time-zone/releases/tag/haiku%2Fv0.1.2

fixed crossbeam-rs/crossbeam#972 (similar fixed in rust repo rust-lang#110089)
```console
$ cargo update -p crossbeam-channel
    Updating crates.io index
    Updating crossbeam-channel v0.5.6 -> v0.5.8
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-channel/CHANGELOG.md#version-058

dedupes memoffset versions:
```console
$ cargo update -p crossbeam-epoch
    Updating crates.io index
    Updating crossbeam-epoch v0.9.13 -> v0.9.14
    Removing memoffset v0.7.1
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-epoch/CHANGELOG.md#version-0914
Gilnaa/memoffset@v0.6.5...v0.8.0
rust-lang#108638 (comment)

dedupes bstr versions
```console
$ cargo update -p ignore -p opener
    Updating crates.io index
    Removing bstr v0.2.17
    Updating globset v0.4.9 -> v0.4.10
    Updating ignore v0.4.18 -> v0.4.20
    Updating opener v0.5.0 -> v0.5.2
```

globset BurntSushi/ripgrep@ac8fecb
ignore https://github.com/BurntSushi/ripgrep/commits/master/crates/ignore hard to track, but drop dep on crossbeam-utils (BurntSushi/ripgrep@e95254a), don't stat git if require_git is false (BurntSushi/ripgrep@009dda1) and added bunch of formats to ignore list
opener Seeker14491/opener@v0.5.0...v0.5.2 nothing interesting
RalfJung pushed a commit to RalfJung/miri that referenced this pull request May 31, 2023
deps: bump crates

Updates few deps:

drops a lot of cxx* crates:
```console
$ cargo update -p iana-time-zone-haiku
    Updating crates.io index
    Updating cc v1.0.77 -> v1.0.79
    Removing codespan-reporting v0.11.1
    Removing cxx v1.0.94
    Removing cxx-build v1.0.94
    Removing cxxbridge-flags v1.0.94
    Removing cxxbridge-macro v1.0.94
    Updating iana-time-zone-haiku v0.1.1 -> v0.1.2
    Removing link-cplusplus v1.0.8
    Removing scratch v1.0.5
```
cc: https://github.com/rust-lang/cc-rs/releases/tag/1.0.78, https://github.com/rust-lang/cc-rs/releases/tag/1.0.79
iana-time-zone-haiku: https://github.com/strawlab/iana-time-zone/releases/tag/haiku%2Fv0.1.2

fixed crossbeam-rs/crossbeam#972 (similar fixed in rust repo rust-lang/rust#110089)
```console
$ cargo update -p crossbeam-channel
    Updating crates.io index
    Updating crossbeam-channel v0.5.6 -> v0.5.8
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-channel/CHANGELOG.md#version-058

dedupes memoffset versions:
```console
$ cargo update -p crossbeam-epoch
    Updating crates.io index
    Updating crossbeam-epoch v0.9.13 -> v0.9.14
    Removing memoffset v0.7.1
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-epoch/CHANGELOG.md#version-0914
Gilnaa/memoffset@v0.6.5...v0.8.0
rust-lang/rust#108638 (comment)

dedupes bstr versions
```console
$ cargo update -p ignore -p opener
    Updating crates.io index
    Removing bstr v0.2.17
    Updating globset v0.4.9 -> v0.4.10
    Updating ignore v0.4.18 -> v0.4.20
    Updating opener v0.5.0 -> v0.5.2
```

globset BurntSushi/ripgrep@ac8fecb
ignore https://github.com/BurntSushi/ripgrep/commits/master/crates/ignore hard to track, but drop dep on crossbeam-utils (BurntSushi/ripgrep@e95254a), don't stat git if require_git is false (BurntSushi/ripgrep@009dda1) and added bunch of formats to ignore list
opener Seeker14491/opener@v0.5.0...v0.5.2 nothing interesting
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 20, 2024
deps: bump crates

Updates few deps:

drops a lot of cxx* crates:
```console
$ cargo update -p iana-time-zone-haiku
    Updating crates.io index
    Updating cc v1.0.77 -> v1.0.79
    Removing codespan-reporting v0.11.1
    Removing cxx v1.0.94
    Removing cxx-build v1.0.94
    Removing cxxbridge-flags v1.0.94
    Removing cxxbridge-macro v1.0.94
    Updating iana-time-zone-haiku v0.1.1 -> v0.1.2
    Removing link-cplusplus v1.0.8
    Removing scratch v1.0.5
```
cc: https://github.com/rust-lang/cc-rs/releases/tag/1.0.78, https://github.com/rust-lang/cc-rs/releases/tag/1.0.79
iana-time-zone-haiku: https://github.com/strawlab/iana-time-zone/releases/tag/haiku%2Fv0.1.2

fixed crossbeam-rs/crossbeam#972 (similar fixed in rust repo rust-lang/rust#110089)
```console
$ cargo update -p crossbeam-channel
    Updating crates.io index
    Updating crossbeam-channel v0.5.6 -> v0.5.8
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-channel/CHANGELOG.md#version-058

dedupes memoffset versions:
```console
$ cargo update -p crossbeam-epoch
    Updating crates.io index
    Updating crossbeam-epoch v0.9.13 -> v0.9.14
    Removing memoffset v0.7.1
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-epoch/CHANGELOG.md#version-0914
Gilnaa/memoffset@v0.6.5...v0.8.0
rust-lang/rust#108638 (comment)

dedupes bstr versions
```console
$ cargo update -p ignore -p opener
    Updating crates.io index
    Removing bstr v0.2.17
    Updating globset v0.4.9 -> v0.4.10
    Updating ignore v0.4.18 -> v0.4.20
    Updating opener v0.5.0 -> v0.5.2
```

globset BurntSushi/ripgrep@ac8fecb
ignore https://github.com/BurntSushi/ripgrep/commits/master/crates/ignore hard to track, but drop dep on crossbeam-utils (BurntSushi/ripgrep@e95254a), don't stat git if require_git is false (BurntSushi/ripgrep@009dda1) and added bunch of formats to ignore list
opener Seeker14491/opener@v0.5.0...v0.5.2 nothing interesting
RalfJung pushed a commit to RalfJung/rust-analyzer that referenced this pull request Apr 27, 2024
deps: bump crates

Updates few deps:

drops a lot of cxx* crates:
```console
$ cargo update -p iana-time-zone-haiku
    Updating crates.io index
    Updating cc v1.0.77 -> v1.0.79
    Removing codespan-reporting v0.11.1
    Removing cxx v1.0.94
    Removing cxx-build v1.0.94
    Removing cxxbridge-flags v1.0.94
    Removing cxxbridge-macro v1.0.94
    Updating iana-time-zone-haiku v0.1.1 -> v0.1.2
    Removing link-cplusplus v1.0.8
    Removing scratch v1.0.5
```
cc: https://github.com/rust-lang/cc-rs/releases/tag/1.0.78, https://github.com/rust-lang/cc-rs/releases/tag/1.0.79
iana-time-zone-haiku: https://github.com/strawlab/iana-time-zone/releases/tag/haiku%2Fv0.1.2

fixed crossbeam-rs/crossbeam#972 (similar fixed in rust repo rust-lang/rust#110089)
```console
$ cargo update -p crossbeam-channel
    Updating crates.io index
    Updating crossbeam-channel v0.5.6 -> v0.5.8
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-channel/CHANGELOG.md#version-058

dedupes memoffset versions:
```console
$ cargo update -p crossbeam-epoch
    Updating crates.io index
    Updating crossbeam-epoch v0.9.13 -> v0.9.14
    Removing memoffset v0.7.1
```
https://github.com/crossbeam-rs/crossbeam/blob/master/crossbeam-epoch/CHANGELOG.md#version-0914
Gilnaa/memoffset@v0.6.5...v0.8.0
rust-lang/rust#108638 (comment)

dedupes bstr versions
```console
$ cargo update -p ignore -p opener
    Updating crates.io index
    Removing bstr v0.2.17
    Updating globset v0.4.9 -> v0.4.10
    Updating ignore v0.4.18 -> v0.4.20
    Updating opener v0.5.0 -> v0.5.2
```

globset BurntSushi/ripgrep@ac8fecb
ignore https://github.com/BurntSushi/ripgrep/commits/master/crates/ignore hard to track, but drop dep on crossbeam-utils (BurntSushi/ripgrep@e95254a), don't stat git if require_git is false (BurntSushi/ripgrep@009dda1) and added bunch of formats to ignore list
opener Seeker14491/opener@v0.5.0...v0.5.2 nothing interesting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Segfault in std::sync::mpsc::list
10 participants