Add scoped threads to the standard library

[stjepang]

Rendered

Add scoped threads to the standard library that allow one to spawn threads borrowing variables from the parent thread. Example:

let var = String::from("foo");

thread::scope(|s| {
    s.spawn(|_| println!("borrowed from thread #1: {}", var));
    s.spawn(|_| println!("borrowed from thread #2: {}", var));
})
.unwrap();

[pitdicker]

Scoped threads in Crossbeam have matured through years of experience and today we have a design that feels solid enough

I am a little curious about this. Can you say something about the history of scoped threads in Crossbeam? In what way did it change to mature over the years?

[stjepang]

@pitdicker Sure!

There are two designs scoped threads went through. The old one is from the time thread::scoped() got removed and we wanted a sound alternative for the Rust 1.0 era. The new one is from the last year's big revamp.

Old: https://docs.rs/crossbeam/0.2.12/crossbeam/fn.scope.html
New: https://docs.rs/crossbeam/0.7.1/crossbeam/fn.scope.html

There are several differences between the old and new scoped threads:

1. scope() now returns a thread::Result<R> rather than R. This is because panics in the old design were just silently ignored, which is not good. By returning a Result, the user can handle panics in whatever way they want.

2. The closure passed to Scope::spawn() now takes a &Scope<'env> argument that allows you to spawn nested threads, which was not possible with the old design. Rayon similarly passes a reference to child tasks.

3. We removed Scope::defer() because it's not really useful, had bugs, and had a non-obvious behavior.

4. Some soundness bugs around lifetimes got fixed. EDIT: Those could've just been bugs in the borrow check.

[Ixrec]

As currently written, the motivation's definitely lacking for me. All the arguments given in the RFC could be made for any other widely used crate (except the historical precedent that this used to be in std, which imo isn't a compelling argument on its own). Unfortunately, I don't know anything about crossbeam, but I strongly suspect we have better arguments that we could put in the RFC.

The usual arguments I find compelling for putting something in std are:

1. It needs a separate implementation for every platform, so providing an official, standard implementation makes the whole ecosystem more "portable by default".

This is why things like std::thread belong in std, which I hope is uncontroversial. But I have no idea if this also applies to scoped threads. Do crossbeam's scoped threads involve abstracting over any OS-specific primitives that regular threads don't?

2. It cannot be implemented "properly" (whether that's for performance or correctness or ergonomic reasons) without using unsafe code.

I'm guessing this applies to crossbeam, but I really don't know. How much non-trivial unsafety is there is crossbeam's scoped threads implementation?

Another argument that's not always compelling but usually moves the needle for me is an analogy with stuff that's already in std. The fact that we recent-ish-ly adopted parking_lot impls of std's concurrency primitives seems like it might be analogous to adopting crossbeam for scoped threads, assuming crossbeam scoped threads are not abstracting over OS primitives and there simply are no OS primitives for scoped threads (the latter feels like a safe assumption since "scope" is not a thing at the OS level, but again I really don't know enough about crossbeam (or parking_lot) to tell how much of this would form a cogent argument).

[stjepang]

@Ixrec Thank you for the thoughtful response!

Do crossbeam's scoped threads involve abstracting over any OS-specific primitives that regular threads don't?

They don't.

How much non-trivial unsafety is there is crossbeam's scoped threads implementation?

There's some crazy unsafety in there that was incredibly tricky to get right. Suffice to say that even after the leakpocalypse, Crossbeam had unsound scoped threads for three years! That's pretty non-trivial. :) EDIT: Perhaps that was a soundness hole in the borrow checker?

Another argument that's not always compelling but usually moves the needle for me is an analogy with stuff that's already in std.

I see the lack of scoped threads more as a gaping hole in std. If one of Rust's selling points is that it erases the boundary between borrowing and threads, why does spawn() still require F: 'static?

We even have this unstable and unsafe std::thread::Builder::spawn_unchecked() function, which is supposed to ease the pain of building scoped thread abstractions. But we could instead have a simple and proper abstraction in std... shrug

[danielhenrymantilla]

I agree that the fact the default example of concurrency in Rust, the language that maximizes zero-cost abstractions, uses Arc to share data around, does look surprising and disappointing to me. That's why I think scoped threads deserve a special treatment that maybe other popular crates don't.

[Centril]

There's some crazy unsafety in there that was incredibly tricky to get right. Suffice to say that even after the leakpocalypse, Crossbeam had unsound scoped threads for three years! That's pretty non-trivial. :)

@stjepang Given this, my only comment is that I would like there to be an equal crazy amount of comments justifying the usage of every unsafe impl and unsafe { ... } in the code that we add. Also, if @RalfJung and/or @jhjourdan et al. can have a look at the implementation that would be nice.

[mark-i-m]

Personally, I've wanted scoped threads a few times, and having them in libstd seems appropriate to me. I agree that restricting threads to 'static feels oddly underpowered and restrictive given the awesomeness and flexibility of the rest of libstd.

That said, I also don't mind adding crossbeam as a dependency when I need this. Often enough, I'm already using crossbeam for something else in the same project.

Also, @stjepang thanks for all your work on crossbeam! I'm continually impressed by the high quality of the concurrency primitives in there... one of my favorite crates.

[Diggsey]

Presumably if a scoped thread returns a value, but is never joined, the return value is dropped at the end of the scope? Or is it dropped when the scoped thread handle is dropped? Might be worth specifying that just for completeness.

Motivation-wise, the fact that it improves the basic concurrency examples so much is the argument I find most convincing for it to be in std. Especially with how important those examples are for showing off the power of the language.

[matklad]

Another reason to have this in std is that scoped threads are arguably a better default than non-scoped threads. With std::Thread, it is possible (I did this in a blog post 😭) to forget to call .join, and that is problematic because then the thread might be abruptly terminated when main thread exits, which prevents drops from occurring. Scoped threads guarantee that the thread is joined.

EDIT: obligatory link to the structured concurrency blog post.

Crossbeam has had scoped threads since Rust 1.0.

I believe this was something like 1.2 though :) crossbeam-rs/crossbeam@c0714a5

[RalfJung]

There's some crazy unsafety in there that was incredibly tricky to get right. Suffice to say that even after the leakpocalypse, Crossbeam had unsound scoped threads for three years! That's pretty non-trivial. :)

Seems I forgot about that or missed it... do you have a pointer for the bug that was only discovered after 3 years?

[RalfJung]

The closure passed to Scope::spawn() now takes a &Scope<'env> argument that allows you to spawn nested threads, which was not possible with the old design. Rayon similarly passes a reference to child tasks.

I am a bit surprised that the thread's closure cannot just use the outer spawn handle. I suppose that is because the closure must outlive 'env but we only have the handle for the anonymous lifetime in &'_ Scope<'env>.

That makes me wonder why you let the user choose 'env. The user might pick a 'env that is bigger than what one might expect, i.e., I could get a &Scope<'static>. There's nothing unsound about that from what I can see (it just means that all spawned closures must outlive 'static), but it does mean that your diagram

'variables_outside: 'env: 'scope: 'variables_inside

is somewhat misleading. 'env can actually be arbitrarily bigger than what is indicated here.

What I would have expected is something more like

fn scope<F, T>(f: F) -> Result<T>
where
    F: for<'env> FnOnce(&'env Scope<'env>) -> T;

impl<'env> Scope<'env> {
    fn spawn<'scope, F, T>(&'scope self, f: F) -> ScopedJoinHandle<'scope, T>
    where
        F: FnOnce() -> T + Send + 'env,
        T: Send + 'env;
}

Wouldn't that also let the spawned thread use the outer Scope, because now it would outlive 'env? In fact at this point one should probably have some kind of type alias for &'env Scope<'env> to avoid the repetition. I suppose that doesn't work but I'd like to see more documentation about why.

[stjepang]

Replying to @Centril, @Diggsey, and @RalfJung

Given this, my only comment is that I would like there to be an equal crazy amount of comments justifying the usage of every unsafe impl and unsafe { ... } in the code that we add.

Absolutely! We have plenty of comments already, but could be even more thorough 👍

Presumably if a scoped thread returns a value, but is never joined, the return value is dropped at the end of the scope? Or is it dropped when the scoped thread handle is dropped?

The return value cannot be dropped when the handle is dropped because at that point the thread may not be finished yet. :) So it gets dropped whenever the thread is joined, which could be automatically at the end of the scope.

Seems I forgot about that or missed it... do you have a pointer for the bug that was only discovered after 3 years?

It's here: crossbeam-rs/crossbeam-utils#36

The issue is that we need to paremetrize ScopedJoinHandle over 'scope so that it doesn't escape the scope.

Reproducible example:

fn main() {
    let mut greeting = "Hello world!".to_string();
    let res = crossbeam::scope(|s| s.spawn(|| &greeting)).join();

    greeting = "DEALLOCATED".to_string();
    drop(greeting);

    println!("thread result: {:?}", res);
}

Output: thread result: pjQ7VTED

However, today the compiler emits warnings, so maybe that was just an issue with the old borrowck and we didn't know back then:

  = warning: this error has been downgraded to a warning for backwards compatibility with previous releases
  = warning: this represents potential undefined behavior in your code and this warning will become a hard error in the future

Now I have a feeling we probably don't need to parametrize ScopedJoinHandle over 'scope anyway and might even use JoinHandle instead of ScopedJoinHandle, making the API even smaller!

I am a bit surprised that the thread's closure cannot just use the outer spawn handle. I suppose that is because the closure must outlive 'env but we only have the handle for the anonymous lifetime in &'_ Scope<'env>.

Yes, you got that right.

The user might pick a 'env that is bigger than what one might expect, i.e., I could get a &Scope<'static>.

Correct.

I believe the problem with

fn scope<F, T>(f: F) -> Result<T>
where
    F: for<'env> FnOnce(&'env Scope<'env>) -> T;

impl<'env> Scope<'env> {
    fn spawn<'scope, F, T>(&'scope self, f: F) -> ScopedJoinHandle<'scope, T>
    where
        F: FnOnce() -> T + Send + 'env,
        T: Send + 'env;
}

is that, while in theory it should work, the borrowck only does local reasoning and cannot infer all the relationships between lifetimes.

More concretely, in a spawn() invocation, 'env could be literally anything and the borrowck cannot figure out whether F: 'env is true. If you try scoped threads with this signature on a simple example, it won't compile:

error[E0373]: closure may outlive the current function, but it borrows `*counter`, which is owned by the current function
  --> crossbeam-utils/tests/thread.rs:17:34
   |
17 |         let handle = scope.spawn(|| {
   |                                  ^^ may outlive borrowed value `*counter`
18 |             counter.store(1, Ordering::Relaxed);
   |             ------- `*counter` is borrowed here
help: to force the closure to take ownership of `*counter` (and any other referenced variables), use the `move` keyword
   |
17 |         let handle = scope.spawn(move || {
   |                                  ^^^^^^^

But if we introduce a new lifetime with fn scope<'env, F, T>, then borrowck knows that 'env is a concrete lifetime that outlives this scope() invocation. I'm not an expert at borrow checking, so apologies for the handwavy explanation.

[RalfJung]

Reproducible example:

So, what this does is it joins after the scope has ended. And why is that a problem? Is it because the scope has already joined, so now we are double-joining?

Output: thread result: pjQ7VTED

However, today the compiler emits warnings, so maybe that was just an issue with the old borrowck and we didn't know back then:

Oh, so it forgets that greeting is borrowed while res is live? Yeah that looks like a rustc bug. Seems that can lead to UB even in safe code.

However I have not managed to actually get the warning you were talking about, can you reproduce that in a self-contained example?

More concretely, in a spawn() invocation, 'env could be literally anything and the borrowck cannot figure out whether F: 'env is true. If you try scoped threads with this signature on a simple example, it won't compile:

Oh, good point. With 'env quantified like that, it is our choice as the implementer of spawn which lifetime we use there -- and there's no upper bound, so nothing will outlive that lifetime.

Looks like indeed quantifying it the way you did is expressing the right thing. Ideally we would communicate to the borrow checker that 'env is equal to the lifetime of the scope closure, but that does not seem possible.

[RalfJung]

Oh, turns out my code does show a migration warning, but playground doesn't show it because there's non-UTF-8 data.

@stjepang but you said you had an example that would actually compile with Rust 2015? Mine does not.

[stjepang]

@RalfJung

So, what this does is it joins after the scope has ended. And why is that a problem? Is it because the scope has already joined, so now we are double-joining?

This is not a problem if we ensure the scope joins the thread and saves the returned result in an Arc<Mutex<_>> and join_handle.join() simply extracts the saved result. Do you agree?

I honestly don't remember anymore why I wanted to prevent join handles from escaping the scope - maybe UB was possible in an older rustc version because the borrowck had a soundness hole? Or maybe I was just overly cautious and conservatively made ScopedJoinHandle generic over 'scope.

Oh, turns out my code does show a migration warning, but playground doesn't show it because there's non-UTF-8 data.

Haha, that's the magic of UB! ✨ If you click "..." next to the "RUN" button and choose "Build", the playground will show warnings.

[stjepang]

All right, I've merged JoinHandle and ScopedJoinHandle together, which makes the API smaller and simplifies the lifetime soup. Also expanded the section on rationale and drawbacks with some additional motivation.

[anp]

@pitdicker Sure!

There are two designs scoped threads went through. The old one is from the time thread::scoped() got removed and we wanted a sound alternative for the Rust 1.0 era. The new one is from the last year's big revamp.

Old: https://docs.rs/crossbeam/0.2.12/crossbeam/fn.scope.html
New: https://docs.rs/crossbeam/0.7.1/crossbeam/fn.scope.html

There are several differences between the old and new scoped threads:

1. scope() now returns a thread::Result<R> rather than R. This is because panics in the old design were just silently ignored, which is not good. By returning a Result, the user can handle panics in whatever way they want.
2. The closure passed to Scope::spawn() now takes a &Scope<'env> argument that allows you to spawn nested threads, which was not possible with the old design. Rayon similarly passes a reference to child tasks.
3. We removed Scope::defer() because it's not really useful, had bugs, and had a non-obvious behavior.
4. Some soundness bugs around lifetimes got fixed.

This is a really interesting and useful summary! Would it be reasonable to include this in the prior art section? Having an official doc summarizing the path taken here would be awesome.

[Lokathor]

Neat, but I deeply object to the "None" under Future Possibilities: The obvious next step is to have a ScopedThreadPool be put into std on top of this.

[oliver-giersch]

It's been a while since I've had anything to do with scoped threads, so I implemented a minimal version without nested spawns the way I remembered things:

https://gist.github.com/oliver-giersch/8878d769e47bd97b96aa6833f01d91eb

I don't see how you could use the regular JoinHandle type for scoped threads. What exactly prevents join handles from being leaked outside of the closure and how would you express the fact, that either the user or the scope function can call join on the handle?

[stjepang]

I don't see how you could use the regular JoinHandle type for scoped threads

A JoinHandle could be represented as:

pub struct JoinHandle<T>(Inner<T>);

enum Inner<T> {
    NormalHandle(/* ... */),
    ScopedHandle(/* ... */),
}

What exactly prevents join handles from being leaked outside of the closure

There's nothing wrong with leaking join handles outside of the closure as long as the thread is joined before the scope is exited.

how would you express the fact, that either the user or the scope function can call join on the handle?

Both JoinHandle and scope() would hold a reference to the actual join handle Arc<Mutex<ActualJoinHandle<()>> and an Arc<Mutex<io::Result<T>>> that gets filled with the join result. The thread will be joined either when JoinHandle is manually joined or when scope() exits, whatever happens first.

Note that join_handle.join() will join the thread if called inside the scope, or return the result saved by scope() if called outside the scope.

[oliver-giersch]

I see, I wasn't aware that this RFC proposes to change to current structure/implementation of the regular JoinHandle, maybe this part could be made a little clearer.

However, I am doubtful that this could work without breaking the current API. Currently, JoinHandle has two methods:

pub fn thread(&self) -> &Thread { ... }
pub fn join(self) -> Result<T, Box<dyn Any + Send + 'static>> { ... }

I am not certain a 'leaked' scoped join handle could return a &Thread reference, after its thread has already been joined, for instance. The return type would have to be changed to Option<&Thread> I suppose.
Another problematic case is the Err case for the join result. I came across this when I tried implementing something like proposed change, even when using separate ScopedJoinHandle type.
Consider the case of a panicking scoped thread and a leaked join handle:
At the end of the scope, the 'inner' join handle is definitely joined and an Err(Box<Any + Send + 'static>) is returned due to the panic. This boxed error is moved into a Vec with potential other errors and the vector returned by the scope function.
When join() is called on a leaked scoped join handle, it can not return the same boxed error, since it has already been returned by the scope function.

I suppose it would be possible to split the Result<T, Box<Any + Send + 'static>> into two separate results, one to be returned by the scope function (with Ok(()) in case of a successful join) and one to be returned by the join handle (with Err(())), but this would also be incompatible with the current API.

EDIT
Perhaps it would be best to let the scope function to return a different error in this case and allow the 'leaked' (scoped) join handle to return the actual error.

[bjorn3]

I am not certain a 'leaked' scoped join handle could return a &Thread reference, after its thread has already been joined, for instance. The return type would have to be changed to Option<&Thread> I suppose.

A thread could already have exited before calling .thread() on a JoinHandle. As far as I know that doesn't crash or panic either.

[oliver-giersch]

I am not certain a 'leaked' scoped join handle could return a &Thread reference, after its thread has already been joined, for instance. The return type would have to be changed to Option<&Thread> I suppose.

A thread could already have exited before calling .thread() on a JoinHandle. As far as I know that doesn't crash or panic either.

You could be correct, I am not sure. However, there may be a difference between a thread having already exited and it being explicitly joined. This is probably the only (potential) issue, that might impede using JoinHandle for scoped threads, the other point I've brought up is likely an implementation detail that could be solved later.

[stjepang]

@oliver-giersch I believe you're right in that it's not possible to extract the panic (the Err result returned by join()) from a JoinHandle that has escaped the scope. For that reason, I've reverted the last change and am now convinced it's best not to allow join handles to escape the scope.

Alternatively, we could change behavior of scope() to return a T rather than Result<T> so that the scope only waits for threads to finish rather than joins threads. However, with that API, one would have to manually join all threads and unwrap the results to propagate panics into the parent thread, which is less ergonomic and more prone to errors.

[oliver-giersch]

@oliver-giersch I believe you're right in that it's not possible to extract the panic (the Err result returned by join()) from a JoinHandle that has escaped the scope. For that reason, I've reverted the last change and am now convinced it's best not to allow join handles to escape the scope.

Alternatively, we could change behavior of scope() to return a T rather than Result<T> so that the scope only waits for threads to finish rather than joins threads. However, with that API, one would have to manually join all threads and unwrap the results to propagate panics into the parent thread, which is less ergonomic and more prone to errors.

I do not know how relevant this implementation stuff is for the RFC at this point, so reign me in if this goes to far at this point ;)
It is possible to extract the error, but it is rather complicated and not necessarily obvious behaviour.
When all threads are joined at the end of the scope, e.g. Arc::strong_count can be used to check if there exists a leaked join handle. If so, the result can be stored so the leaked handle can extract it later and a boxed dummy error returned by the scope.

The overall benefit from being able to leak join handles soundly does not outweigh this, so I'd definitely agree with scrapping this idea and keeping a dedicated and unleakable ScopedJoinHandle<'scope, T>.

[oliver-giersch]

I would also suggest changing the return type of the scope function to Result<T, Box<[Box<dyn Any + Send + 'static>]>>, since I don't think there is a good reason (semantically) why this function should return a thread::Result. The return value of the scope itself is not necessarily thread related and can in fact be almost anything (the only bound is T: 'env).
Also, the alternative signature makes it clearer that actually the collected failure results of (potentially) several threads are returned. With Box<Any> you need to know (from reading the documentation), which type is actually returned and then make the correct downcast (which is unergonomic). I'd assume most people would at most want to know the number of panicked threads (i.e. the length of the vector/slice) and returning a boxed slice allows you to get this value easily. Using a boxed slice also communicates intent of the type better, since there is no reason why anybody would want to e.g. add more elements to a vector of returned errors. Last but not least, it also allows us to avoid the awkward boxing of a Vec just to shoehorn it into this specific type alias.