Alan wants structured concurrency and parallel data processing

[nikomatsakis]

Brief summary

Alan wants to intermix data processing and he finds it difficult. He misses Kotlin's support for coroutines. Barbara laments the lack of structured concurrency or Rayon-like APIs.

Optional details

• (Optional) Which character(s) would be the best fit and why?
  • Alan: the experienced "GC'd language" developer, new to Rust
  • Grace: the systems programming expert, new to Rust
  • Niklaus: new programmer from an unconventional background
  • Barbara: the experienced Rust developer
• (Optional) Which project(s) would be the best fit and why?
  • Not sure.
• (Optional) What are the key points or morals to emphasize?
  • Async libraries are very focused on I/O, but many folks mention wanting improved support for parallel tasks.

[jzrake]

Two cents from the computational astrophysicist -- our work is indeed largely CPU-bound, not IO-bound. We have many compute tasks that run in parallel, but then must block waiting for the completion of "neighbor" tasks. Co-routines satisfy this, and the future / executor strategy I'm currently using "works", but it's sub-optimal:

• none of my students can understand it
• long compile times with async code
• error propagation is awkward
• compute closures must be 'static (see Scoped tasks tokio-rs/tokio#3162)

Here is an example of how we are currently using async blocks and the Tokio executor. Note how runtime.spawn(future).map(|f| f.unwrap()) is littered in with computations. Also note the excessive use of Arc and Clone due to 'static requirement. We cannot convince Tokio the borrowed data will outlive the task, even though we know it will.

I should also mention that we're only CPU-bound for single-node / multi-core calculations. On distributed-memory systems the "neighbor" tasks must be communicated across nodes, and it would be nice to treat upstream async tasks uniformly, whether it's a compute task or a network request.

In regards to Niko's effort to collect priorities from various user bases, here are ours:

1. code correctness, reproducibility
2. performance (< C will not do)
3. stupidness: I'm working with astronomy grad students and post-docs; we're not CS PhD's
4. portability: HPC sites typically have federalized module systems and Rust restores control to the scientist

[eminence]

A quick question about your specific case @jzrake -- what motivated you to use async for your code in the first place? If it was just a single-node computation, I'm guessing that a non-async threadpool (or rayon) would have worked well. Did async stem from the need to support multi-node computations (where async to talk over the network is useful)? Or was it something else?

[jzrake]

@eminence -- I've had the impression that an async executor would minimize core idling from blocking tasks. Since .await signals the executor that a task is blocking, it can go look for tasks that could make progress. We do attain 60-70% core scaling with this approach, so it's at least viable, but the code's, well, kind of ugly. As I'm learning more, it seems like this compute pattern can be implemented with parallel iterators, and if so I much prefer the simplicity. That's why I got in touch with Niko ;)

[velvia]

As a data engineer who came to Rust from Scala and has worked extensively with Spark and Scala's fabulous async ecosystem, I'd like to offer some thoughts (and if I have time, could try writing up this story as well).

Points:

• I find what I'm missing from Scala in Rust the most are the fabulous Future and async stream processing libraries. Example is https://monix.io, with an extremely rich Observable API (this is like the Stream). I'm aware of and have used Rust's streams, stream-ext, and stream-ext-try. What I miss from Scala: not having to worry about lifetimes, whether I need to use async or async move, cloning or making something Arc, etc; not having to worry about Result and how that completely changes what operators I need to use in the backend to extract out the Results; and the overall completeness and smoothness of the APIs.
• It's not the easiest to figure out exactly what paradigm to use, and to debug error messages from that. For example, let's say I want to start really simple. I have this:

   let writes = batches.iter().map(|batch| self.serialize_batch_to_parquet(....) ).collect::<Vec<_>>();
   join_all(writes)

I just want to join a list of multiple futures.... this would be super easy pattern in Scala. In Rust the above, if serialize_to_parquet is async and takes &mut self, this does not compile, because the compiler believes we cannot have multiple futures that mutate self I think. In this case however I can guarantee that self is not really mutated at the same Time (actually self is not mutated at all, it is just mut because it is a "write" function that mutates data on disk) Maybe this isn't a great example, but there are frictions like that to think through. I'm not sure how to express it properly.

As an aside, concurrent processing is very different from the "data processing" that most people do. Most data scientists and engineers aren't really distributed systems people, and use tools without understanding async. They mostly come from Python (is there a character for the Python data scientist? If there isn't, there should be, this is a very important category! I guess Alan is that person, but JVM is very different from Python/Ruby). They might use tools like Spark, or Ray, which enable parallelism and distribution, but without the user really having to understand it. They write code that is almost always single threaded. So keep that in mind - the key to making async work for these folks is likely to make async "disappear" effectively behind other APIs which "look blocking".

[pickfire]

Up till now parallel processing is definitely very easy. Just change iter to iter_par but that won't work for sync rust. At least everything needs to change to async function. It is still easy (when not from scratch) but requires more effort than a single line change. Changes include:

• figuring out which function and all code within it that blocks (it's like everything is unsafe and you need to audit everything to prevent getting deadlock)
• if any issue with sync, need to change it to async counterpart, like sync lock to async lock
• changing function signatures to async
• adding await points for all async functions
• use join to merge multiple futures together and await

I think only the first and second part is more troublesome such that a lot of stuff needs to be check one by one.

[smurfix]

Some links WRT Structured Concurrency to consider:

https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
https://en.wikipedia.org/wiki/Structured_concurrency

Frankly I consider SC to be essential.

[velvia]

What are the gaps in Rust today wrt structured concurrency? One could argue that waiting for async work to finish and being able to return errors in a natural way that fits the language paradigm is what current async-await is designed to do already.

…

On Jul 11, 2021, at 11:31 PM, Matthias Urlichs ***@***.***> wrote: Some links WRT Structured Concurrency to consider: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ <https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/> https://en.wikipedia.org/wiki/Structured_concurrency <https://en.wikipedia.org/wiki/Structured_concurrency> Frankly I consider SC to be essential. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#107 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAIDPW6NG5SJQJKREVVKLLLTXKD4HANCNFSM42AG43FQ>.

[smurfix]

There's also the ability to cancel other tasks seamlessly. You need them for a heap of reasons I could expound upon for the next ten pages; suffice it to say that SC with cancellation is what makes any nontrivial code using Python+asyncio (unstructured, abstraction: tasks+futures) a tedious and barely-debuggable mess while Python+trio (structured, has scoped cancellation, doesn't have/need user-visible Task nor Future classes) is the polar opposite IME.

I'd like async Rust to be the latter, and I'd go as far as to say that if Niklaus and Barbara want to keep their sanity they need it to be the latter.

https://rust-lang.github.io/async-book/01_getting_started/04_async_await_primer.html has these nonexisting chapters 6.3 thru 6.5 …

[smurfix]

NB, it's possible to code up an SC-style library using Future/Task as low-level primitives. The Python anyio library does this, so writing the same thing in Rust is certainly possible.

Still needs decent cancellation support though.

[mmirate]

Current cancellation system makes this impossible, as reminded here: https://www.reddit.com/r/rust/comments/wltcf8/announcing_rust_1630/ijwx3mz/

[npuichigo]

Can be leverage this from C++ community? https://github.com/kirkshoop/async_scope/blob/main/asyncscope.md