Not able to await on write! in spawned async task

[rylev]

This is very easy to reproduce:

use async_std::prelude::*;

fn main() {
    async_std::task::spawn(async {
        let mut buf: Vec<u8> = vec![];
        write!(&mut buf, "foo").await;
    });
}

This leads to the following error:

error[E0277]: `*mut (dyn std::ops::Fn() + 'static)` cannot be shared between threads safely
  --> src/main.rs:4:5
   |
4  |     async_std::task::spawn(async {
   |     ^^^^^^^^^^^^^^^^^^^^^^ `*mut (dyn std::ops::Fn() + 'static)` cannot be shared between threads safely
   |
  ::: /Users/ryanlevick/.cargo/registry/src/github.com-1ecc6299db9ec823/async-std-1.0.1/src/task/spawn.rs:28:29
   |
28 |     F: Future<Output = T> + Send + 'static,
   |                             ---- required by this bound in `async_std::task::spawn::spawn`
   |
   = help: within `core::fmt::Void`, the trait `std::marker::Sync` is not implemented for `*mut (dyn std::ops::Fn() + 'static)`
   = note: required because it appears within the type `std::marker::PhantomData<*mut (dyn std::ops::Fn() + 'static)>`
   = note: required because it appears within the type `core::fmt::Void`
   = note: required because of the requirements on the impl of `std::marker::Send` for `&core::fmt::Void`
   = note: required because it appears within the type `std::fmt::ArgumentV1<'_>`
   = note: required because it appears within the type `[std::fmt::ArgumentV1<'_>; 0]`
   = note: required because it appears within the type `for<'r, 's, 't0, 't1, 't2, 't3, 't4, 't5, 't6, 't7, 't8, 't9, 't10, 't11, 't12, 't13, 't14, 't15> {std::vec::Vec<u8>, std::vec::Vec<u8>, &'r mut std::vec::Vec<u8>, &'s mut std::vec::Vec<u8>, fn(&'t0 [&'t0 str], &'t0 [std::fmt::ArgumentV1<'t0>]) -> std::fmt::Arguments<'t0> {std::fmt::Arguments::<'t0>::new_v1}, &'t1 str, &'t2 str, [&'t3 str; 1], &'t4 [&'t5 str], &'t6 [&'t7 str; 1], (), [std::fmt::ArgumentV1<'t8>; 0], &'t9 [std::fmt::ArgumentV1<'t10>], &'t11 [std::fmt::ArgumentV1<'t12>; 0], std::fmt::Arguments<'t13>, async_std::io::write::write_fmt::WriteFmtFuture<'t14, std::vec::Vec<u8>>, async_std::io::write::write_fmt::WriteFmtFuture<'t15, std::vec::Vec<u8>>, ()}`
   = note: required because it appears within the type `[static generator@src/main.rs:4:34: 7:6 for<'r, 's, 't0, 't1, 't2, 't3, 't4, 't5, 't6, 't7, 't8, 't9, 't10, 't11, 't12, 't13, 't14, 't15> {std::vec::Vec<u8>, std::vec::Vec<u8>, &'r mut std::vec::Vec<u8>, &'s mut std::vec::Vec<u8>, fn(&'t0 [&'t0 str], &'t0 [std::fmt::ArgumentV1<'t0>]) -> std::fmt::Arguments<'t0> {std::fmt::Arguments::<'t0>::new_v1}, &'t1 str, &'t2 str, [&'t3 str; 1], &'t4 [&'t5 str], &'t6 [&'t7 str; 1], (), [std::fmt::ArgumentV1<'t8>; 0], &'t9 [std::fmt::ArgumentV1<'t10>], &'t11 [std::fmt::ArgumentV1<'t12>; 0], std::fmt::Arguments<'t13>, async_std::io::write::write_fmt::WriteFmtFuture<'t14, std::vec::Vec<u8>>, async_std::io::write::write_fmt::WriteFmtFuture<'t15, std::vec::Vec<u8>>, ()}]`
   = note: required because it appears within the type `std::future::GenFuture<[static generator@src/main.rs:4:34: 7:6 for<'r, 's, 't0, 't1, 't2, 't3, 't4, 't5, 't6, 't7, 't8, 't9, 't10, 't11, 't12, 't13, 't14, 't15> {std::vec::Vec<u8>, std::vec::Vec<u8>, &'r mut std::vec::Vec<u8>, &'s mut std::vec::Vec<u8>, fn(&'t0 [&'t0 str], &'t0 [std::fmt::ArgumentV1<'t0>]) -> std::fmt::Arguments<'t0> {std::fmt::Arguments::<'t0>::new_v1}, &'t1 str, &'t2 str, [&'t3 str; 1], &'t4 [&'t5 str], &'t6 [&'t7 str; 1], (), [std::fmt::ArgumentV1<'t8>; 0], &'t9 [std::fmt::ArgumentV1<'t10>], &'t11 [std::fmt::ArgumentV1<'t12>; 0], std::fmt::Arguments<'t13>, async_std::io::write::write_fmt::WriteFmtFuture<'t14, std::vec::Vec<u8>>, async_std::io::write::write_fmt::WriteFmtFuture<'t15, std::vec::Vec<u8>>, ()}]>`
   = note: required because it appears within the type `impl std::future::Future`

This seems to be caused by core::fmt::Void not being Send. I would expect this to be able to work.

[skade]

Acknowledged, that's a serious problem of the std family of macros, they are not Send.

[rylev]

@skade any thoughts on using locks to help with this or do you think the ergonomics hit is not worth the performance penalty?

[skade]

/cc @stjepang @yoshuawuyts

We had some discussions around what the best solution is. We prefer a solution of some kind that works, but are not exactly sure which one.

[yoshuawuyts]

The two options seem to be: either we introduce spawn_local, or std make Arguments Send.

The latter seems really hard but yields the best ergonomics, but the former we could do ourselves (likely at the cost of some ergonomics).

[yoshuawuyts]

It seems format!("hello {}", "world"); doesn't work inside spawned tasks either. This should probably be passed back to the libs team.

[yoshuawuyts]

Related issue from the compiler: rust-lang/rust#64856

[jimblandy]

It seems like format! can be used in cross-thread spawns now.

This works fine:

use async_std::task;
use async_std::io;
use async_std::io::prelude::*;

async fn write_hello() {
    let msg = format!("Hello, World!\n");
    io::stdout().write_all(msg.as_bytes()).await.unwrap()
}

fn main() {
    let handle = task::spawn(write_hello());

    task::block_on(handle);
}

But write! is still not Send:

  --> src/bin/write-macro-is-send.rs:12:18
   |
12 |     let handle = task::spawn(write_hello());
   |                  ^^^^^^^^^^^ `core::fmt::Opaque` cannot be shared between threads safely
   | 
  ::: .../async-std-1.6.0/src/task/spawn.rs:28:29
   |
28 |     F: Future<Output = T> + Send + 'static,
   |                             ---- required by this bound in `async_std::task::spawn::spawn`
   |

[viruscamp]

I'm sorry to delete the old code.

For async-std-1.9.0, macro async_std::write is Send now. But print println eprint eprintln are still !Send.

macro_rules! println {
    ($($arg:tt)*) => (async {
        let mut stdout = async_std::io::stdout(); // make stdout live longer than .await
        if let Err(e) = {
            let x = writeln!(stdout, $($arg)*);
            // drop Arguments<'_> and [ArgumentV1<'a>] to make them live shorter than .await
            x
        }.await {
            panic!("failed printing to stdout: {}", e);
        }
    });
}

It depends on that async_std::io::write::write_fmt::WriteFmtFuture does not hold an Arguments<'_>.