Possible stack overflow using await! inside generator

[realcr]

Hi, I am dealing with a stack overflow issue in my code when using futures-preview = "0.3.0-alpha.9". I believe that it might be related to the futures crate. Unfortunately I could not reproduce this using a minimal example.

I am using rustc 1.32.0-nightly (65204a97d 2018-11-12), my other dependencies are:
log, pretty_env_logger, untrusted, bytes, futures-cpupool, futures-preview, num-biginit, num-traits, serde, serde-derive, serde-json, base64, atomicwrites, im, byteorder, ring, rand.
I have no unsafe {} clauses in my code. The stack overflow occurs in one test case in my code.

The test case contains the following two lines (Inside an async function):

loop {} // I added this line for debugging purposes
let (outgoing_comms, _outgoing_control) = await!(apply_funder_incoming(funder_incoming, &mut state2, &mut ephemeral2, 
                                 rng.clone(), identity_client2.clone())).unwrap();

The code compiles successfully. When run, it is stuck in an infinite loop. However, if the loop {} is moved after the await!() line, I get a stack overflow:

$ cargo test -p offst-funder handler_pair
   Compiling offst-funder v0.1.0 (/home/real/projects/d/offst/components/funder)

    Finished dev [unoptimized + debuginfo] target(s) in 5.14s
     Running target/debug/deps/offst_funder-2301b5d9d090c072

running 1 test

thread 'handler::tests::test_handler_pair_basic' has overflowed its stack
fatal runtime error: stack overflow
error: process didn't exit successfully: `/home/real/projects/d/offst/target/debug/deps/offst_funder-2301b5d9d090c072 handler_pair` (signal: 6, SIGABRT: process abort signal)

Therefore I believe that the stack overflow occurs when the await!() statement is executed.

I also noticed that if I split the await!() line to two lines (with a temporary fut variable):

loop {}
let fut = apply_funder_incoming(funder_incoming, &mut state2, &mut ephemeral2, 
                                 rng.clone(), identity_client2.clone());
let (outgoing_comms, _outgoing_control) = await!(fut).unwrap();

A stack overflow occurs. This probably means that it happens before the loop {} line in this case. The fact that the stack overflow changes its place makes me believe it might be related to the generator that contains the async code, though I'm not sure about it.

The full code is here: freedomlayer/offst@876ddf3
The stack overflow occurs at the test_handler_pair_basic test case.

To reproduce, run the following:

sudo apt install capnproto
git clone https://github.com/freedomlayer/offst
cd offst
git checkout 876ddf389
rustup override set nightly-2018-11-13
cargo test -p offst-funder test_handler_pair_basic

I realize that you might not have the time to check this issue, because the reproducing example is very large. In such a case, is there anything I can do to help solve this problem?

[realcr]

Update: I managed to simplify the test case that causes the stack overflow a bit.

async fn task_reproduce_overflow(identity_client1: IdentityClient, 
                                 identity_client2: IdentityClient) {
    // Sort the identities. identity_client1 will be the first sender:
    let pk1 = await!(identity_client1.request_public_key()).unwrap();
    let pk2 = await!(identity_client2.request_public_key()).unwrap();
    let (identity_client1, pk1, identity_client2, pk2) = if is_public_key_lower(&pk1, &pk2) {
        (identity_client1, pk1, identity_client2, pk2)
    } else {
        (identity_client2, pk2, identity_client1, pk1)
    };

    let mut state1 = FunderState::<u32>::new(&pk1);
    let mut ephemeral1 = FunderEphemeral::new(&state1);
    let mut state2 = FunderState::<u32>::new(&pk2);
    let mut ephemeral2 = FunderEphemeral::new(&state2);

    let rng = RngContainer::new(DummyRandom::new(&[3u8]));

    // Initialize 1:
    let funder_incoming = FunderIncoming::Init;
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
    await!(apply_funder_incoming(funder_incoming.clone(), &mut state1, &mut ephemeral1, rng.clone(), identity_client1.clone())).unwrap();
}

If I remove the last await!() line, the stack overflow will not happen. I hope to be able to further simplify the code to get a smaller example.

EDIT: If I use a for loop instead of inlining all the await!() lines, the stack overflow does not occur.

[cramertj]

Yes, using await! a lot like this on a large future in a large function will cause the compiler to generate massive types which will overflow the stack. We need something like rust-lang/rust#52924 in order to prevent this.

[realcr]

@cramertj : Thanks for the quick reply!
What do you recommend as a workaround for this issue at the moment?

[cramertj]

@realcr I'd recommend breaking large functions out into smaller ones and wrapping the futures generated by the sub-functions in Box::pinned.

[realcr]

@cramertj : Thank you for this advice, it allowed me to make my tests green again!

I want to note though that I'm still worried about this issue. I am used to be able to write whatever I want with Rust, and not bump into crashes. If there is some upper limit for the size of generator futures (Imposed by the upper limit size for the stack), maybe a program could work well for me in my tests, but when I hand it over to someone else the program will suddenly crash because in his environment the stack is of different length?

In addition, the stack overflow crash does not give any information about the source of the problem. Therefore it was difficult for me to understand if the problem is with futures, or with another crate I was using. It's my first stack overflow in Rust. Am I expected to get a similar error message if I do something like infinite recursion using a function that calls itself?

Maybe it is possible to have some compile time protection that gives a warning (or even an error) in a case where a future is too large?

[cramertj]

All the problems you highlight are real and are concerning, but they're the same for any program which uses a lot of stack space (e.g. is highly recursive, makes large arrays on the stack, etc.). There are tools under development to help catch functions and types which may cause overly deep stacks to be created, but none of this is specific to futures-- it's a general problem with large types in any language which uses fixed-sized stacks.