Repeated calls to the same `async` fn can cause stack overflow

[xmh0511]

I tried this code:

#[tokio::main]
async fn main() {
    test_buf().await;
    test_buf().await;
    test_buf().await;
    // ... Execute n times, do not use for loop
}
async fn test_buf(){
    let buf = [0;65536];
    test_buf0(&buf).await;
}
async fn test_buf0(_buf:&[u8]){}

I expected to see this code can work

Instead, this happened:

fatal runtime error: stack overflow

rustc --version --verbose:

rustc 1.82.0 (f6e511eec 2024-10-15)
binary: rustc
commit-hash: f6e511eec7342f59a25f7c0534f1dbea00d01b14
commit-date: 2024-10-15
host: x86_64-apple-darwin
release: 1.82.0
LLVM version: 19.1.1

This issue can also be directly reproduced on playground, since the stack size on Linux is larger than that on Windows, the n times that can reproduce the issue is greater than that on Windows

Exited with signal 6 (SIGABRT): abort program

Standard Error

   Compiling playground v0.0.1 (/playground)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 1.38s
     Running `target/debug/playground`

thread 'main' has overflowed its stack
fatal runtime error: stack overflow

The tokio side think this should be a bug of the compiler, tokio-rs/tokio#5909

[saethlin]

I'm not really sure what we should do with an issue like this. On the one hand, this is clearly a bummer.

But also, if you could see the desugaring, I think you would agree that this code clearly has problematic stack use.

So I'm concerned that the actual problem here is that the async/await syntax has hidden the state machine (which is the point!) and that has mislead your intuition. And I do not think we are going to solve that problem. Perhaps a lint could be added?

[yingmanwumen]

But also, if you could see the desugaring, I think you would agree that this code clearly has problematic stack use.

After desugaring to HIR, the .await is translated to:

|mut _task_context: ResumeTy| {
        match #[lang = "into_future"](test_buf()) {
                        mut __awaitee =>
                            loop {
                                    match unsafe {
                                                    #[lang = "poll"](#[lang = "new_unchecked"](&mut __awaitee),
                                                        #[lang = "get_context"](_task_context))
                                                } {
                                            #[lang = "Ready"] {  0: result } => break result,
                                            #[lang = "Pending"] {} => { }
                                        }
                                    _task_context = (yield ());
                                },
                    };

                match #[lang = "into_future"](test_buf()) {
                        // ...
                }
}

But I still cannot understand why the stack overflow happened. It seems like the sequential .awaits should not effect each other and the resource on the stack should be release or reused once a .await completed.

And I find that there're a lot of confusing allocations in the async closure refering to the LLVM IR generated by rustc:

; ...
; core::ptr::drop_in_place<test_async::main::{{closure}}>
; Function Attrs: uwtable
define internal void @"_ZN4core3ptr66drop_in_place$LT$test_async..main..$u7b$$u7b$closure$u7d$$u7d$$GT$17h38fd88d6d6090469E"(ptr align 8 %_1) unnamed_addr #1 personality ptr @rust_eh_personality !dbg !5776 {
; ...
  %_677 = alloca [65568 x i8], align 8
  %_676 = alloca [65568 x i8], align 8
  %_675 = alloca [65568 x i8], align 8
  %_660 = alloca [65568 x i8], align 8
  %_659 = alloca [65568 x i8], align 8
; ...

Apperantly those allocations cause the stack overflow but it is weird for me for I remember that the async/await in Rust is stackless...

I would be very appreciated if someone could tell me why...

[y21]

FWIW, clippy's large_stack_frames lint detects that this function uses an "unreasonable" amount of stack space: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=014d009b31899a67c88dbbe02ec9bccf

warning: this function may allocate 590480 bytes on the stack
 --> src/main.rs:3:1
  |
3 | #[tokio::main]
  | ^^^^^^^^^^^^^^
...
7 |     test_buf().await;
  |     ---------------- this is the largest part, at 65568 bytes for type `{async fn body of test_buf()}`
  |
  = note: 590480 bytes is larger than Clippy's configured `stack-size-threshold` of 512000
  = note: allocating large amounts of stack space can overflow the stack and cause the program to abort

[saethlin]

but it is weird for me for I remember that the async/await in Rust is stackless...

Yes we have stackless coroutines and essentially, that is the problem. Coroutines don't get their own stacks, so all their storage is placed on one stack.

[Jarcho]

Just adding in, a release build compiles this just fine. It's only an unoptimized build that doesn't overlap the arrays.