Skip to content

get_all_awaited_by() shows incorrect call stacks in awaited_by relationships #135371

Closed
@pablogsal

Description

@pablogsal

The current asyncio debugging tools (python -m asyncio ps and python -m asyncio pstree) provide limited visibility into complex async execution patterns, making it extremely difficult to debug production asyncio applications.

The fundamental issue is that the current implementation only shows external task dependencies without revealing the internal coroutine call stack. This creates a wrong output because developers cannot see where within a task's execution the code is currently suspended.

Current vs Correct Output

Consider this example of a complex async application with nested coroutines:

async def foo1():
    await foo11()

async def foo11():
    await foo12()

async def foo12():
    x = asyncio.create_task(foo2(), name="foo2")
    x2 = asyncio.create_task(foo2(), name="foo2")
    await asyncio.gather(x2, x, return_exceptions=True)

async def foo2():
    await asyncio.sleep(500)

async def runner():
    async with taskgroups.TaskGroup() as g:
        g.create_task(foo1(), name="foo1.0")
        g.create_task(foo1(), name="foo1.1") 
        g.create_task(foo1(), name="foo1.2")
        g.create_task(foo2(), name="foo2")
        await asyncio.sleep(1000)

asyncio.run(runner())

Current Implementation

The existing debugging output makes it nearly impossible to understand what's happening:

Table output:

tid        task id              task name            coroutine chain                                    awaiter name         awaiter id     
---------------------------------------------------------------------------------------------------------------------------------------
2103857    0x7f2a3f87d660       Task-1                                                                                       0x0            
2103857    0x7f2a3f154440       foo1.0               sleep -> runner              Task-1               0x7f2a3f87d660 
2103857    0x7f2a3f154630       foo1.1               sleep -> runner              Task-1               0x7f2a3f87d660 
2103857    0x7f2a3fb32be0       foo2                 sleep -> runner       foo1.0           0x7f2a3f154440 

Tree output:

└── (T) Task-1
    └──  /home/user/app.py 27:runner
        └──  /home/user/Lib/asyncio/tasks.py 702:sleep
            ├── (T) foo1.0
            │   └──  /home/user/app.py 5:foo1
            │       └──  /home/user/app.py 8:foo11
            │           └──  /home/user/app.py 13:foo12
            │               ├── (T) foo2
            │               └── (T) foo2

This output is problematic because for the leaf tasks (like the foo2 tasks), there's no indication of their internal execution state - developers can't tell that these tasks are suspended in asyncio.sleep() calls.

Correct Implementation

The correct debugging output transforms the debugging experience by providing correct execution context:

Table output with dual information display:

tid        task id              task name            coroutine stack                                    awaiter chain                                      awaiter name    awaiter id     
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
2139407    0x7f70af08fe30       Task-1               sleep -> runner                                                                                                       0x0            
2139407    0x7f70ae424050       foo1.0               foo12 -> foo11 -> foo1                             sleep -> runner                                    Task-1          0x7f70af08fe30 
2139407    0x7f70ae542890       foo2                 sleep -> foo2                                      foo12 -> foo11 -> foo1                             foo1.0          0x7f70ae424050 

Tree output with complete execution context:

└── (T) Task-1
    └──  runner /home/user/app.py:27
        └──  sleep /home/user/Lib/asyncio/tasks.py:702
            ├── (T) foo1.0
            │   └──  foo1 /home/user/app.py:5
            │       └──  foo11 /home/user/app.py:8
            │           └──  foo12 /home/user/app.py:13
            │               ├── (T) foo2
            │               │   └──  foo2 /home/user/app.py:18
            │               │       └──  sleep /home/user/Lib/asyncio/tasks.py:702
            │               └── (T) foo2
            │                   └──  foo2 /home/user/app.py:18
            │                       └──  sleep /home/user/Lib/asyncio/tasks.py:702

The correct output immediately reveals crucial information that was previously hidden. Developers can now see that the foo2 tasks are suspended in sleep calls, the foo1.0 task is suspended in the foo12 function, and the main Task-1 is suspended in the runner function. This level of detail transforms debugging from guesswork into precise analysis.

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.14bugs and security fixes3.15new features, bugs and security fixesstdlibPython modules in the Lib dirtopic-asynciotype-bugAn unexpected behavior, bug, or error

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions