extmod/asyncio: Add Task methods from CPython

[imnotjames]

This updates the way that Task is defined in the C module for asyncio which implements tasks & task queues. The goal is to bring asyncio in MicroPython slightly closer to CPython.

This adds the following methods to Task:

• get_coro() - how CPython exposes the coroutine backing the task (CPython Docs)
• result() - a helper method from CPython for returning the successful response (CPython Docs)
• exception() - a helper method from CPython for returning the raised values (CPython Docs)
• cancelled() - helper for true / false if the task has been cancelled (CPython Docs)
• add_done_callback() - adds a callback to the task for completion or if already complete executes it (CPython docs)
• remove_done_callback() - removes a specific "done" callback from the task (CPython docs)

Adds a hash unary operation so Tasks can be added to sets like in CPython.

Also adds set_result and set_exception for consistency but neither of these do anything - they raise a RuntimeError because they aren't supported on Tasks. 🤷

[github-actions]

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64: +1760 +0.220% standard[incl +224(data)]
      stm32:  +620 +0.158% PYBV10
     mimxrt:  +672 +0.185% TEENSY40
        rp2:  +688 +0.210% RPI_PICO
       samd:  +664 +0.253% ADAFRUIT_ITSYBITSY_M4_EXPRESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (a968888) 98.43% compared to head (8d260d8) 98.43%.
Report is 13 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #13000   +/-   ##
=======================================
  Coverage   98.43%   98.43%           
=======================================
  Files         158      158           
  Lines       20978    21027   +49     
=======================================
+ Hits        20649    20698   +49     
  Misses        329      329           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[andrewleech]

I personally wouldn't worry about adding:

Also adds set_result and set_exception for consistency but neither of these do anything - they raise a RuntimeError because they aren't supported on Tasks.

Adding even function names for these costs space, and not adding them will just raise an exception anyway, it's really not much different!

[imnotjames]

I personally wouldn't worry about adding:

Also adds set_result and set_exception for consistency but neither of these do anything - they raise a RuntimeError because they aren't supported on Tasks.

Adding even function names for these costs space, and not adding them will just raise an exception anyway, it's really not much different!

I guess so. My concern was that the exception would be a different kind of error but that's probably ok.

[imnotjames]

Okay, dropped those functions and fixed the commit messages.

[dpgeorge]

Thanks for the contribution.

MicroPython aims to be compact so it can run on devices without many resources. As such, these kinds of methods/functions were left out of the asyncio implementation on purpose (among many others).

That said, we can add them if they are important and useful.

What motivated you to add these particular features? Do they add anything that you can't already do?

I don't think the done-callback functions are really needed. MicroPython doesn't support any kind of callback in asyncio. Instead you can wait on the task to complete then run the required code.

[dpgeorge]

This signature of the callback does not match CPython. In CPython the callback takes only one argument.

[imnotjames]

That's right - I'm hoping to follow up to bring this in line with CPython in cases where only one argument is accepted in the function in a follow up PR.

However, changing the callback signature here would mean it's different from where it gets called elsewhere.

[imnotjames]

I could add a comment to clarify that if it helps?

[dpgeorge]

Hashing of Task should actually work, since commit eaccaa3. So I think this bit of code can be removed.

[imnotjames]

I'll confirm it without this, it's very likely that's the case!

[imnotjames]

Yup, seems fine with that patch, awesome. Removed that change from this PR.

[peterhinch]

MicroPython doesn't support any kind of callback in asyncio. Instead you can wait on the task to complete then run the required code.

Definitely +1.

[imnotjames]

Thanks for the contribution.

Of course! :)

MicroPython aims to be compact so it can run on devices without many resources. As such, these kinds of methods/functions were left out of the asyncio implementation on purpose (among many others).

Compact is a great goal, and I do recognize that a lot of effort was put in to make asyncio as small as possible -- and also a lot of effort to make the compatibility with the CPython asyncio implementation as high as possible within that constraint.

That said, we can add them if they are important and useful.

What motivated you to add these particular features? Do they add anything that you can't already do?

All of these features expose existing "private" APIs (plumbing?) via the CPython public API (porcelain?). So everything can technically be done today - but in a way that's more brittle.

In particular, this PR heavily simplifies understanding if a task is cancelled or not, and if it's "done" what the result of it running was. It's not documented anywhere how the state or data properties works and I think that's intentional - it's internal aspects of the implementation that work well but shouldn't be touched by other parties.

The use case I had for this was to implement the wait and other waiting primitives in CPython as well as Task groups via these kinds of features.

The example CPython gives for tasks groups is as such:

async with asyncio.TaskGroup() as tg:
  task1 = tg.create_task(some_coro(...))
  task2 = tg.create_task(another_coro(...))
print(f"Both tasks have completed now: {task1.result()}, {task2.result()}")

Where the result is retrieved via task.result()

I don't think the done-callback functions are really needed. MicroPython doesn't support any kind of callback in asyncio.

I had added them because micropython kind of supports them via state (if you squint hard enough) but it's not clear without looking at the internals that they work that way, nor the rules you would need to follow so as not to break the standard async/await functionality. This encodes some of those rules into the add_done_callback and remove_done_callback methods.

Instead you can wait on the task to complete then run the required code.

If you await the task you'll get them serially rather than the result in parallel. The example case I have is supporting the waiting primitives on multiple tasks with the FIRST_COMPLETED or FIRST_EXCEPTION option.

[peterhinch]

For task groups see #8791. A very useful enhancement.

[projectgus]

This is an automated heads-up that we've just merged a Pull Request
that removes the STATIC macro from MicroPython's C API.

See #13763

A search suggests this PR might apply the STATIC macro to some C code. If it
does, then next time you rebase the PR (or merge from master) then you should
please replace all the STATIC keywords with static.

Although this is an automated message, feel free to @-reply to me directly if
you have any questions about this.