Calling .NET async/await functions

[barrybriggs]

Environment

• Pythonnet version: 2.3.0 ("featured")
• Python version: 3.6
• Operating System: Windows

Details

Say I load a .NET DLL like so:

clr.AddReference("MyDLL")

and say in MyDLL I have methods of the form:

      async public Task<string> Foo(string x) {
            string y=await Bar(x); 
            return y;
      }

Should it be possible to invoke this from Python? Is there a different syntax for this?

[dmitriyse]

Currently C# async to python async bridge is not implemented. And it's very hard task, and there are too many abstraction leaks in both languages.
So you need to use C# TPL System.Threading.Tasks.Task sync API from the python over C# async methods.

[barrybriggs]

(I agree with your assessment of abstraction leaks.) So in Python you would simply call the function like so?

f=MyDLL.Foo("a").Wait()

[dmitriyse]

f=MyDll.Foo("a").Result;

[barrybriggs]

I knew that :-)

(There's actually a problem with .Result in that in can cause deadlocks, which it does in my app -- I'm using Python as an embedded scripting language within a highly distributed app whose public API is almost all async await'd. This is why I was hoping that the async metaphor had been fully implemented.)

[lucasgl]

Does GetAwaiter().GetResult() works better?

[lostmsu]

If you need asynchronous behavior, you can use Task.ContinueWith method.

[phxnsharp]

I'm not going to speak to whether this is a good idea or not, but a potential solution is to wrap all calls using an AnyIO BlockingPortal.

The above authors are correct that the abstractions are leaky and deadlock is a potential (you probably want to be careful to call ConfigureAwait(false) liberally in your .NET libraries. If you don't know why, you probably shouldn't be using this).

Performance may be terrible, I didn't test it.

Tested on Python 3.8 with pythonnet 3.0.0a2 on Linux using DotNet Core 6.

import anyio
from clr_loader import get_coreclr
from pythonnet import set_runtime

# I'm testing on .NET Core with 3.0.0a2
rt = get_coreclr("conf.json")
set_runtime(rt)

import clr
import System
from System.Threading.Tasks import Task
from System.IO import File

# Create an alias for Action with one generic attr
Action1 = getattr(System, "Action`1")


async def call_async(func, *args, **kwargs):
    async with anyio.from_thread.BlockingPortal() as portal:
        result = func(*args, **kwargs)
        result.ContinueWith(Action1[Task](lambda t: portal.start_task_soon(portal.stop)))
        await portal.sleep_until_stopped()
        if hasattr(result, "Result"):
            return result.Result
        else:
            result.Wait()


async def main():
    await call_async(Task.Delay, 1000)
    print("I waited")
    contents = await call_async(File.ReadAllLinesAsync, __file__)
    print(f"My First Line is {contents[0]}")
    # I return errors too
    await call_async(File.ReadAllLinesAsync, "BOGUS")


if __name__ == '__main__':
    # call_async should work from trio or asyncio
    import trio
    trio.run(main)

[rohanjain101]

Another option which avoids Task.Result or Task.Wait to block current thread (since by the time it used, task is completed):

    import asyncio
    from System import Action
    def create_future(clr_task):
        loop = asyncio.get_running_loop()
        future = asyncio.Future()
        callback = Action(
            lambda: on_completion(
                future, loop, clr_task
            )
        )

        clr_task.GetAwaiter().OnCompleted(callback)
        return future

    def on_completion(future, loop, task):
        if task.IsFaulted:
            clr_error = task.Exception.GetBaseException()
            future.set_exception(clr_error)
        else:
          loop.call_soon_threadsafe(
              lambda: future.set_result(task.GetAwaiter().GetResult())
          )

For awaiting async C# method in python, you could do something like:

await create_future(MyClrAsyncMethod())

[lostmsu]

Some reading in case somebody wants to take on implementing Python's await for .NET tasks:

• https://tenthousandmeters.com/blog/python-behind-the-scenes-12-how-asyncawait-works-in-python/
• Task and ValueTask have GetAwaiter methods, that return an object, that has to implement a few members. See https://learn.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.valuetaskawaiter?view=net-8.0 for an example.

[phxnsharp]

Another option which avoids Task.Result or Task.Wait to block current thread (since by the time it used, task is completed):

Note that in my sample code I only ever call Task.Result or Task.Wait after I know the task is completed in order to properly bring exceptions back to Python. It will never block on those calls.