[API Proposal]: Public API for the Runtime Async

[VSadov]

Background and motivation

The purpose of this proposal is to introduce the changes in the public API as required by the Runtime Async feature.

For the detailed documentation of IL level interface of Runtime Async feature see proposed changes to ECMA-335 document.

For reference, the location of corresponding C# spec - https://github.com/dotnet/roslyn/blob/main/docs/compilers/CSharp/Runtime%20Async%20Design.md

NOTE: referenced documents are work-in-progress, so could be slightly behind on details.

API Proposal

The API adds a new Async member in the MethodImplOptions enum, and a corresponding change to System.Reflection.MethodImplAttributes.

The purpose of this flag is to indicate that a particular method is an Async method. There are certain restrictions on when this flag is applicable. For example the method must be Task[<T>] or ValueTask[<T>]-returning, can't be Synchronized, etc... The method also needs to satisfy certain correctness invariants in it's body. For example, it must return a value with a type compatible to the unwrapped type of the formal return type in the method signature.

The key effect of a method being Async is that such method can use a group of Await functions to delegate the "awaiting" to the runtime.
The Await methods are special methods that provide functionality not otherwise expressible in IL - namely asynchronous waiting for potentially incomplete tasks or awaiters.

These Await helper methods are only callable from Async methods. Calling one of these Await methods from a method not decorated as MethodImplOptions.Async is an invalid IL sequence and will result in an appropriate error at IL-compiling or execution time.
NOTE: It is permitted for Await helpers to call other Await helpers and some of them do that. That is legal because these helpers are themselves Async methods. This part makes no difference to the end user, just something worth mentioning.

namespace System.Runtime.CompilerServices
{
    [Flags]
    public enum MethodImplOptions
    {
        Unmanaged = 0x0004,
        NoInlining = 0x0008,
        ForwardRef = 0x0010,
        Synchronized = 0x0020,
        NoOptimization = 0x0040,
        PreserveSig = 0x0080,
        AggressiveInlining = 0x0100,
        AggressiveOptimization = 0x0200,
        InternalCall = 0x1000,
        Async = 0x2000,
    }
}

namespace System.Reflection
{
    public enum MethodImplAttributes
    {
        IL = 0,
        Managed = 0,
        Native = 1,
        OPTIL = 2,
        CodeTypeMask = 3,
        Runtime = 3,
        ManagedMask = 4,
        Unmanaged = 4,
        NoInlining = 8,
        ForwardRef = 16,
        Synchronized = 32,
        NoOptimization = 64,
        PreserveSig = 128,
        AggressiveInlining = 256,
        AggressiveOptimization = 512,
        InternalCall = 4096,
        Async = 8192,
        MaxMethodImplVal = 65535,
    }
}

namespace System.Runtime.CompilerServices
{
    [System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
    [System.Diagnostics.CodeAnalysis.ExperimentalAttribute("SYSLIB5007", UrlFormat = "https://aka.ms/dotnet-warnings/{0}")]
    public static partial class AsyncHelpers
    {
        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// </summary>
        public static void UnsafeAwaitAwaiter<TAwaiter>(TAwaiter awaiter) where TAwaiter : ICriticalNotifyCompletion { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// </summary>
        public static void AwaitAwaiter<TAwaiter>(TAwaiter awaiter) where TAwaiter : INotifyCompletion { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then execution continues or an exception is thrown according to the kind of completion.
        /// </summary>
        public static void Await(System.Threading.Tasks.Task task) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then returns the result value or throws an exception according to the kind of completion.
        /// </summary>
        public static T Await<T>(System.Threading.Tasks.Task<T> task) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then execution continues or an exception is thrown according to the kind of completion.
        /// </summary>
        public static void Await(System.Threading.Tasks.ValueTask task) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then returns the result value or throws an exception according to the kind of completion.
        /// </summary>
        public static T Await<T>(System.Threading.Tasks.ValueTask<T> task) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then execution continues or an exception is thrown according to the kind of completion.
        /// The execution and synchronization context flow will be arranged according to the configuration of 
        /// the awaitable
        /// </summary>
        public static void Await(System.Runtime.CompilerServices.ConfiguredTaskAwaitable configuredAwaitable) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then returns the result value or throws an exception according to the kind of completion.
        /// The execution and synchronization context flow will be arranged according to the configuration of 
        /// the awaitable
        /// </summary>
        public static T Await<T>(System.Runtime.CompilerServices.ConfiguredTaskAwaitable<T> configuredAwaitable) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then execution continues or an exception is thrown according to the kind of completion.
        /// The execution and synchronization context flow will be arranged according to the configuration of 
        /// the awaitable
        /// </summary>
        public static void Await(System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable configuredAwaitable) { }

        /// <summary>
        /// Suspends evaluation of the enclosing `Async` method until the asynchronous operation represented 
        /// by the operand completes.
        /// Then returns the result value or throws an exception according to the kind of completion.
        /// The execution and synchronization context flow will be arranged according to the configuration of 
        /// the awaitable
        /// </summary>
        public static T Await<T>(System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable<T> configuredAwaitable) { }
    }
}

• We intend to mark this API as Experimental until fully supported and available on all runtime flavors.
• SYSLIB5007 is just the next experimental warning ID available right now.
  The warning will say something like: Runtime Async is experimental

API Usage

The API is not designed or optimized for using directly in the code. The intended purpose is to be used by IL generators such as C# compiler and similar.

From the runtime perspective, there is nothing wrong with invoking the API directly, although compilers may put restrictions on direct use of this API.

Here is an example of a typical use.
When Runtime Async code generation is enabled, C# compiler will emit the following code:

        public async Task<int> M1(int arg)
        {
            return await M2(arg);
        }

        public async Task<int> M2(int arg)
        {
            await Task.Yield();

            ValueTask<int> vt = ValueTask.FromResult(arg);
            return await vt.ConfigureAwait(false);
        }

as an IL equivalent of this:

        [MethodImpl(MethodImplOptions.Async)]
        public Task<int> M1(int arg)
        {
            return AsyncHelpers.Await<int>(M2(arg));
        }

        [MethodImpl(MethodImplOptions.Async)]
        public Task<int> M2(int arg)
        {
            YieldAwaitable.YieldAwaiter awaiter = Task.Yield().GetAwaiter();
            if (!awaiter.IsCompleted)
            {
                AsyncHelpers.UnsafeAwaitAwaiterFromRuntimeAsync<YieldAwaitable.YieldAwaiter>(awaiter);
            }
            awaiter.GetResult();

            ValueTask<int> vt = ValueTask.FromResult(arg);
            return AsyncHelpers.Await<int>(vt.ConfigureAwait(false));
        }

Alternative Designs

This API has evolved from an alternative design where the "await" in Async methods would be encoded by calling Task-returning methods via an alternative signature.
In addition to that, the Async methods were defined by using an alternative signature as well.

Compared to the scheme that encodes await through Await helpers, that scheme had a few inconveniences:

• it is more confusing for IL generator to call methods that "do not exist"
• similar additional complications exist in overriding/hiding/implementing scenarios as IL generator needs to understand both signature flavors when reasoning about bases and interfaces.
• using alternative, and previously illegal, signatures has more potential for breaking existing tooling. Even if tools are unconcerned about method bodies.
• IL generator needs to be aware if the awaited argument is a method call (not a field, local, ...), as only a method could have an alternative signature.
• the scenario with configured awaiters did not have a good solution.

Overall, this API requires slightly more complexity on VM and JIT implementation side, while being much friendlier to IL generators.

Risks

Current implementation is CoreCLR-only and employs JIT compiler to perform transformation of async methods into state machines and to provide thunks for interop with ordinary Task-based asynchronous operations.
There is some risk that implementing semantics of this API could be harder on non-JIT runtimes. However, we do not see fundamental reasons why the same semantic could not be implemented in AOT or in Interpreter mode.

[neon-sunset]

Public API consumable by guest languages would be really nice to have in .NET as a platform.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime-compilerservices
See info in area-owners.md if you want to be subscribed.

[jkotas]

RuntimeHelpers

It is quite a few new methods and I expect that there will be more. Would it make sense to put all runtime async helpers into its own type?

There is some risk that implementing semantics of this API could be harder on non-JIT runtimes.

I assume that the APIs are going to be exposed with ExperimentalAttribute until the implementation is available everywhere. You can add the ExperimentalAttribute on the proposed APIs to make this clear.

Roslyn will use RuntimeAsyncMethodGenerationAttribute to opt-in opt-out methods from using the feature.

I think so.

[VSadov]

It is quite a few new methods and I expect that there will be more.

I think we have an Await for everything that can be awaited now, but can't rule out introducing some special cases.

Would it make sense to put all runtime async helpers into its own type?

Under something like System.Runtime.CompilerServices.AsyncHelpers ?

The only concern, I think, is that there are few things already named AsyncXXX under System.Runtime.CompilerServices, like AsyncTaskMethodBuilder, but they are the machinery for the "classic" kind of Async.
Perhaps that is ok.

[VSadov]

Also we are almost expecting that ...AwaiterFromRuntimeAsync... could have better names, but the proposal has what we could come up with.
The "From" is historical, as initially this was the only way for a runtime async method to await a "classic" awaitable.

We do not want to make them overloads of Await though, since they do not implement complete await semantics. Their main purpose now is for custom awaitables that match the pattern by providing their own IsCompleted/GetResult. In such case C# will have to emit a bit more code to really implement await operator.
(see an example with Task.Yield() )

[jakobbotsch]

[MethodImpl(MethodImplOptions.Async)]

Nit -- this is part of the implementation details of these methods, not part of the signature (i.e. not something that a caller should ever pay attention to). So I am not sure they belong in the API proposal.

[VSadov]

   [MethodImpl(MethodImplOptions.Async)]

Nit -- this is part of the implementation details of these methods, not part of the signature (i.e. not something that a caller should ever pay attention to). So I am not sure they belong in the API proposal.

I left the [MethodImpl(MethodImplOptions.Async)] thinking that they make these methods special, thus adding/removing is compat breaking.

But you are right. These methods are special because of their use and purpose, and methodimpl is just how we mark them. It is an implementation detail that is not a lot different from [Intrinsic]

[reflectronic]

The enum value should also be added to System.Reflection.MethodImplAttributes, right?

[reflectronic]

Also, the value 0x400 conflicts with the SecurityMitigations flag which was added in .NET Framework 4.8 only. Is that an issue?

[jkotas]

But you are right. These methods are special because of their use and purpose, and methodimpl is just how we mark them. It is an implementation detail that is not a lot different from [Intrinsic]

Should the implementation be changed to just use [Intrinsic]? Or does [MethodImpl(MethodImplOptions.Async)] on these helpers have some extra benefits?

[jakobbotsch]

The implementation of these methods need to be emitted as an async state machine, so it's just a convenient way to propagate this information through the VM/JIT. Before we had MethodImpl.Async we just recognized these methods specially (by name).

[VSadov]

Conceptually they are non-Task returning async methods. Thus you can’t make ordinary calls to them from ordinary, non-async, methods as how’d you return an incomplete result.
Otherwise, they are treated as Async everywhere where it matters (i.e. emitted as state machines, called via CORINFO_CALLCONV_ASYNCCALL), so we just use Async.
That also naturally fits the rule "only Async methods can call these helpers" when some of Async helpers call other Async helpers.

We will not allow non-Task returning async methods outside of the runtime.
I can imagine to have more of them internally though, if they can be useful in async/Task infrastructure.

We could just match by name. We did that when we had only two of these helpers. I think MethodImpl scales a bit better if more helpers will be added.

[VSadov]

Also, the value 0x400 conflicts with the SecurityMitigations flag which was added in .NET Framework 4.8 only. Is that an issue?

It does not look like SecurityMitigations was added to CoreCLR.
https://github.com/dotnet/runtime/blob/a87fd8bc2fe1bfa3a96d85b05d20a8341c1eb895/src/libraries/System.Private.CoreLib/src/System/Reflection/MethodImplAttributes.cs

I think we could reuse the bit, but there are a few more bits available. So to be safe, let's use something else.

There is a gap between AggressiveOptimization = 0x0200 and InternalCall = 0x1000. So I guess 0x800 was also used for something.
Perhaps taking 0x2000 is the safest option.

[agocke]

We'll also need a new runtime feature flag

--- a/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/RuntimeFeature.cs
+++ b/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/RuntimeFeature.cs
@@ -47,6 +47,12 @@ public static partial class RuntimeFeature
         /// </summary>
         public const string NumericIntPtr = nameof(NumericIntPtr);

+        /// <summary>
+        /// Indicates that this version of the runtime supports async methods.
+        /// </summary>
+        [RequiresPreviewFeatures]
+        public const string Async = nameof(Async);
+
         /// <summary>
         /// Checks whether a certain feature is supported by the Runtime.
         /// </summary>
@@ -54,7 +60,18 @@ public static bool IsSupported(string feature)
         {
             return feature switch
             {
-                PortablePdb or CovariantReturnsOfClasses or ByRefFields or ByRefLikeGenerics or UnmanagedSignatureCallingConvention or DefaultImplementationsOfInterfaces or VirtualStaticsInInterfaces or NumericIntPtr => true,
+                PortablePdb or
+                CovariantReturnsOfClasses or
+                ByRefFields or
+                ByRefLikeGenerics or
+                UnmanagedSignatureCallingConvention or
+                DefaultImplementationsOfInterfaces or
+                VirtualStaticsInInterfaces or
+                NumericIntPtr or
+#pragma warning disable CA2252 // Using preview features
+                Async=> true,
+#pragma warning restore CA2252 // Using preview features
+
                 nameof(IsDynamicCodeSupported) => IsDynamicCodeSupported,
                 nameof(IsDynamicCodeCompiled) => IsDynamicCodeCompiled,
                 _ => false,

[jkotas]

We'll also need a new runtime feature flag

The support is gated by availability of the new APIs. It makes the new runtime feature flag unnecessary. We typically add a feature flag only when there is no obvious API associated with the feature.

Moving the runtime async into a dedicated type will make the gating based on API availability cleaner.

[VSadov]

Re: RuntimeAsyncMethodGenerationAttribute - after discussing with Roslyn team the idea is that the attribute will be very rarely needed, so it is preferable to not ship it as a public API. If someone needs such attribute it would need to be defined in the app/library that needs it.

That is at least the plan for previews. We may know better how often the attribute is really used later.

[VSadov]

I think I have added all that I had for this API and what was suggested in the discussion.

[jkotas]

ExperimentalAttribute allows targeting types. You can mark the whole type as experimental, no need to mark each method individually.

There are definitions in Reflection that represent MethodImplOptions. They will need to be updated accordingly.

I assume that you mean enum MethodImplAttributes. You can include it in the API proposal for clarity. Or is there more to it?

[reflectronic]

The APIs are not called directly by user code. Does ExperimentalAttribute do anything in this case? Should the RequiresPreviewFeatures analyzer be updated to support runtime async instead?

[jkotas]

Does ExperimentalAttribute do anything in this case?

Nothing prevents these APIs from being called by user code. ExperimentalAttribute is insurance in case somebody ends up calling these APIs in their code for some reason.

Runtime async is a transparent implementation detail for user code. RequiresPreviewFeatures is meant to be used on new APIs that can be called by user code only when the compiler preview features are turned on. Runtime async has no APIs like so far.

[reflectronic]

OK, I understand--since runtime async methods return regular Task etc., callers don't need to treat them specially. So, there's no "viral" element, and RequiresPreviewFeatures does not make sense. Sorry for the noise

[VSadov]

ExperimentalAttribute allows targeting types. You can mark the whole type as experimental, no need to mark each method individually.

I did not realize that. It would be useful here.

I assume that you mean enum MethodImplAttributes. You can include it in the API proposal for clarity. Or is there more to it?

Yes. MethodImplAttributes

I did a search and saw a few hits - MethodImplOptions, MethodImplAttributes (CoreCLR and NativeAOT flavors).
There is also EcmaMethod.MethodFlags, which is not public. So I generalized - whatever represents MethodImpl may need an update.

But really the only other public API to update is just MethodImplAttributes.
I'll mention it more directly in the proposal.

[jkotas]

there is also EcmaMethod.MethodFlags, which is not public

Right, this is internal implementation of AOT compilers. We will need it once we get to implementing runtime async for R2R and NAOT.

[VSadov]

Mentioned System.Reflection.MethodImplAttributes change explicitly

[agocke]

We'll also need a new runtime feature flag

The support is gated by availability of the new APIs. It makes the new runtime feature flag unnecessary. We typically add a feature flag only when there is no obvious API associated with the feature.

Moving the runtime async into a dedicated type will make the gating based on API availability cleaner.

From the Roslyn perspective, I think every change that's corresponds to a differing runtime capability is supposed to be in RuntimeFeature. Even if there are other API changes that would be sufficient, I think the RuntimeFeature API is supposed to be the single check point. It's always present in corelib, so it can't be faked, and the type itself is already considered a SpecialType by Roslyn, so it has special behavior.

It looks like the current Roslyn compiler also has this requirement. So if we want to redesign this, we will need to push the requirement back up to Roslyn.

[333fred]

I don't think it would be particularly difficult to simply gate on the presence of AwaitHelpers in corlib.

[jkotas]

I think every change that's corresponds to a differing runtime capability is supposed to be in RuntimeFeature.

It is not how RuntimeFeature has evolved so far. When the runtime feature has one canonical API associated with it, we use the API as the gate. One recent example from many: InlineArrayAttribute. It is runtime capability, Roslyn will light up when it sees InlineArrayAttribute in CoreLib, but it does not have RuntimeFeature.

If we want to change the rules for what's gets included in RuntimeFeature, it should be a separate discussion.

[agocke]

I don't think it would be particularly difficult to simply gate on the presence of AwaitHelpers in corlib.

If it's not a problem for Roslyn, not a problem for me either. Only question is whether we gate on one API or many.

[agocke]

@bartonjs I think this is ready for review. Anything more we need to do?

[bartonjs]

Anything more we need to do?

With no assigned milestone it was at the bottom of the review queue. Assigning the milestone bumped it, and with blocking it's at the top... but since those were done after the meeting started, I didn't see them. (Really, since they were done after 4pm Monday it didn't make it onto the agenda, but blocking gets extra eyes before the meeting starts).

Since it's now marked as blocking, this'll be first up in the next meeting. (Unless it somehow gets trumped by something even-more-blocking)

[bartonjs]

Anything more we need to do?

Whenever something gets marked as ready for review, its place in the queue can be seen at https://apireview.net/. OrderBy(Blocking).ThenBy(Milestone).ThenBy(HowLongInTheQueue)

[bartonjs]

Video

• Nothing in this new type should ever be called by anyone other than the compiler (and doing so may cause errors like InvalidProgramException), so the whole type should be EB-Never
• The "Helpers" suffix isn't great, but CompilerServices already has some. And we already marked it as EB-Never
• AwaitAwaiterFromRuntimeAsync => AwaitAwaiter

namespace System.Runtime.CompilerServices
{
    [Flags]
    public partial enum MethodImplOptions
    {
        Async = 0x2000,
    }
}

namespace System.Reflection
{
    public partial enum MethodImplAttributes
    {
        Async = 0x2000,
    }
}

namespace System.Runtime.CompilerServices
{
    [System.ComponentModel.EditorBrowsable(System.ComponentModel.EditorBrowsableState.Never)]
    [System.Diagnostics.CodeAnalysis.ExperimentalAttribute("SYSLIB5007", UrlFormat = "https://aka.ms/dotnet-warnings/{0}")]
    public static partial class AsyncHelpers
    {
        public static void UnsafeAwaitAwaiter<TAwaiter>(TAwaiter awaiter) where TAwaiter : ICriticalNotifyCompletion { }
        public static void AwaitAwaiter<TAwaiter>(TAwaiter awaiter) where TAwaiter : INotifyCompletion { }

        public static void Await(System.Threading.Tasks.Task task) { }
        public static T Await<T>(System.Threading.Tasks.Task<T> task) { }
        public static void Await(System.Threading.Tasks.ValueTask task) { }
        public static T Await<T>(System.Threading.Tasks.ValueTask<T> task) { }
        public static void Await(System.Runtime.CompilerServices.ConfiguredTaskAwaitable configuredAwaitable) { }
        public static T Await<T>(System.Runtime.CompilerServices.ConfiguredTaskAwaitable<T> configuredAwaitable) { }
        public static void Await(System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable configuredAwaitable) { }
        public static T Await<T>(System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable<T> configuredAwaitable) { }
    }
}