[RuntimeAsync] Merge from labs, all except JIT #114675

VSadov · 2025-04-15T07:53:36Z

This PR includes everything from the runtimelab implementation of Runtime Async feature, except: JIT changes and tests.
Some changes in between VM and JIT are hard to separate, so included here as well.

The main effect of the VM changes is classification of methods by MethodReturnKind. We recognize methods that return Task/ValueTask/Task<T>/ValueTask<T> as special.

A task-returning method definition will now get two methoddescs in the type system:

one has the original Task-returning signature
another is T-returning, but called with a calling convention that uses a continuation object to handle cases of incomplete result leading to suspending/resuming the chain of calls.

We call the pair "variants". For most purposes the methods are identical and parallel - i.e. if generic, both have exactly same set of type parameters, if task-returning variant overrides another one in the base class, the async variant overrides the async variant in the base, etc...
Both variants can signal that the result is not ready yet - via incomplete Task or by returning a Continuation object.

The purpose of this duality is that async variants can be much more efficient when suspension does not happen, leading to substantial savings when a chain of awaits may be turned into a chain of calls in "async space". Here the JIT part that optimizes Await pattern into async calls.

From the public/external point of view the async variants are an implementation detail that "does not exist", and can change in future releases. Async variants should not show up in reflection, for example.

In terms of implementation logic one variant contains the user code/IL and another is a thunk that forwards to it. Which one contains the user code depends on presence of MethodImpl.Async. If there is no MethodImpl.Async the user code belongs to the task-returning variant, otherwise to the async variant.

Note that for the most part Runtime Async feature is pay-for-play. JIT features will only engage if the user code is compiled with a compiler that is enlightened to use the new API for async/await.
However, the creation of variant pairs for task-returning methods must happen unconditionally prior to possible use. Since these are just entry points in the type system, the impact should not be high. We will, of course measure and see if some action is needed on that.

In this PR the generation of variants will require FEATURE_RUNTIME_ASYNC, which will not be defined by default, so by default none of the above will happen.

ghost · 2025-04-15T07:53:43Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

ghost · 2025-04-15T07:53:45Z

Note regarding the new-api-needs-documentation label:

This serves as a reminder for when your PR is modifying a ref *.cs file and adding/modifying public APIs, please make sure the API implementation in the src *.cs file is documented with triple slash comments, so the PR reviewers can sign off that change.

dotnet-policy-service · 2025-04-15T07:54:36Z

Tagging subscribers to this area: @mangod9
See info in area-owners.md if you want to be subscribed.

VSadov · 2025-04-15T08:04:20Z

CC @jakobbotsch

Copilot

Pull Request Overview

This PR introduces support for runtime async variants by adding new callbacks, updated async structures, and modifications to the runtime’s internal async continuation and await handling logic. Key changes include:

New async callbacks in JIT interface headers and corresponding method implementations.
Addition of the CORINFO_ASYNC_INFO structure, async continuation logic, and related flag updates in the runtime and EE headers.
Updates in CoreLib to support async continuations and improved error/patchpoint handling.

Reviewed Changes

Copilot reviewed 99 out of 100 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/coreclr/tools/aot/jitinterface/jitinterface_generated.h	Added new async callbacks for obtaining async info and resumption stub.
src/coreclr/tools/Common/JitInterface/ThunkGenerator/ThunkInput.txt	Updated thunk definitions to include async info.
src/coreclr/tools/Common/JitInterface/CorInfoTypes.cs	Introduced the CORINFO_ASYNC_INFO struct.
src/coreclr/tools/Common/JitInterface/CorInfoImpl*.{cs,generated.cs}	Added async methods with placeholder implementations for getAsyncInfo and getAsyncResumptionStub.
src/coreclr/inc/*	Extended various headers with new async tokens, flags, and helper functions.
src/coreclr/System.Private.CoreLib/*	Integrated async continuation allocation and dispatch logic within CoreLib.

Files not reviewed (1)

src/coreclr/clrdefinitions.cmake: Language not supported

Comments suppressed due to low confidence (1)

src/coreclr/System.Private.CoreLib/src/System/Runtime/CompilerServices/RuntimeHelpers.CoreCLR.cs:829

[nitpick] The continuation dispatch logic in DispatchContinuations is complex and central to async resumption. Ensure that this new logic is thoroughly exercised by unit/integration tests to verify proper handling of both normal and exceptional control flows.

private static unsafe Continuation? DispatchContinuations(Continuation? continuation)

src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs

VSadov · 2025-04-16T15:17:04Z

I recommend to review this with dot . Most interesting changes are under VM

VSadov · 2025-04-16T15:20:20Z

src/coreclr/vm/method.hpp

 #define METHOD_TOKEN_REMAINDER_MASK ((1 << METHOD_TOKEN_REMAINDER_BIT_COUNT) - 1)
 #define METHOD_TOKEN_RANGE_BIT_COUNT (24 - METHOD_TOKEN_REMAINDER_BIT_COUNT)
 #define METHOD_TOKEN_RANGE_MASK ((1 << METHOD_TOKEN_RANGE_BIT_COUNT) - 1)

+enum class AsyncMethodKind
+{


Comments in AsyncMethodKind may be a good start with the async method classification

VSadov · 2025-04-16T15:25:35Z

src/coreclr/vm/clrtocomcall.cpp

@@ -230,6 +230,11 @@ I4ARRAYREF SetUpWrapperInfo(MethodDesc *pMD)

        GCX_PREEMP();

+        if (pMD->IsAsyncMethod())


There are quite a few changes like this where we do not expect async methods to participate in some scenarios (like COM interop here).
Sometimes it is less clear if scenario may actually need to support async methods somehow.

We prefer throw rather than asserts in this and similar cases - so it is more apparent if some scenario makes it here.

Yes, many of these can be hit. For example, this throws NotSupportedException in the runtimelab async branch and prints 7 in main:

using System; using System.Runtime.InteropServices; using System.Threading.Tasks; Console.WriteLine(Marshal.GetEndComSlot(typeof(IFoo))); public interface IFoo { Task M(); }

Before we turn on runtime async support in main, we need to go over all places where you are adding these NotSupportedException, validate whether they can be hit, and fix things up as appropriate to avoid breaking changes. We know that the test coverage for built-in COM interop is very spotty, but customers depend on every corner case behavior of built-in COM interop. cc @AaronRobinsonMSFT

Could you please tag all these places that need review with async TODO so that they can be found easily and none of them falls through the cracks?

VSadov · 2025-04-16T15:28:49Z

src/coreclr/vm/clsload.cpp

+            // For {Task-returning, Async} variants of the same definition
+            // we associate the methoddef with the Task-returning variant since it
+            // matches the methadata signature.
+            if (!pMD->IsUnboxingStub() && !pMD->IsAsyncVariantMethod())
            {


This could be interesting. Both variants have the same memberDef, so it is 2->1 relationship.
However when going from memberDef to methoddesc is 1->1 map, and starting with memberDef will produce the non-async methoddesc - the one that matches the metadata signature. (but it may not be the one that owns the IL!)

VSadov · 2025-04-16T15:32:40Z

src/coreclr/vm/amd64/AsmHelpers.asm

@@ -203,10 +203,11 @@ NESTED_ENTRY OnHijackTripThread, _TEXT
        push                rax ; make room for the real return address (Rip)
        push                rdx
        PUSH_CALLEE_SAVED_REGISTERS
+        push_vol_reg        rcx


There are some asm changes in Hijacking. The purpose will become more clear when JIT changes are brought in. In short - we are adding a possible extra return (the continuation, which is a managed object), thus we need to add an extra calee-trash register to the captured contexts.

On amd64, if a call ends up being async call, the continuation return register is RCX.

VSadov · 2025-04-16T15:38:02Z

src/coreclr/vm/comutilnative.cpp

@@ -1851,6 +1851,14 @@ FCIMPL1(MethodTable*, MethodTableNative::InstantiationArg0, MethodTable* mt);
 }
 FCIMPLEND

+FCIMPL1(OBJECTHANDLE, MethodTableNative::GetLoaderAllocatorHandle, MethodTable *mt)


When suspending, the continuation holds the state of the call. The state may need to include loader allocator reference as the method frame could be the only thing holding it alive. This helper is used in related logic.

VSadov · 2025-04-16T15:41:10Z

src/coreclr/vm/frames.cpp

@@ -1357,7 +1357,8 @@ void HijackFrame::GcScanRoots_Impl(promote_func *fn, ScanContext* sc)
 {
    LIMITED_METHOD_CONTRACT;

-    ReturnKind returnKind = m_Thread->GetHijackReturnKind();
+    bool hasAsyncRet;
+    ReturnKind returnKind = m_Thread->GetHijackReturnKind(&hasAsyncRet);


Reporting the continuation to GC relies on liveness info at call site, just like for other returns.

As usual x86 is special. As its GC info does not cover volatile registers, we need some extra code to figure if continuation needs to be reported. (this is somewhat similar to ReturnKind, which also has special case on x86)

The async ABI should be documented in https://github.com/dotnet/runtime/blob/main/docs/design/coreclr/botr/clr-abi.md

Will do. It is not very complicated though.

There is an extra optional continuation return. If it is not NULL, means the result is not ready and the continuation has captured the state.
The format of continuation is opaque, but changing the format will likely be R2R breaking.

There is also an extra continuation parameter (position depends on platform, x86 is different, as usual).
If the continuation argument is not NULL, means this is not the first time the method runs, and it needs to unpack the continuation and continue running.
The method may return a continuation again, if there is another await that is not ready with results. That is why we do not resume the entire continuation stack - there could be an await in a loop.

This is all there is from the ABI point, I think.

added a section on Async calling convention in the ABI document

VSadov · 2025-04-16T15:44:36Z

src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs

@@ -2448,6 +2448,7 @@ public TaskAwaiter GetAwaiter()
        /// true to attempt to marshal the continuation back to the original context captured; otherwise, false.
        /// </param>
        /// <returns>An object used to await this task.</returns>
+        [Intrinsic]
        public ConfiguredTaskAwaitable ConfigureAwait(bool continueOnCapturedContext)


ConfigureAwait can participate in JIT optimizations (JIT changes are not in this PR), thus needs to be matched by name.

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

VSadov · 2025-04-27T17:13:37Z

Could you please tag all these places that need review with async TODO so that they can be found easily and none of them falls through the cracks?

Added // TODO: (async) revisit and examine if this needs to be supported and a tracking issue to revisit/remove the TODOs: #115094

VSadov · 2025-04-27T18:09:23Z

In RuntimeMethodHandle::GetMethodDef there is a call to GetCallingConventionInfo. we should check what impact that has.

I think, if starting with methoddef, we will always get the Task-returning variant with a regular call convention.

There is a bit of a corner case for the non-Task returning methods that are explicitly MethodImpl.Async - like AsyncHelpers.Await(). Need to think how that should be represented. Perhaps just a regular calling convention. These methods should not be callable via reflection anyways (need to make sure, test).

I've logged a general issue to follow up with Reflection scenarios: #115099

VSadov · 2025-04-27T18:26:25Z

src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncHelpers.cs

+        // "BypassReadyToRun" is until AOT/R2R typesystem has support for MethodImpl.Async
+        // Must be NoInlining because we use AsyncSuspend to manufacture an explicit suspension point.
+        // It will not capture/restore any local state that is live across it.
+        [BypassReadyToRun]


Removing [BypassReadyToRun] on non-Task Async helpers breaks some scenarios. I had to put [BypassReadyToRun] back until R2R typesystem at least understands MethodImpl.Async and its impact on call convention.

The followup issue for R2R is here: #115098

VSadov · 2025-04-27T23:54:39Z

I think I have addressed all suggestions/concerns, but there were many, so I may have missed some.
@jkotas @davidwrighton - please take a look again if there is something more needed. Thanks!

davidwrighton

Looks good to me, but please get an approval from Jan too.

jkotas

LGTM as well!

VSadov · 2025-04-28T18:16:27Z

Thanks!!!

Add the JIT parts of runtime async. This introduces a new transformation that runs before lowering and that transforms the function into a state machine, with states set up to allow suspension and resumption at async calls (the suspension points). Suspension is indicated by the callee returning a continuation object using a new calling convention. When suspension happens, the caller similarly suspends by capturing all live state and saving it in a continuation object. These continuation objects are then linked together and continue to be propagated back to the callers until we finally get to a caller that is not async anymore (i.e. that expects to see a `Task`/`Task<T>`). The VM synthesizes a `Task`/`Task<T>` wrapper for the async functions that hide away the management the continuation objects. See #114675 for more details around this and around the calling convention. The continuation objects can later be used to resume the function at the point where suspension happened. This is accomplished by async functions also taking a continuation object as a parameter. When such a parameter is passed (i.e. it is non-null), the JIT will restore all live state from the continuation and resume from the correct location. The continuations store a state number so that resumption can know where to resume. The full resumption process also involves an IL stub called the async resumption stub. This stub is responsible for calling the async function with the right stack frame setup for arguments and simultaneously passing a non-null continuation. The stack frame setup done by the IL async resumption stub is important as the JIT uses this space to restore live parameters from the continuation. Continuations similarly support propagation of the return values from the callee and of potential exceptions thrown by the callee. Return values are stored in a known location in the continuation object, and the async resumption stubs are responsible for propagating these values into the next continuation when suspension/resumption has happened. The JIT's resumption code will fetch the return value from the known location and copy it to the right place in the caller. Similarly, exceptions are kept in a known place and are handled by being rethrown from the right location when present. OSR functions come with complications. Since they rely on frame setup done by the corresponding tier-0 method the resumption of these methods needs to happen through the tier 0 method. When OSR methods suspend they store their IL offset in the continuation object, while tier 0 methods with patchpoints will store -1 in the continuation object. The VM then always resumes these methods in the tier 0 method, which knows to use the IL offset to determine whether resumption should happen in the tier 0 method or whether control needs to continue into an OSR method.

max-charlamb · 2025-05-02T18:47:30Z

src/coreclr/vm/arm64/stubs.cpp


     pRD->volatileCurrContextPointers.X0 = &m_Args->X0;
     pRD->volatileCurrContextPointers.X1 = &m_Args->X1;
+     pRD->volatileCurrContextPointers.X1 = &m_Args->X2;


Is this intended to overwrite the value written above? It looks like this should be:

Suggested change

pRD->volatileCurrContextPointers.X1 = &m_Args->X2;

pRD->volatileCurrContextPointers.X2 = &m_Args->X2;

Right. Looks like a typo. Thanks!

VSadov added area-VM-coreclr runtime-async labels Apr 15, 2025

ghost added the new-api-needs-documentation label Apr 15, 2025

dotnet-policy-service bot assigned VSadov Apr 15, 2025

build-analysis bot mentioned this pull request Apr 15, 2025

SIGKILL (OOM?) while running LibraryImportGenerator.Tests w/o actionable log messages or artifacts #114241

Closed

VSadov added runtime-coreclr and removed new-api-needs-documentation labels Apr 15, 2025

VSadov force-pushed the asyncMove2 branch from 83b25bc to 2784521 Compare April 16, 2025 15:02

VSadov marked this pull request as ready for review April 16, 2025 15:15

Copilot AI review requested due to automatic review settings April 16, 2025 15:15

VSadov requested a review from MichalStrehovsky as a code owner April 16, 2025 15:15

Copilot AI reviewed Apr 16, 2025

View reviewed changes

src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs Show resolved Hide resolved

src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs Show resolved Hide resolved

VSadov commented Apr 16, 2025

View reviewed changes

VSadov and others added 9 commits April 26, 2025 17:39

Use approved value for MethodImplAttributes.Async

31744c8

added some TODOs

42e42e1

Update src/coreclr/vm/prestub.cpp

89a68b1

Co-authored-by: Jan Kotas <jkotas@microsoft.com>

some simple PR feedback

2489290

Fail if EnC targets an Async method

74f4da3

moved IsEnCAddedMethod bit to Flags4

61c63d3

Implement IL_ThrowExact for x86/funclets

55caeca

Addres PR feedback

ad5cb52

couple TODOs for EnC NYI

6f2c815

build-analysis bot mentioned this pull request Apr 27, 2025

System.Net.Quic.Tests.MsQuicTests.WriteTests failed with "System.Net.Quic.QuicException : The connection timed out from inactivity." #105177

Open

VSadov force-pushed the asyncMove2 branch from ea0b132 to 6f2c815 Compare April 27, 2025 01:06

add a simple ldtoken test scenario for Task-returning method

7dbc621

VSadov mentioned this pull request Apr 27, 2025

[RuntimeAsync] Implement ilasm/ildasm support for the MethodImpl.Async #115093

Closed

This was referenced Apr 27, 2025

[RuntimeAsync] PGO support. #115096

Open

[RuntimeAsync] Multicore JIT support. #115097

Open

[RuntimeAsync] Reflection/introspection support #115099

Open

Put back BypassReadyToRun untill we are sure about R2R support

49f651f

VSadov commented Apr 27, 2025

View reviewed changes

davidwrighton approved these changes Apr 28, 2025

View reviewed changes

jkotas approved these changes Apr 28, 2025

View reviewed changes

jkotas merged commit dea4f97 into dotnet:main Apr 28, 2025
148 checks passed

VSadov deleted the asyncMove2 branch April 28, 2025 18:16

VSadov mentioned this pull request Apr 28, 2025

[RuntimeAsync] Move runtime async to the main repo #113976

Closed

33 tasks

max-charlamb reviewed May 2, 2025

View reviewed changes

VSadov mentioned this pull request May 13, 2025

[RuntimeAsync] Make runtime async tests to build and pass locally. #115499

Merged

		@@ -230,6 +230,11 @@ I4ARRAYREF SetUpWrapperInfo(MethodDesc *pMD)

		GCX_PREEMP();

		if (pMD->IsAsyncMethod())

	pRD->volatileCurrContextPointers.X1 = &m_Args->X2;
	pRD->volatileCurrContextPointers.X2 = &m_Args->X2;

[RuntimeAsync] Merge from labs, all except JIT #114675

[RuntimeAsync] Merge from labs, all except JIT #114675

Uh oh!

Conversation

VSadov commented Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Apr 15, 2025

Uh oh!

ghost commented Apr 15, 2025

Uh oh!

dotnet-policy-service bot commented Apr 15, 2025

Uh oh!

VSadov commented Apr 15, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

VSadov commented Apr 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VSadov Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VSadov Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VSadov Apr 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VSadov Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VSadov commented Apr 27, 2025

Uh oh!

VSadov commented Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

VSadov Apr 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

VSadov commented Apr 27, 2025

Uh oh!

davidwrighton left a comment

Choose a reason for hiding this comment

Uh oh!

jkotas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

VSadov commented Apr 28, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

VSadov commented Apr 15, 2025 •

edited

Loading

VSadov Apr 16, 2025 •

edited

Loading

VSadov Apr 16, 2025 •

edited

Loading

VSadov Apr 17, 2025 •

edited

Loading

VSadov Apr 16, 2025 •

edited

Loading

VSadov commented Apr 27, 2025 •

edited

Loading

VSadov Apr 27, 2025 •

edited

Loading