[NativeAOT] Add cDAC data descriptor infrastructure#126972
[NativeAOT] Add cDAC data descriptor infrastructure#126972max-charlamb wants to merge 8 commits intomainfrom
Conversation
|
Tagging subscribers to this area: @agocke, @dotnet/ilc-contrib |
There was a problem hiding this comment.
Pull request overview
Adds cDAC contract descriptor generation to the NativeAOT runtime, plus an ILC-emitted managed sub-descriptor so diagnostic tools can inspect NativeAOT runtime/managed state via the shared contract mechanism.
Changes:
- Integrates NativeAOT cDAC contract descriptor (and GC sub-descriptors) into the NativeAOT CMake build and runtime libraries.
- Introduces a managed type layout sub-descriptor emitted by ILC (
DotNetManagedContractDescriptor) and wires it into the NativeAOT descriptor as a sub-descriptor. - Exposes select private NativeAOT runtime offsets/constants to the descriptor via the
cdac_data<T>friend pattern and exports the main contract descriptor symbol for diagnostics.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/coreclr/tools/aot/ILCompiler/Program.cs | Adds the managed descriptor root provider to ILC compilation roots. |
| src/coreclr/tools/aot/ILCompiler.Compiler/ILCompiler.Compiler.csproj | Includes new managed descriptor provider/node sources in the build. |
| src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/ManagedDataDescriptorProvider.cs | Registers managed types to be described and roots the descriptor + JSON blob. |
| src/coreclr/tools/aot/ILCompiler.Compiler/Compiler/DependencyAnalysis/ManagedDataDescriptorNode.cs | Emits a ContractDescriptor-shaped symbol containing JSON type layout data. |
| src/coreclr/nativeaot/Runtime/threadstore.h | Exposes ThreadStore private offsets for descriptor generation via cdac_data<>. |
| src/coreclr/nativeaot/Runtime/inc/MethodTable.h | Exposes MethodTable offsets and flag constants for descriptor consumption via cdac_data<>. |
| src/coreclr/nativeaot/Runtime/datadescriptor/datadescriptor.inc | Defines the NativeAOT data descriptor types/globals/contracts and sub-descriptors. |
| src/coreclr/nativeaot/Runtime/datadescriptor/datadescriptor.h | Provides includes and declares the managed sub-descriptor symbol address for inclusion. |
| src/coreclr/nativeaot/Runtime/datadescriptor/CMakeLists.txt | Adds descriptor generation targets for NativeAOT runtime + GC (wks/svr). |
| src/coreclr/nativeaot/Runtime/RuntimeInstance.h | Exposes RuntimeInstance private offsets via cdac_data<>. |
| src/coreclr/nativeaot/Runtime/Full/CMakeLists.txt | Links the generated descriptor libraries into WorkstationGC/ServerGC runtime libs. |
| src/coreclr/nativeaot/Runtime/CMakeLists.txt | Adds the datadescriptor subdirectory to the NativeAOT runtime build (non-WASM). |
| src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.targets | Exports DotNetRuntimeContractDescriptor symbol for diagnostics on all OSes. |
Comments suppressed due to low confidence (1)
src/coreclr/nativeaot/Runtime/datadescriptor/CMakeLists.txt:73
target_compile_definitionsentries should be raw preprocessor symbols (e.g.,SERVER_GC), not compiler flags. Passing-DSERVER_GChere will typically result in an invalid definition being forwarded to the compiler. UseSERVER_GC(orSERVER_GC=1) instead.
| } | ||
| } | ||
|
|
||
| private static void TryRegisterConditionalWeakTableTypes(ManagedDataDescriptorNode descriptorNode, EcmaModule systemModule) |
There was a problem hiding this comment.
We avoid hardcoding BCL implementation details like this in the compiler. Can the list of fields to include in the data contract .json be specified via an input file?
There was a problem hiding this comment.
Either input file or custom attributes on the types/fields in question (can be a corelib private attribute that we match by name).
Do we need to generate these even if the type was never allocated, or only for types that could really exist on the GC heap?
There was a problem hiding this comment.
We avoid hardcoding BCL implementation details like this in the compiler. Can the list of fields to include in the data contract .json be specified via an input file?
Makes sense, I'll switch to using an attribute.
Do we need to generate these even if the type was never allocated, or only for types that could really exist on the GC heap?
In theory we would not. I do think that all of the types we would access are part of the nativeaot internal system and would likely exist. If they don't exist, we wouldn't need this data.
There was a problem hiding this comment.
would likely exist
It depends on how far we go with marking things with this attribute. For example, some debugging scenarios need offset for async infrastructure. The async infrastructure won't be part of the app if the app does not use async.
|
|
||
| CDAC_TYPE_BEGIN(RuntimeInstance) | ||
| CDAC_TYPE_INDETERMINATE(RuntimeInstance) | ||
| CDAC_TYPE_FIELD(RuntimeInstance, T_POINTER, ThreadStore, cdac_data<RuntimeInstance>::ThreadStore) |
There was a problem hiding this comment.
We can move this to ThreadStore, so it is looks the same as regular CoreCLR
| } | ||
| } | ||
|
|
||
| private static void TryRegisterConditionalWeakTableTypes(ManagedDataDescriptorNode descriptorNode, EcmaModule systemModule) |
There was a problem hiding this comment.
Either input file or custom attributes on the types/fields in question (can be a corelib private attribute that we match by name).
Do we need to generate these even if the type was never allocated, or only for types that could really exist on the GC heap?
| <IlcArg Condition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'" Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader,DATA" /> | ||
| <IlcArg Condition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'" Include="--export-dynamic-symbol:DotNetRuntimeDebugHeader" /> | ||
| <IlcArg Condition="'$(_targetOS)' == 'win' and '$(DebuggerSupport)' != 'false'" Include="--export-dynamic-symbol:DotNetRuntimeContractDescriptor,DATA" /> | ||
| <IlcArg Condition="'$(_targetOS)' != 'win' and '$(DebuggerSupport)' != 'false'" Include="--export-dynamic-symbol:DotNetRuntimeContractDescriptor" /> |
There was a problem hiding this comment.
What's the relationship between DotNetRuntimeDebugHeader and DotNetRuntimeContractDescriptor?
There was a problem hiding this comment.
DotNetRuntimeDebugHeader export exposes a set of offsets that the current NativeAOT SOS uses to implement some commands ad-hoc.
The DotNetRuntimeContractDescriptor is the cDAC contract blob, similar to CoreCLR builds. Once the cDAC NAOT infrastructure is in place and working, the legacy DotNetRuntimeDebugHeader export and blob will be removed.
| CDAC_GLOBAL(ObjectToMethodTableUnmask, uint8, 3) | ||
| #endif | ||
|
|
||
| // Contracts: declare which contracts this runtime supports |
There was a problem hiding this comment.
We can add stresslog here as well (without the module table). It can use the same contract version as CoreCLR.
Add the native cDAC data descriptor for NativeAOT, enabling diagnostic tools (cDAC reader, SOS) to inspect NativeAOT runtime state through the same contract-based mechanism used by CoreCLR. Includes: - datadescriptor.inc with Thread, ThreadStore, MethodTable, ExInfo, EEAllocContext, GCAllocContext, RuntimeInstance types and globals - CMake integration using shared clrdatadescriptors.cmake infrastructure - GC sub-descriptor for workstation and server GC - Contract declarations: Thread (1001), Exception (1), RuntimeTypeSystem (1001) - Symbol export via --export-dynamic-symbol in build targets Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add ILC infrastructure to emit managed type field offsets as a cDAC sub-descriptor, enabling diagnostic tools to inspect managed type instances without runtime metadata. Types opt-in via [CdacType] and [CdacField] attributes in CoreLib. ILC discovers annotated types at compile time, computes field offsets, and emits a ContractDescriptor (DotNetManagedContractDescriptor) that the native runtime references as a sub-descriptor. Includes: - CdacTypeAttribute / CdacFieldAttribute in NativeAOT CoreLib - ManagedDataDescriptorNode: emits ContractDescriptor with JSON layout - ManagedDataDescriptorProvider: attribute-based type discovery with validation (rejects generics, duplicates, empty names) - Thread.NativeAot.cs annotated with ManagedThread descriptor fields Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
9462d5c to
f226bc3
Compare
| private ManagedThreadId _managedThreadId; | ||
| [CdacField("Name")] | ||
| private string? _name; | ||
| [CdacField("StartHelper")] |
There was a problem hiding this comment.
Is the StartHelper field actually used in the diagnostic stack somewhere?
There was a problem hiding this comment.
I don't think so, I was using these values to test that the descriptor was generated properly.
- Move ThreadStore from RuntimeInstance to direct global pointer (jkotas) - Update doc comment to clarify SystemModule-only scope (copilot-bot) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
| } | ||
|
|
||
| /// <summary> | ||
| /// The cDAC descriptor field name. If not specified, the field's declared name is used. |
There was a problem hiding this comment.
We may want to always publish managed types and fields under their actual names and disallow any renaming, so that the lookup using cdac contract and lookup using symbols is seamlessly interchangeable.
| rootProvider.AddCompilationRoot(descriptorNode.JsonBlobNode, "Managed descriptor JSON data"); | ||
| } | ||
|
|
||
| private void DiscoverAnnotatedTypes(ManagedDataDescriptorNode descriptorNode) |
There was a problem hiding this comment.
Instead of doing this, we should do this in EETypeNode so that:
a) we don't need to load all types from corelib
b) we don't generate this info for types that don't have a MethodTable
(EETypeNode is the thing that generates the MethodTable)
EETypeNode.ComputeNonRelocationDependencies should check for the custom attribute on the type and if it exists, drop a factory.ManagedTypeDataDescriptor(_type) to the graph. This will be a new node (e.g. ManagedTypeDataDescriptorNode) that will largely work the same as e.g. ExactMethodInstantiationsEntryNode - there will be one of these nodes in the graph for each type with a CDAC descriptor. We'll collect them in MetadataManager as they show up (same as ExactMethodInstantiationsEntryNode) and there will be an API on MetadataManager to get a list of them when we're writing the object file (same as for ExactMethodInstantiationsEntryNode).
There was a problem hiding this comment.
I expect we will end up with some static fields too. Is this going to work well for static fields?
There was a problem hiding this comment.
One more node then - not just ManagedTypeDataDescriptorNode: instead of that, add InstanceFieldDataDescriptorNode and StaticFieldDataDescritorNode. The static field one will be added when the static base for the type in question is generated (GCStaticsNode, NonGCStaticsNode, ThreadStaticsNode).
How are we going to embed their addresses in a textual JSON though?
There was a problem hiding this comment.
How are we going to embed their addresses in a textual JSON though?
The textual JSON has an index into a side-table with absolute addresses.
There was a problem hiding this comment.
Thinking about this more, we could also get away without the node, we already have API on MetadataManager to go over all EETypes (GetTypesWithEETypes) and all static bases (GetTypesWithStaticBases). We can just go over those and check for the attributes once we're ready to generate the JSON. So skip the new nodes.
Add ClrNativeAotSubset to the HasCdacBuildTool condition in runtime.proj so the cdac-build-tool processes datadescriptor.inc for NativeAOT instead of linking a stub contract descriptor. Add missing GPTR_DECL for g_pFreeObjectEEType in datadescriptor.h. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| if (desc.FieldMappings is not null) | ||
| { | ||
| // Check if any cDAC name maps to this managed field name | ||
| cdacFieldName = null; | ||
| foreach (var kvp in desc.FieldMappings) | ||
| { | ||
| if (kvp.Value == fieldName) | ||
| { | ||
| cdacFieldName = kvp.Key; | ||
| break; | ||
| } | ||
| } |
There was a problem hiding this comment.
Field selection with FieldMappings currently does an O(fields × mappings) scan by iterating the entire FieldMappings dictionary for every instance field to find a matching managed field name. This is fine for a few fields, but will scale poorly as more managed types/fields are added. Consider building a reverse map (managed field name → cDAC field name) once per ManagedTypeDescriptor so the loop is O(fields).
- Refactor ManagedDataDescriptorNode to use MetadataManager.GetTypesWithEETypes() for type discovery (Michal), ensuring only types with MethodTables are included - Remove name overriding from CdacTypeAttribute and CdacFieldAttribute (Jan), types and fields are published under their actual managed names - Remove StartHelper field annotation (Jan) — not used in diagnostics - Fix Linux build break: add SKIP_TRACING_DEFINITIONS to datadescriptor CMake interface to avoid clretwallmain.h dependency - Embed JSON inline in descriptor node instead of separate BlobNode - Restore RuntimeInstance.h whitespace Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| }; | ||
| typedef DPTR(RuntimeInstance) PTR_RuntimeInstance; | ||
|
|
||
|
|
There was a problem hiding this comment.
revert this code diff
| } | ||
|
|
||
| GPTR_IMPL_INIT(RuntimeInstance, g_pTheRuntimeInstance, NULL); | ||
| GPTR_IMPL_INIT(ThreadStore, g_pThreadStore, NULL); |
There was a problem hiding this comment.
This should be a static field on ThreadStore like Jan mentioned. We want it to match the coreclr implementation
Move g_pThreadStore global to ThreadStore::s_pThreadStore static member, matching how CoreCLR exposes its ThreadStore pointer for the cDAC. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
| // Use 0 (indeterminate) for reference types | ||
| int typeSize = type.IsValueType ? type.InstanceFieldSize.AsInt : 0; | ||
|
|
||
| sb.Append('"').Append(type.GetName()).Append("\":["); | ||
| sb.Append(typeSize); | ||
| sb.Append(",{"); | ||
|
|
There was a problem hiding this comment.
The JSON emitted for each entry in the "types" dictionary is not in the compact data-descriptor format expected by ContractDescriptorParser.TypeDescriptorConverter. The converter requires each type value to be a JSON object (with optional "!" size sigil and field-name properties), but this code emits an array [typeSize,{...}], which will fail to deserialize and cause the managed sub-descriptor to be ignored by the reader.
| // Use 0 (indeterminate) for reference types | ||
| int typeSize = type.IsValueType ? type.InstanceFieldSize.AsInt : 0; | ||
|
|
||
| sb.Append('"').Append(type.GetName()).Append("\":["); | ||
| sb.Append(typeSize); | ||
| sb.Append(",{"); | ||
|
|
There was a problem hiding this comment.
Reference types are currently encoded with a size value of 0 (see comment and typeSize computation). In the compact descriptor format, an indeterminate-size type must omit the "!" size property entirely; specifying size 0 makes the type determinate with an incorrect size and can break pointer arithmetic logic in consumers.
| // Use 0 (indeterminate) for reference types | |
| int typeSize = type.IsValueType ? type.InstanceFieldSize.AsInt : 0; | |
| sb.Append('"').Append(type.GetName()).Append("\":["); | |
| sb.Append(typeSize); | |
| sb.Append(",{"); | |
| sb.Append('"').Append(type.GetName()).Append("\":["); | |
| if (type.IsValueType) | |
| { | |
| sb.Append(type.InstanceFieldSize.AsInt); | |
| sb.Append(",{"); | |
| } | |
| else | |
| { | |
| sb.Append('{'); | |
| } |
| private static void EmitTypeJson(StringBuilder sb, EcmaType type) | ||
| { | ||
| // Use 0 (indeterminate) for reference types | ||
| int typeSize = type.IsValueType ? type.InstanceFieldSize.AsInt : 0; | ||
|
|
||
| sb.Append('"').Append(type.GetName()).Append("\":["); |
There was a problem hiding this comment.
type.GetName() returns only the metadata type name (no namespace), so the descriptor keys can easily collide (e.g., two annotated types named "Timer" in different namespaces). Consider using a fully-qualified managed name (namespace + nesting) for the JSON key to ensure uniqueness within the descriptor.
| private static void EmitTypeJson(StringBuilder sb, EcmaType type) | |
| { | |
| // Use 0 (indeterminate) for reference types | |
| int typeSize = type.IsValueType ? type.InstanceFieldSize.AsInt : 0; | |
| sb.Append('"').Append(type.GetName()).Append("\":["); | |
| private static string GetQualifiedTypeName(MetadataType type) | |
| { | |
| if (type.ContainingType is not null) | |
| { | |
| return GetQualifiedTypeName(type.ContainingType) + "+" + type.GetName(); | |
| } | |
| if (string.IsNullOrEmpty(type.Namespace)) | |
| { | |
| return type.GetName(); | |
| } | |
| return type.Namespace + "." + type.GetName(); | |
| } | |
| private static void EmitTypeJson(StringBuilder sb, EcmaType type) | |
| { | |
| // Use 0 (indeterminate) for reference types | |
| int typeSize = type.IsValueType ? type.InstanceFieldSize.AsInt : 0; | |
| sb.Append('"').Append(GetQualifiedTypeName(type)).Append("\":["); |
Move add_subdirectory(datadescriptor) after add_subdirectory(eventpipe) so that the generated clretwallmain.h header exists before the datadescriptor intermediary compiles. This removes the need for the SKIP_TRACING_DEFINITIONS workaround. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
7f16fd5 to
3b5d334
Compare
The datadescriptor compilation unit only needs type layouts (offsetof), not ETW tracing headers. Define SKIP_TRACING_DEFINITIONS on the interface target to skip the clretwallmain.h include from gcenv.h. This matches the pattern used by clrgc.enabled.cpp. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
3b5d334 to
656e2c7
Compare
🤖 Copilot Code Review — PR #126972Note This review was generated by GitHub Copilot using multi-model analysis (Claude Opus 4.6 primary, Claude Sonnet 4.5 sub-agent). A GPT-5.3-Codex sub-agent was also launched but did not complete within the 10-minute timeout. Holistic AssessmentMotivation: Well-justified. NativeAOT needs cDAC support for diagnostic tool compatibility. The infrastructure enables SOS and the cDAC reader to inspect NativeAOT runtime state through the same contract-based mechanism used by CoreCLR. Approach: Sound. The PR reuses the existing shared Summary: Detailed Findings
|
Note
This PR was created with assistance from GitHub Copilot.
Summary
Adds the cDAC data descriptor infrastructure for NativeAOT, enabling diagnostic tools (cDAC reader, SOS) to inspect NativeAOT runtime state through the same contract-based mechanism used by CoreCLR.
Changes
Native data descriptor (
datadescriptor.inc)cdac_data<>friend patternILC managed type descriptor (
ManagedDataDescriptorNode)ContractDescriptor(DotNetManagedContractDescriptor) with JSON-encoded type layoutsCDAC_GLOBAL_SUB_DESCRIPTORSystem.Threading.Threadfields (ManagedThreadId, Name, StartHelper)Build integration
clrdatadescriptors.cmakeinfrastructure--export-dynamic-symbolinMicrosoft.NETCore.Native.targetsValidation
build.cmd clr.aot+libs -rc release— ✅ 0 errors, 0 warningsRuntime.WorkstationGC.libvia dumpbin