Amazon DynamoDBContext - NET 7 AOT Incompatibilities #2542

rac146 · 2023-02-15T21:57:03Z

Describe the bug

Was running some NET7 AOT experiments with DynamoDBContext and was receiving runtime errors after doing an AOT compilation:

System.NotSupportedException: 'System.Collections.Generic.HashSet`1[System.Boolean]' is missing native code or metadata. This can happen for code that is not compatible with trimming or AOT. Inspect and fix trimming and AOT related warnings that were generated when the app was published. For more information see https://aka.ms/nativeaot-compatibility
   at System.Reflection.Runtime.General.TypeUnifier.WithVerifiedTypeHandle(RuntimeConstructedGenericTypeInfo, RuntimeTypeInfo[]) + 0x88
   at Amazon.DynamoDBv2.CollectionConverter.<GetTargetTypes>d__1.MoveNext() + 0x1d4
   at Amazon.DynamoDBv2.ConverterCache.AddConverter(Converter converter, DynamoDBEntryConversion conversion) + 0x9b
   at Amazon.DynamoDBv2.DynamoDBEntryConversion.AddConverter(Type type) + 0x41
   at Amazon.DynamoDBv2.DynamoDBEntryConversion.AddConverters(String suffix) + 0x184
   at Amazon.DynamoDBv2.DynamoDBEntryConversion..ctor(ConversionSchema schema, Boolean isImmutable) + 0x6e
   at Amazon.DynamoDBv2.DynamoDBEntryConversion..cctor() + 0x25
   at System.Runtime.CompilerServices.ClassConstructorRunner.EnsureClassConstructorRun(StaticClassConstructionContext*) + 0xc6

Expected Behavior

For DynamoDB to be AOT NET 7 supported

Current Behavior

Errors out after dotnet publish in NET7 AOT mode

Reproduction Steps

Attempt to run any DynamoDB queries that use DynamoDBContext with latest version of AWS SDK

Possible Solution

Decided to hack into the SDK to find the culprit. Discovered .MakeGenericType is throwing the runtime errors.
Switched these 4 lines for experimentation purposes:

In DynamoDBEntryConversion.cs:
Line 651: var nullableType = typeof(Nullable<>).MakeGenericType(type)
Potential Fix?: yield return typeof(Nullable<>);

In SchemaV1.cs:

Line 379: yield return pt.MakeArrayType();
Fix?: yield return typeof(Array);
Line 382: yield return listType.MakeGenericType(pt);
Fix?: yield return typeof(List<>);
Line 384: yield return setType.MakeGenericType(pt);
Fix?: yield return typeof(HashSet<>);

This allowed my test application to run and work correctly. Is this is right long-term fix to the problem? I don't know, but just wanted to let you know!

Additional Information/Context

With AWS starting to push AOT further, it would be great to start working through the bugs that prevent an AOT build from working. I believe this fits as a bug opposed to a feature request, especially with the push towards AOT compatible lambdas!

AWS .NET SDK and/or Package version used

AWS SDK DynamoDBv2 3.7.101.44

Targeted .NET Platform

.NET 7

Operating System and version

Windows 10

The text was updated successfully, but these errors were encountered:

ashishdhingra · 2023-02-15T22:21:52Z

Hi @rac146,

Good afternoon.

Thanks for opening the issue. Just wanted to check if you referred the blog post Building serverless .NET applications on AWS Lambda using .NET 7 for the Native AOT support. As mentioned, currently, the AWSSDK.Core library used by all the .NET AWS SDKs must also be excluded from trimming. I guess this is also the case with AWSSDK.* assemblies. Could you please try adding AWSSDK.DynamoDBv2 in rd.xml and see if it works. We have the internal backlog item to support trimming for AWS SDK for .NET assemblies, however, do not have the concrete timeline by which it would be supported.

Thanks,
Ashish

rac146 · 2023-02-16T13:38:22Z

Hey Ashish,

Even excluding AWSSDK.DynamoDBv2 from trimming, the error still occurs. This error will present itself with or without trimming - this is flagged as an AOT analysis warning, not a trimming warning when running dotnet publish.

rac146 · 2023-02-16T13:56:25Z

Here are the detailed errors when running dotnet publish with <TrimmerSingleWarn>false</TrimmerSingleWarn>

D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Services\DynamoDBv2\Custom\Conversion\DynamoDBEntryConversion.cs(652): AOT analysis warning IL3050: Amazon.DynamoDBv2.Converter`1.d__0.MoveNext(): Using member 'System.Type.MakeGenericType(Type[])' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. The native code for this instantiation might not be available at runtime.

D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Services\DynamoDBv2\Custom\Conversion\SchemaV1.cs(379): AOT analysis warning IL3050: Amazon.DynamoDBv2.CollectionConverter.d__1.MoveNext(): Using member 'System.Type.MakeArrayType()' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. The code for an array of the specified type might not be available.

D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Services\DynamoDBv2\Custom\Conversion\SchemaV1.cs(382): AOT analysis warning IL3050: Amazon.DynamoDBv2.CollectionConverter.d__1.MoveNext(): Using member 'System.Type.MakeGenericType(Type[])' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. The native code for this instantiation might not be available at runtime.

D:\JenkinsWorkspaces\trebuchet-stage-release\AWSDotNetPublic\sdk\src\Services\DynamoDBv2\Custom\Conversion\SchemaV1.cs(384): AOT analysis warning IL3050: Amazon.DynamoDBv2.CollectionConverter.d__1.MoveNext(): Using member 'System.Type.MakeGenericType(Type[])' which has 'RequiresDynamicCodeAttribute' can break functionality when AOT compiling. The native code for this instantiation might not be available at runtime.

jesmiatka · 2023-04-13T07:15:04Z

I've got the current version of the AWSSDK working for now with DynamoDB if I add the following to the rd.xml

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
    <Assembly Name="AWSSDK.Core" Dynamic="Required All"></Assembly>
    <Assembly Name="AWSSDK.DynamoDBv2" Dynamic="Required All"></Assembly>
    <Assembly Name="System.Collections">
      <Type Name="System.Collections.Generic.HashSet`1[[System.Boolean,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Byte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Byte[],System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Char,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.DateTime,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Decimal,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Double,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Guid,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Int16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Int32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Int64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.UInt16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.UInt32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.UInt64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.SByte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Single,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.String,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Boolean,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Byte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Byte[],System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Char,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.DateTime,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Decimal,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Double,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Guid,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Int16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Int32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Int64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.UInt16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.UInt32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.UInt64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.SByte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Single,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.String,System.Runtime]]" Dynamic="Required All" />
    </Assembly>
  </Application>
</Directives>

Hope this helps other for now.

MrZoidberg · 2023-04-13T08:11:54Z

Adding to @jesmiatka. I needed these lines additionally to make it work:

<Type Name="System.Nullable`1[System.Byte]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.SByte]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.UInt16]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Int32]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.UInt32]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Int16]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Int64]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.UInt64]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Single]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Double]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Decimal]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Char]" Dynamic="Required All"/>'
<Type Name="System.Nullable`1[System.DateTime]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Guid]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Enum]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.Boolean]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.IO.MemoryStream]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.String]" Dynamic="Required All"/>

smartshader · 2023-05-12T13:35:41Z

@MrZoidberg In which assembly is System.Nullable? Can you share the whole Assembly entry in rd.xml?

branog68 · 2023-05-12T14:00:42Z

In System.Runtime. Here the whole rd.xml:

<Directives xmlns="http://schemas.microsoft.com/netfx/2013/01/metadata">
  <Application>
	<Assembly Name="AWSSDK.Core" Dynamic="Required All"></Assembly>
	<Assembly Name="AWSSDK.DynamoDBv2" Dynamic="Required All"></Assembly>
	<Assembly Name="System.Collections">
	  <Type Name="System.Collections.Generic.HashSet`1[[System.Boolean,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Byte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Byte[],System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Char,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.DateTime,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Decimal,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Double,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Guid,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Int16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Int32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Int64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.UInt16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.UInt32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.UInt64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.SByte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.Single,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.HashSet`1[[System.String,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Boolean,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Byte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Byte[],System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Char,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.DateTime,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Decimal,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Double,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Guid,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Int16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Int32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Int64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.UInt16,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.UInt32,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.UInt64,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.SByte,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.Single,System.Runtime]]" Dynamic="Required All" />
      <Type Name="System.Collections.Generic.List`1[[System.String,System.Runtime]]" Dynamic="Required All" />
	</Assembly>
	<Assembly Name="System.Runtime">
	  <Type Name="System.Nullable`1[System.Byte]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.SByte]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.UInt16]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Int32]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.UInt32]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Int16]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Int64]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.UInt64]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Single]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Double]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Decimal]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Char]" Dynamic="Required All"/>'
	  <Type Name="System.Nullable`1[System.DateTime]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Guid]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Enum]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.Boolean]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.IO.MemoryStream]" Dynamic="Required All"/>
	  <Type Name="System.Nullable`1[System.String]" Dynamic="Required All"/>
	</Assembly>
  </Application>
</Directives>

pjmvp · 2024-01-27T16:27:09Z

Any updates on this one? Or maybe with regard to .NET 8?

h4570 · 2024-02-24T22:43:55Z

In my scenario (dotnet8, managed runtime) I've used @branog68 modifications, but had to exclude these three lines due to compilation errors:

<Type Name="System.Nullable`1[System.Enum]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.IO.MemoryStream]" Dynamic="Required All"/>
<Type Name="System.Nullable`1[System.String]" Dynamic="Required All"/>

After this fix, DynamoDbContext started working.

Beau-Gosse-dev · 2024-02-26T19:13:42Z

Although it doesn't fix the issue, this PR marked these as not compatible and has some explanation of what it would take to fix.

Also, Microsoft recommends moving away from rd.xml files as they are no longer support and instead using the trimming options here: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options?pivots=dotnet-8-0

beeradmoore · 2024-03-06T21:39:04Z

Hey @Beau-Gosse-dev , I may be missing it but can you point to where Microsoft are moving away from rd.xml? I can only see that it is not listed on that site at all, not that they recommend not using it.

Beau-Gosse-dev · 2024-03-07T18:50:57Z

@beeradmoore There are other reference from Michal, but here is one: dotnet/runtime#72989 (comment)

beeradmoore · 2024-03-07T20:15:47Z

Perfect. Thanks @Beau-Gosse-dev

rac146 added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 15, 2023

ashishdhingra added the dynamodb label Feb 15, 2023

ashishdhingra added response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels Feb 15, 2023

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. label Feb 17, 2023

ashishdhingra added the needs-review label Feb 17, 2023

eddiemcs3 added p2 This is a standard priority issue aot Ahead of Time and removed needs-review labels Feb 24, 2023

ashishdhingra added the queued label Feb 24, 2023

ashishdhingra mentioned this issue Mar 24, 2023

DynamoDb SDK is not trim-safe for Native AOT at Table.LoadTable #2574

Closed

2 tasks

Beau-Gosse-dev mentioned this issue Apr 25, 2023

DynamoDB doesn't work with native AOT awslabs/dotnet-nativeaot-labs#10

Closed

h4570 mentioned this issue Feb 24, 2024

Tracking .NET 8 Support aws/aws-lambda-dotnet#1611

Closed

Dreamescaper mentioned this issue Apr 29, 2024

DynamoDBEntryConversion AOT compatibility #3300

Open

10 tasks

firenero mentioned this issue May 25, 2024

Native AOT AllocZero/EfficientDynamoDb#245

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Amazon DynamoDBContext - NET 7 AOT Incompatibilities #2542

Amazon DynamoDBContext - NET 7 AOT Incompatibilities #2542

rac146 commented Feb 15, 2023

ashishdhingra commented Feb 15, 2023

rac146 commented Feb 16, 2023

rac146 commented Feb 16, 2023

jesmiatka commented Apr 13, 2023

MrZoidberg commented Apr 13, 2023

smartshader commented May 12, 2023

branog68 commented May 12, 2023

pjmvp commented Jan 27, 2024

h4570 commented Feb 24, 2024

Beau-Gosse-dev commented Feb 26, 2024

beeradmoore commented Mar 6, 2024

Beau-Gosse-dev commented Mar 7, 2024

beeradmoore commented Mar 7, 2024

Amazon DynamoDBContext - NET 7 AOT Incompatibilities #2542

Amazon DynamoDBContext - NET 7 AOT Incompatibilities #2542

Comments

rac146 commented Feb 15, 2023

Describe the bug

Expected Behavior

Current Behavior

Reproduction Steps

Possible Solution

Additional Information/Context

AWS .NET SDK and/or Package version used

Targeted .NET Platform

Operating System and version

ashishdhingra commented Feb 15, 2023

rac146 commented Feb 16, 2023

rac146 commented Feb 16, 2023

jesmiatka commented Apr 13, 2023

MrZoidberg commented Apr 13, 2023

smartshader commented May 12, 2023

branog68 commented May 12, 2023

pjmvp commented Jan 27, 2024

h4570 commented Feb 24, 2024

Beau-Gosse-dev commented Feb 26, 2024

beeradmoore commented Mar 6, 2024

Beau-Gosse-dev commented Mar 7, 2024

beeradmoore commented Mar 7, 2024