clr build fails on Windows ARM64

[AndyAyersMS]

Windows arm box. Synced to bd7e60a

.\build.cmd -subset clr -c Release

...

 ILCompiler -> C:\repos\runtime0\artifacts\bin\coreclr\windows.arm64.Release\ilc\ilc.dll
  Optimizing assemblies for size. This process might take a while.
C:\repos\runtime0\.dotnet\sdk\7.0.100-preview.7.22377.5\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.CrossGen.targets(465,5): error : Process terminated. Access Violation:
 Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not sup
port throwing an AccessViolationException. [C:\repos\runtime0\src\coreclr\tools\aot\ILCompiler\ILCompiler.csproj]
C:\repos\runtime0\.dotnet\sdk\7.0.100-preview.7.22377.5\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.CrossGen.targets(353,5): error NETSDK1096: Optimizing assemblies for p
erformance failed. You can either exclude the failing assemblies from being optimized, or set the PublishReadyToRun property to false. [C:\repos\runtime0\src\coreclr\tools
\aot\ILCompiler\ILCompiler.csproj]

Check also fails, and the same thing happens at 06350b7

VS isn't working on my box for some reason, will try and debug when I get the chance.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[AndyAyersMS]

Possibly a dup of #72645

[AndyAyersMS]

Both the commits above should include the workaround #74211.

[AntonLapounov]

Could you please try to create a crash dump?

[MichalStrehovsky]

This is using the Preview 7 crossgen2 that has the bug and crashes most of the time. It will only resolve once we update the SDK that is used to build the repo to RC1.

We could consider rolling back the change that publishes ILCompiler as ready to run. We only really need it in the release branch.

[AndyAyersMS]

Extracted the args from the binlog. Running on the command line I get two different behaviors:

C:\home>C:\Users\andya\.nuget\packages\microsoft.netcore.app.crossgen2.win-arm64\7.0.0-preview.7.22375.6\tools\crossgen2.exe @x.rsp
Process terminated. Access Violation: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. The application will be terminated since this platform does not support throwing an AccessViolationException.

C:\home>C:\Users\andya\.nuget\packages\microsoft.netcore.app.crossgen2.win-arm64\7.0.0-preview.7.22375.6\tools\crossgen2.exe @x.rsp
Error: Object reference not set to an instance of an object.
System.NullReferenceException: Object reference not set to an instance of an object.
   at Internal.Runtime.MethodTable.get_RareFlags() + 0x8
   at Internal.Runtime.MethodTable.get_DynamicModule() + 0x1c
   at System.Runtime.DispatchResolve.FindInterfaceMethodImplementationTarget(MethodTable*, MethodTable*, UInt16, MethodTable**) + 0x34
   at System.Runtime.CachedInterfaceDispatch.RhResolveDispatchWorker(Object, Void*, DispatchCellInfo&) + 0x38
   at System.Runtime.CachedInterfaceDispatch.RhpCidResolve_Worker(Object, IntPtr) + 0x30
   at System.Runtime.CachedInterfaceDispatch.RhpCidResolve(IntPtr, IntPtr) + 0x10
   at Internal.TypeSystem.Ecma.EcmaModule.EcmaObjectLookupHashtable.CompareKeyToValue(EntityHandle, EcmaModule.IEntityHandleObject) + 0x24
   at Internal.TypeSystem.LockFreeReaderHashtable`2.TryGetValue(TKey, TValue&) + 0xd4
   at Internal.TypeSystem.Ecma.EcmaModule.GetObject(EntityHandle, NotFoundBehavior) + 0x34
   at Internal.TypeSystem.Ecma.EcmaType.<GetVirtualMethods>d__44.MoveNext() + 0x284
   at Internal.TypeSystem.MetadataVirtualMethodAlgorithm.FindMatchingVirtualMethodOnTypeByNameAndSig(MethodDesc, DefType, Boolean, Func`3) + 0x134
   at Internal.TypeSystem.MetadataVirtualMethodAlgorithm.FindBaseUnificationGroup(MetadataType, MetadataVirtualMethodAlgorithm.UnificationGroup) + 0xa8
   at Internal.TypeSystem.MetadataVirtualMethodAlgorithm.FindVirtualFunctionTargetMethodOnObjectType(MethodDesc, MetadataType) + 0x100
   at Internal.TypeSystem.Ecma.EcmaType.GetFinalizer() + 0x8c
   at ILCompiler.CompilerTypeSystemContext.EnsureLoadableTypeUncached(TypeDesc) + 0x650
   at Internal.TypeSystem.LockFreeReaderHashtable`2.CreateValueAndEnsureValueIsInTable(TKey) + 0x20
   at ILCompiler.CompilerTypeSystemContext.EnsureLoadableMethod(MethodDesc) + 0x60
   at ILCompiler.ReadyToRunRootProvider.CheckCanGenerateMethod(MethodDesc) + 0x50
   at ILCompiler.ReadyToRunRootProvider.RootMethods(TypeDesc, String, IRootingServiceProvider) + 0xd8
   at ILCompiler.ReadyToRunRootProvider.AddCompilationRoots(IRootingServiceProvider) + 0x280
   at ILCompiler.Compilation..ctor(DependencyAnalyzerBase`1, NodeFactory, IEnumerable`1, ILProvider, DevirtualizationManager, IEnumerable`1, Logger, InstructionSetSupport) + 0x16c
   at ILCompiler.ReadyToRunCodegenCompilation..ctor(DependencyAnalyzerBase`1, NodeFactory, IEnumerable`1, ILProvider, Logger, DevirtualizationManager, IEnumerable`1, String, InstructionSetSupport, Boolean, Boolean, Boolean, Boolean, Func`2, String, Boolean, String, Int32, Boolean, Int32, ProfileDataManager, ReadyToRunMethodLayoutAlgorithm, ReadyToRunFileLayoutAlgorithm, Int32, Boolean) + 0x104
   at ILCompiler.ReadyToRunCodegenCompilationBuilder.ToCompilation() + 0x578
   at ILCompiler.Program.RunSingleCompilation(Dictionary`2, InstructionSetSupport, String, Dictionary`2, HashSet`1, CompilerTypeSystemContext) + 0x13e4
   at ILCompiler.Program.Run(String[]) + 0xbbc
   at ILCompiler.Program.Main(String[]) + 0x34

Uploading a dump now (it will take a while); will send you the link.

[AntonLapounov]

Thank you. I am pretty sure it is a dup of #72645; you can delete the crash dump.

[AndyAyersMS]

Is there any workaround?

[MichalStrehovsky]

Is there any workaround?

Comment out:

runtime/src/coreclr/tools/aot/ILCompiler/ILCompiler.csproj

Line 14 in 0fe57ca

<PublishReadyToRun>true</PublishReadyToRun>

We could also just check that in if more people are hitting it.

[MichalStrehovsky]

(Or make it conditional on being on an ARM64 host - that might be even bettter. Our build machines are never ARM64 because we would have seen this bug long time ago.)

[AntonLapounov]

Disabling server GC (e.g., by hacking C:\Users\andya\.nuget\packages\microsoft.netcore.app.crossgen2.win-arm64\7.0.0-preview.7.22375.6\tools\crossgen2.runtimeconfig.json) may also work if you are on a machine with ≤ 64 processors.

[adamsitnik]

I confirm that the workaround shared by @MichalStrehovsky in #74375 (comment) works.

[AndyAyersMS]

Yes, it does. Thanks Michal. I will close this as a dup.

We could also just check that in if more people are hitting it.

If we're not going to update the SDK anytime soon this seems like a good idea.