[wasm][aot] Blazor .NET 8: llvm-size.exe command line args too long

[gdunit]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

After upgrading to .NET 8, AOT compilation of my Blazor WASM application fails with the following very long error message ([...] is a placeholder for the AOT'd DLL paths):

WasmApp.Native.targets(418,7): Error MSB3073 : The command "llvm-size.exe -d --format=sysv [...] exited with code 9020.

This occurs after the assemblies themselves are AOt'd, following the "Compiling assembly bitcode files with -O2 ..." process.
I am running Windows 10.

The area in the WasmApp.Native.targets file is as follows: interestingly, my IDE says that the 'ConsoleToMsBuild' attribute is not allowed here, and highlights it as an error, I am not sure if this is a red herring:

<!-- for AOT builds we use llvm-size tool to collect size of the DATA segment in each object file -->
      <Exec Command="llvm-size$(_ExeExt) -d --format=sysv @(_AOTObjectFile, ' ')"
          Condition="'$(_WasmShouldAOT)' == 'true'"
          IgnoreStandardErrorWarningFormat="true"
          ConsoleToMsBuild="true"
          StandardOutputImportance="low" StandardErrorImportance="low"
          EnvironmentVariables="@(EmscriptenEnvVars)" >

My .csproj file has the following blazor-related properties:

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <RunAOTCompilation>true</RunAOTCompilation>
    <BlazorWebAssemblyLoadAllGlobalizationData>true</BlazorWebAssemblyLoadAllGlobalizationData>
    <BlazorEnableTimeZoneSupport>true</BlazorEnableTimeZoneSupport>
    <Nullable>disable</Nullable>
  </PropertyGroup>

The AOT process previously worked on .NET 7.

Expected Behavior

The AOT compilation process should execute as per .NET 7.

Steps To Reproduce

I have been unable to reproduce on a fresh .NET 8 project so far.

Exceptions (if any)

WasmApp.Native.targets(418,7): Error MSB3073 : The command "llvm-size.exe -d --format=sysv [...] exited with code 9020.

.NET Version

8.0.100

Anything else?

I am invoking the publish process from the command line to an IIS folder on my local machine.
My IDE is Rider 2023.3.2.

[ghost]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

After upgrading to .NET 8, AOT compilation of my Blazor WASM application fails with the following very long error message ([...] is a placeholder for the AOT'd DLL paths):

WasmApp.Native.targets(418,7): Error MSB3073 : The command "llvm-size.exe -d --format=sysv [...] exited with code 9020.

This occurs after the assemblies themselves are AOt'd, following the "Compiling assembly bitcode files with -O2 ..." process.
I am running Windows 10.

The area in the WasmApp.Native.targets file is as follows: interestingly, my IDE says that the 'ConsoleToMsBuild' attribute is not allowed here, and highlights it as an error, I am not sure if this is a red herring:

<!-- for AOT builds we use llvm-size tool to collect size of the DATA segment in each object file -->
      <Exec Command="llvm-size$(_ExeExt) -d --format=sysv @(_AOTObjectFile, ' ')"
          Condition="'$(_WasmShouldAOT)' == 'true'"
          IgnoreStandardErrorWarningFormat="true"
          ConsoleToMsBuild="true"
          StandardOutputImportance="low" StandardErrorImportance="low"
          EnvironmentVariables="@(EmscriptenEnvVars)" >

My .csproj file has the following blazor-related properties:

  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <RunAOTCompilation>true</RunAOTCompilation>
    <BlazorWebAssemblyLoadAllGlobalizationData>true</BlazorWebAssemblyLoadAllGlobalizationData>
    <BlazorEnableTimeZoneSupport>true</BlazorEnableTimeZoneSupport>
    <Nullable>disable</Nullable>
  </PropertyGroup>

The AOT process previously worked on .NET 7.

Expected Behavior

The AOT compilation process should execute as per .NET 7.

Steps To Reproduce

I have been unable to reproduce on a fresh .NET 8 project so far.

Exceptions (if any)

WasmApp.Native.targets(418,7): Error MSB3073 : The command "llvm-size.exe -d --format=sysv [...] exited with code 9020.

.NET Version

8.0.100

Anything else?

I am invoking the publish process from the command line to an IIS folder on my local machine.
My IDE is Rider 2023.3.2.

Author:	gdunit
Assignees:	-
Labels:	arch-wasm, untriaged, area-Codegen-AOT-mono
Milestone:	-

[lambdageek]

/cc @maraf @akoeplinger

[lambdageek]

@gdunit could you try running dotnet publish with the /bl command line option. that should leave a binary msbuild log msbuild.log in the current directory. That would help us to see if llvm-size printed any additional output before crashing

[ghost]

This issue has been marked needs-author-action and may be missing some important information.

[gdunit]

@lambdageek thanks - I have run this and obtained the log.
I believe the log file contains some sensitive information. Is there a secure location I can upload it to?

[gdunit]

Also, I tried running the llvm-size.exe file locally from cmd with the arguments that are specified in the logs.
It prints information, but the command line can't accept the length of the arguments (the args are nearly 38k characters) so it fails part-way through. I wonder if this could be the issue?

[lambdageek]

@lambdageek thanks - I have run this and obtained the log. I believe the log file contains some sensitive information. Is there a secure location I can upload it to?

@gdunit I'm not sure what's the best way. Trying to find out.

It seems like you already found the place in the binlog where the llvm-size tool is invoked, though. Does it print any output when you run it?

But I think actually you diagnosed the issue:

the command line can't accept the length of the arguments (the args are nearly 38k characters) so it fails part-way through. I wonder if this could be the issue?

This is most likely the problem, yea.

llvm-size --help says it can read arguments from a file: llvm-size @FILE, @akoeplinger I wonder if we can update the target to use that.

[lambdageek]

I believe the log file contains some sensitive information. Is there a secure location I can upload it to?

@gdunit Please create an issue at https://developercommunity.visualstudio.com/dotnet/report and attach the binlog there, and then post a link to the created issue back here. (Sorry this is kind of a convoluted process).

If you already looked at the binlog (using the MSBuild binary log viewer, for example https://msbuildlog.com/) - all I'm really trying to get is the output from the llvm-size Exec task. If you can copy/paste it and redact the object file names (if they're sensitive) that is probably good enough.

I think you're probably right to suspect that it's just an issue of a huge command line. But I just want to verify that we're not suppressing some real problem - like LLVM can't read the object files for some reason.

[gdunit]

If you already looked at the binlog (using the MSBuild binary log viewer, for example https://msbuildlog.com/) - all I'm really trying to get is the output from the llvm-size Exec task. If you can copy/paste it and redact the object file names (if they're sensitive) that is probably good enough.

@lambdageek Sure. Here are the outputs, unfortunately no more information than what I originally posted (again removed the object file paths / info for brevity):

The system cannot execute the specified program.
Errors:
    The command "llvm-size.exe -d --format=sysv [redacted].dll.o (repeat ~200x)" exited with code 9020.

llvm-size --help says it can read arguments from a file: llvm-size @file

That would be perfect if that is possible to change. I tried it, and it works a charm (I copied all the parameters into a file in the same dir as the llvm-size executable called parameters.txt):

Command: llvm-size.exe @parameters.txt

[akoeplinger]

As a workaround to skip the _WasmCalculateInitialHeapSizeFromBitcodeFiles target which runs the problematic llvm-size command you can set this property in the csproj:

<WasmInitialHeapSize>16777216</WasmInitialHeapSize>

[gdunit]

@akoeplinger Thanks for the suggestion.
Unfortunately, I still get exactly the same error, it looks like the llvm-size.exe is still called.

From my .csproj file:

 <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <WasmInitialHeapSize>16777216</WasmInitialHeapSize>
    <RunAOTCompilation>true</RunAOTCompilation>

I tried putting this before / after the 'RunAOTCompilation' command, but no effect. I also tried adding the property to all of the .csproj files for projects referenced from the main blazor project, but again no dice.

[akoeplinger]

@gdunit ah sorry, my bad, in .net 8.0 the property was called EmccInitialHeapSize.

[gdunit]

@akoeplinger Thanks - it works now, albeit initially I got a different error which I managed to resolve (recording here in case it helps others):

wasm-ld : error : initial memory too small, 22667344 bytes needed

I updated the allocation in line with the error, note that it has to be a multiple of 65536 (64 * 1024), so:

<EmccInitialHeapSize>22675456</EmccInitialHeapSize>

AOT now works fine. Thanks for your support and the workaround.