[release/2.1] Clean up `E_FAIL` compilation errors

[dougbu]

The SharedFx and later jobs emit loads (around 1000) of errors like:

Unspecified error (Exception from HRESULT: 0x80004005 (E_FAIL)) while compiling method MessagePackSerializer.SerializeAsync

These errors do not fail builds but do make for rather noisy logs❕ Would like to get to the bottom of this…

Problems are limited to the internal build pipeline and first show up around https://dev.azure.com/dnceng/internal/_build/results?buildId=997298. It's not clear whether the problem existed before then because most builds were unsuccessful in the earlier "Build and test: Windows" job.

[dougbu]

/fyi @dotnet/aspnet-build I need to download a binary log to debug timeouts I'm seeing in internal (i.e. actual validation😀) builds of my branch for #30729. I'll self-assign this and fix it if the problem is fairly obvious. But, because this issue doesn't cause build failures, I won't spend too much time on it.

[dougbu]

Found errors mostly or all are about await-able methods. This problem likely relates to dotnet/runtime#11126, an issue should have been addressed in dotnet/coreclr#24386 We or dotnet/runtime likely don't have the right version of something in our 2.1 branches.

@trylek @AntonLapounov @jkotas given that context, how can we clean up our messy builds❔

[AntonLapounov]

We have dotnet/coreclr#24386 fix in .NET Core 3.x branches, but not in 2.x ones. Considering .NET Core 2.1 is near its end-of-support, I am not sure whether there will be another release.

[dougbu]

.NET Core 2.1 is near its end-of-support

This isn't really true for ASP.NET Core because .NET Framework on Windows exists. From https://dotnet.microsoft.com/platform/support/policy/aspnet

To help facilitate migrating applications to ASP.NET Core on .NET Core, the following ASP.NET Core 2.1 (latest patched version only) packages will be supported on the .NET Framework and follow the support cycle for those .NET Framework versions.

Admittedly, https://dotnet.microsoft.com/platform/support/policy/aspnetcore-2.1 lists the packages from this repo and dotnet/extensions that will be supported going forward and the platform-specific packages derived from Microsoft.AspNetCore.App and Microsoft.AspNetCore.All e.g. runtime.win-x86.Microsoft.AspNetCore.All (where we use crossgen) are not listed. I doubt that's the end of the story however.

/fyi @Pilchie is working with a few others to add dotnet/corefx / dotnet/core-setup packages to the package listing.

From #30729

Have you tried <Exec StandardOutputImportance="Low" StandardErrorImportance="Low" …?

Doesn't sound like a great option because important problems could be lost. My strong preference is to avoid the E_FAIL errors because they make finding real issues nearly impossible.

[AntonLapounov]

@dougbu You may discuss with danmoseley whether that would meet the bar for release/2.1.

[dougbu]

You may discuss with danmoseley whether that would meet the bar for release/2.1.

@AntonLapounov thanks. Will follow up once I have confirmation on packages like runtime.win-x86.Microsoft.AspNetCore.All. @Pilchie will we need those longer term❔

[dougbu]

I should have mentioned this issue has existed forever. What's new is building on AzDO instead of TeamCity. The extra verbiage is much more difficult to get through because TeamCity automatically hides large swaths of build output unless you really want to see it.

[Pilchie]

will we need those longer term❔

I'm not sure yet - there is a meeting next week to sort out the plan for supporting ASP.NET Core 2.1 on .NET Framework after .NET Core 2.1 goes out of support. Hopefully that will bring some clarity here.

[dougbu]

To confirm: You meant to close this @Pilchie and will reopen it if we need the runtime packs post-2.1-EOL❔

[Pilchie]

Whoops - sorry! No, I didn't mean to close it.

[dougbu]

Errors are definitely annoying but we can wade through them forever more, just as we have been.