Add missing RID combinations for Gentoo AND/OR hard-set RID for .NET build

[xgqt]

Description

Hello!

@vimproved raised a issue on the Gentoo IRC that after installing the from-source build of dotnet-sdk on musl the packages depending on dotnet-sdk would fail to find Microsoft.NETCore.App.Host.linux-x64
see: https://paste.gentoo.zip/uQFu5vOg

This really got us both confused cause even if .NET failed to find the nuget it should have the linux-musl-x64 RID and not linux-x64 on musl.
After a bit of digging we think we found the issue, that there are definitions for gentoo missing from the runtime.json file.
see: https://github.com/dotnet/runtime/blob/main/src/libraries/Microsoft.NETCore.Platforms/src/runtime.json

I did not open any PRs because i do not know if just adding it should be enough and I wanted to bring this up ASAP.

Reproduction Steps

compile the dontnet-sdk on Gentoo with musl variant, observe that that RID is gentoo.2.14-x64

Expected behavior

RID is gentoo.2.14-musl-x64

Actual behavior

RID is gentoo.2.14-x64

Regression?

No, unsupported yet.

Known Workarounds

Patch the runtime.json file - maybe?

Configuration

• 8.0.101
• Gentoo, VERSION_ID="2.14"
• amd64 (a.k.a. x64)
• yes (specific to config)

Other information

dotnet --info from a Gentoo musl chroot:

.NET SDK:
 Version:           8.0.101
 Commit:            6eceda187b
 Workload version:  8.0.100-manifests.2fd734c4

Runtime Environment:
 OS Name:     gentoo
 OS Version:  2.14
 OS Platform: Linux
 RID:         gentoo.2.14-x64
 Base Path:   /usr/lib/dotnet-sdk-8.0/sdk/8.0.101/

.NET workloads installed:
 Workload version: 8.0.100-manifests.2fd734c4
There are no installed workloads to display.

Host:
  Version:      8.0.1
  Architecture: x64
  Commit:       bf5e279d92

.NET SDKs installed:
  8.0.101 [/usr/lib/dotnet-sdk-8.0/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.1 [/usr/lib/dotnet-sdk-8.0/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.1 [/usr/lib/dotnet-sdk-8.0/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  DOTNET_ROOT       [/opt/dotnet-sdk-bin-8.0]

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download

[xgqt]

Tangent:
imho the Gentoo RID is unnecessary (or maybe even harmful) because:

1. Gentoo is not officially supported
2. Gentoo can build any linux RID combination, so a proper way to add Gentoo rid would be to just add one for each specific linux RID (gentoo-x86, gentoo-musl-x86, gentoo-arm, ...)

On the other hand if somehow adding a RID is just as easy as modding runtime.json Gentoo project can do it on is's own.

I would like to also hear opinion of a .NET dev on this subject.

[AraHaan]

I feel like there is no need as the runtime assembly code that gets generated would be the same as the normal linux-* RIDs that already exist.

[vimproved]

Currently experimenting with just building with -p:TargetRid=linux-musl-x64 -p:HostOSName=linux-musl like the CI does, if this works I think this will be the best solution. I agree that the Gentoo-specific RIDs seem unnecessary here, and it would be better to have the RIDs produced by the source packages match the ones from the upstream binaries, following our approach for other languages with similar systems (e.g. Rust).

[agocke]

Our recommendation is to avoid all distro-specific RIDs unless you have some reason for creating binaries that are different from our official ones.

You should be able to use the "portable" RIDs (linux-x64, linux-musl-x64, etc.) to produce binaries like .NET does.

[xgqt]

You should be able to use the "portable" RIDs (linux-x64, linux-musl-x64, etc.) to produce binaries like .NET does.

That does not seem to be the case, @vimproved reported that .NET SDK fails to compile if RID is enforced in a way he described above.

[agocke]

@vimproved reported that .NET SDK fails to compile if RID is enforced in a way he described above.

Where is the build log?

[xgqt]

@vimproved reported that .NET SDK fails to compile if RID is enforced in a way he described above.

Where is the build log?

I hope @vimproved will upload his last build log here.

But a different question is whether the options that he used are even supported outside of the CI environment. @agocke, do you know how this case would be handled in terms of passed configure options?

[agocke]

You will have to provide more details. I don't know which options are being used.

[tmds]

Currently experimenting with just building with -p:TargetRid=linux-musl-x64 -p:HostOSName=linux-musl

This is not the way to go. Distro specific build should have non-portable rids. This allows the SDK to distinguish between the portable assets that Microsoft provides through nuget.org and those built for the distro.

By default the vmr (https://github.com/dotnet/dotnet) will determine a non-portable rid based on /etc/os-release. If you prefer a different name, you can choose one by setting TargetRid. You should not set it to a rid used by Microsoft for their portable builds.

@vimproved raised a issue on the Gentoo IRC that after installing the from-source build of dotnet-sdk on musl the packages depending on dotnet-sdk would fail to find Microsoft.NETCore.App.Host.linux-x64
see: https://paste.gentoo.zip/uQFu5vOg

Since linux-x64 is a portable name, the SDK should be looking and finding the Microsoft built package for it on nuget.org: https://www.nuget.org/packages/Microsoft.NETCore.App.Host.linux-x64.

 * Running "dotnet-pkg-base_restore" for project: "GentooDotnetInfo.csproj" ...
 * /usr/lib/dotnet-sdk-8.0/dotnet restore --runtime linux-musl-x64 --source /var/tmp/portage/dev-dotnet/csharp-gentoodotnetinfo-1.1.1-r1/temp/nugets -maxCpuCount:16 Source/v1/gentoo-dotnet-info-app/GentooDotnetInfo/GentooDotnetInfo.csproj

What is GentooDotnetInfo for? And why is there an explicit --runtime linux-musl-x64 argument here?

Can you provide a set of steps that reproduce the issue?

[xgqt]

@tmds

This allows the SDK to distinguish between the portable assets that Microsoft provides through nuget.org and those built for the distro.

We would like to use official assets from nuget.org if that's possible.

Since linux-x64 is a portable name, the SDK should be looking and finding the Microsoft built package for it on nuget.org: https://www.nuget.org/packages/Microsoft.NETCore.App.Host.linux-x64.

You are missing the point.
It should not be looking for linux-x64 in the 1st place, since this is on musl it should be looking for linux-musl-x64.
This shows that when dotnet SDK is built form source on Gentoo with musl libc, then instead of trying to use linux-musl-x64 it will want to use linux-x64.

What is GentooDotnetInfo for?

GentooDotnetInfo is Gentoo's tool that shows dotnet+system information, in this case it is irrelevant, it could as well be any other package.

And why is there an explicit --runtime linux-musl-x64 argument here?

Because we restore nuget packages for this specific architecture-dependent package build only.

[tmds]

You are missing the point.
It should not be looking for linux-x64 in the 1st place, since this is on musl it should be looking for linux-musl-x64.
This shows that when dotnet SDK is built form source on Gentoo with musl libc, then instead of trying to use linux-musl-x64 it will want to use linux-x64.

What's in the SDK's Microsoft.NETCoreSdk.BundledVersions.props file for NETCoreSdkRuntimeIdentifier and NETCoreSdkPortableRuntimeIdentifier?

[tmds]

--source /var/tmp/portage/dev-dotnet/csharp-gentoodotnetinfo-1.1.1-r1/temp/nugets

You're specifying a --source argument on the restore.
What Microsoft.NETCore.App.Host.* packages are in the source folder?

Because we restore nuget packages for this specific architecture-dependent package build only.

I'm looking at the sources in https://gitlab.gentoo.org/dotnet/csharp-gentoodotnetinfo, and it's not clear why this would require specifying a rid. If you don't specify a rid, it should use the host that comes with Gentoo instead of looking for one elsewhere.

[xgqt]

--source /var/tmp/portage/dev-dotnet/csharp-gentoodotnetinfo-1.1.1-r1/temp/nugets

You're specifying a --source argument on the restore. What Microsoft.NETCore.App.Host.* packages are in the source folder?

This directory hosts all nuget packages that a given package requires.
We fetch them before built to satisfy the Gentoo Portage (the system package manager) sandbox.
It used to have packages only for the system RID selected based on the architecture + ones that are not architecture bound but due to this bug we had to collect all nuget packages that are bound to a specific .NET SDK version.

Btw this is very similar to what NixOS does.

Because we restore nuget packages for this specific architecture-dependent package build only.

I'm looking at the sources in https://gitlab.gentoo.org/dotnet/csharp-gentoodotnetinfo, and it's not clear why this would require specifying a rid. If you don't specify a rid, it should use the host that comes with Gentoo instead of looking for one elsewhere.

We had to specify the RID explicitly for each package (in restore (src_prepare), build (src_compile) and test (src_test) phases) to have a general way of building most .NET-based packages. In our tests most pkgs would fail to restore with our old way of collecting SDK-version-bound nugets (see above explanation in this comment).

[xgqt]

@tmds

What's in the SDK's Microsoft.NETCoreSdk.BundledVersions.props file for NETCoreSdkRuntimeIdentifier and NETCoreSdkPortableRuntimeIdentifier?

-> pwd
/usr/lib/dotnet-sdk-8.0
-> uname -a
Linux magma-chroot-gentoo-amd64-musl-gcc-stable 6.6.21-gentoo-magentalane-0-local-workstation-2024.03.27 #1 SMP PREEMPT_DYNAMIC Wed Mar 27 02:49:27 CET 2024 x86_64 AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx AuthenticAMD GNU/Linux
-> grep 'RuntimeIdentifier>' ./sdk/8.0.102/Microsoft.NETCoreSdk.BundledVersions.props
    <NETCoreSdkRuntimeIdentifier>gentoo.2.14-x64</NETCoreSdkRuntimeIdentifier>
    <NETCoreSdkPortableRuntimeIdentifier>linux-musl-x64</NETCoreSdkPortableRuntimeIdentifier>

[tmds]

If you change this line to match the .NET SDK framework version you'll no longer have the error:

https://gitlab.gentoo.org/dotnet/csharp-gentoodotnetinfo/-/blob/master/Source/v1/gentoo-dotnet-info-app/Directory.Build.props?ref_type=heads#L13

It's currently restoring the linux-x64 host for .NET 6.

[tmds]

This directory hosts all nuget packages that a given package requires.

And if you drop the --runtime argument, this directory will no longer have to include any linux-musl-x64 packages.

[xgqt]

If you change this line to match the .NET SDK framework version you'll no longer have the error:

https://gitlab.gentoo.org/dotnet/csharp-gentoodotnetinfo/-/blob/master/Source/v1/gentoo-dotnet-info-app/Directory.Build.props?ref_type=heads#L13

It's currently restoring the linux-x64 host for .NET 6.

I think you mean that I have to change NETCoreSdkRuntimeIdentifier? Otherwise I do not understand what you are saying.

Can NETCoreSdkRuntimeIdentifier be specified via -p: options during dotnet command execution or is it set-in-stone in the Microsoft.NETCoreSdk.BundledVersions.props file?

[tmds]

I think you mean that I have to change NETCoreSdkRuntimeIdentifier? Otherwise I do not understand what you are saying.

When you're building for .NET 8, set TargetFramework to net8.0.

[xgqt]

I think you mean that I have to change NETCoreSdkRuntimeIdentifier? Otherwise I do not understand what you are saying.

When you're building for .NET 8, set TargetFramework to net8.0.

How is this is relevant to this issue? We are discussing SDK architecture mismatch only.

We do not have to set it to net8.0 since we use RollForward=Major.
https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props#rollforward

[tmds]

We do not have to set it to net8.0 since we use RollForward=Major.
https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props#rollforward

When the TargetFramework is net6.0, the .NET 8 SDK tries to restore the Microsoft.NETCore.App.Host.linux-x64 for .NET 6, which is the cause of the restore failure.

When you set TargetFramework to match the SDK version, and you drop the --runtime argument, the SDK will use the host that got built for Gentoo.

[vimproved]

@vimproved reported that .NET SDK fails to compile if RID is enforced in a way he described above.

Where is the build log?

My main testing machine is currently not setup but I'll try to get you the log ASAP.

[vimproved]

When the TargetFramework is net6.0, the .NET 8 SDK tries to restore the Microsoft.NETCore.App.Host.linux-x64 for .NET 6, which is the cause of the restore failure.

Why does it do that despite the machine itself being musl? Is this intended behavior?

[tmds]

Why does it do that despite the machine itself being musl? Is this intended behavior?

Without looking deeper in the SDK, I don't know why it does this.

For most users it doesn't pose an issue because it can find the package on nuget.org.

Whereas the GentooDotnetInfo build doesn't use nuget.org and is restricted to what it finds in the --source directory.