Enable setting RID for `dotnet restore` with `-a` and `--os`

[richlander]

I thought we had enabled this earlier, but apparently not. This would make it easier and more efficient to produce platform-specific assets in platform-agnostic Dockerfiles.

I removed --no-restore and that works. It's not ideal. The other option is to set the RID directly, but that is a worse solution given that the goal is platform agnostic Dockerfile and project file. For example, caching works much better with --no-restore.

ich@merritt:~/git/dotnet-docker/samples/globalapp$ docker build --pull -t globalapp .
[+] Building 0.7s (12/14)                                                                              docker:default
=> [internal] load build definition from Dockerfile                                                             0.0s
=> => transferring dockerfile: 639B                                                                             0.0s
=> [internal] load metadata for mcr.microsoft.com/dotnet/runtime-deps:10.0                                      0.1s
=> [internal] load metadata for mcr.microsoft.com/dotnet/sdk:10.0                                               0.1s
=> [internal] load .dockerignore                                                                                0.0s
=> => transferring context: 2B                                                                                  0.0s
=> [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:10.0@sha256:c7445f141c04f1a6b454181bd098dcfa606c61ba0bd213d0a  0.0s
=> [stage-1 1/3] FROM mcr.microsoft.com/dotnet/runtime-deps:10.0@sha256:69ee6d1a1b7a92cc82a71001342554d4611a50  0.0s
=> [internal] load build context                                                                                0.0s
=> => transferring context: 869B                                                                                0.0s
=> CACHED [build 2/6] WORKDIR /source                                                                           0.0s
=> CACHED [build 3/6] COPY --link *.csproj .                                                                    0.0s
=> CACHED [build 4/6] RUN dotnet restore -a amd64 --os linux                                                    0.0s
=> CACHED [build 5/6] COPY --link . .                                                                           0.0s
=> ERROR [build 6/6] RUN dotnet publish -a amd64 --no-restore -o /app                                           0.5s
------                                                                                                                
> [build 6/6] RUN dotnet publish -a amd64 --no-restore -o /app:                                                      
0.484 /usr/share/dotnet/sdk/10.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.NET.Sdk.FrameworkReferenceResolution.targets(544,5): error NETSDK1112: The runtime pack for Microsoft.NETCore.App.Runtime.linux-x64 was not downloaded. Try running a NuGet restore with the RuntimeIdentifier 'linux-x64'. [/source/globalapp.csproj]
------
Dockerfile:13
--------------------
  11 |     # Copy source code and publish app
  12 |     COPY --link . .
  13 | >>> RUN dotnet publish -a $TARGETARCH --no-restore -o /app
  14 |     
  15 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c dotnet publish -a $TARGETARCH --no-restore -o /app" did not complete successfully: exit code: 1

[lbussell]

When this is resolved, the https://github.com/dotnet/dotnet-docker "globalapp" sample referenced here will need to be updated. See dotnet/dotnet-docker#6802 (comment)

[marcpopMSFT]

So you are running a restore but ignoring one or both of those options? Do you know what it's actually restoring for (ie which one is being ignored)?
dotnet restore -a amd64 --os linux

[baronfel]

binlogs.zip

Here's some binlogs from the repro thread reported for future investigation.

[baronfel]

full-scenario-binlogs.zip

Here are better binlogs that explain the scenario. The root of the problem is that for the Publish* properties, the ProcessFrameworkReference Target does not create PackageDownload items for the Runtime(s) required to successfully publish the app. This process does work for SelfContained, but fails for PublishSelfContained, PublishTrimmed, and PublishAOT (and maybe PublishReadyToRun but we didn't test that scenario).

Because these package download items aren't downloaded (and tracked in the generated project.assets.json) they aren't available for the no-restore publish that comes later.

[richlander]

Thanks for the investigation! Much appreciated.

[sbomer]

I think this is related to #26324 - I believe restore won't know which assets need to be restored for the publish scenario, unless _IsPublishing is set at restore time.

[baronfel]

@sbomer I don't think we need to care about _IsPublishing at all - if a user restores a project for a specific RID or RIDs and has PublishTrimmed set, that's enough signal for us to go get the apphost, runtime pack, IlLink Tasks, etc. similarly for PublishAOT - we can go get the NativeAOT runtime instead, and so on. If we want to enable --no-restore scenarios then Restore has to be more 'maximal' in nature.

[richlander]

@agocke asked me to try adding RuntimeIndentifers. It didn't change the result.

I'm also sharing my configuration to make it easier for folks to understand what I'm doing.

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <EnableSdkContainerSupport>true</EnableSdkContainerSupport>
    <PublishSingleFile>true</PublishSingleFile>
    <PublishTrimmed>true</PublishTrimmed>
    <RuntimeIdentifiers>linux-x64;linux-arm64</RuntimeIdentifiers>
  </PropertyGroup>

</Project>

FROM --platform=$BUILDPLATFORM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG TARGETARCH
WORKDIR /source

# Copy project file and restore as distinct layers
COPY --link *.csproj .
RUN dotnet restore -a $TARGETARCH --os linux

# Copy source code and publish app
COPY --link . .
RUN dotnet publish -a $TARGETARCH -o /app --no-restore


# Runtime stage
FROM mcr.microsoft.com/dotnet/runtime-deps:10.0
WORKDIR /app
COPY --link --from=build /app .
USER $APP_UID
ENTRYPOINT ["./globalapp"]

Strangely, I now have a different error. They may be more than one thing going on here.

$ docker build --pull -t app .
[+] Building 5.5s (16/17)                                                                                       docker:default
 => [internal] load build definition from Dockerfile                                                                      0.0s
 => => transferring dockerfile: 639B                                                                                      0.0s
 => [internal] load metadata for mcr.microsoft.com/dotnet/sdk:10.0                                                        0.1s
 => [internal] load metadata for mcr.microsoft.com/dotnet/runtime-deps:10.0                                               0.2s
 => [internal] load .dockerignore                                                                                         0.0s
 => => transferring context: 2B                                                                                           0.0s
 => [build 1/6] FROM mcr.microsoft.com/dotnet/sdk:10.0@sha256:c7445f141c04f1a6b454181bd098dcfa606c61ba0bd213d0a702489e5b  0.0s
 => => resolve mcr.microsoft.com/dotnet/sdk:10.0@sha256:c7445f141c04f1a6b454181bd098dcfa606c61ba0bd213d0a702489e5bd4cd71  0.0s
 => [stage-1 1/3] FROM mcr.microsoft.com/dotnet/runtime-deps:10.0@sha256:69ee6d1a1b7a92cc82a71001342554d4611a509d880114b  0.0s
 => => resolve mcr.microsoft.com/dotnet/runtime-deps:10.0@sha256:69ee6d1a1b7a92cc82a71001342554d4611a509d880114b2d5653ff  0.0s
 => [internal] load build context                                                                                         0.0s
 => => transferring context: 19.52kB                                                                                      0.0s
 => CACHED [build 2/6] WORKDIR /source                                                                                    0.0s
 => CACHED [build 3/6] COPY --link *.csproj .                                                                             0.0s
 => [build 4/6] RUN dotnet restore -a amd64 --os linux                                                                    4.6s
 => [build 5/6] COPY --link . .                                                                                           0.2s
 => => merging                                                                                                            0.1s
 => ERROR [build 6/6] RUN dotnet publish -a amd64 -o /app --no-restore                                                    0.5s
------                                                                                                                         
 > [build 6/6] RUN dotnet publish -a amd64 -o /app --no-restore:                                                               
0.457 /usr/share/dotnet/sdk/10.0.100/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(266,5): error NETSDK1064: Package Microsoft.NET.ILLink.Tasks, version 10.0.0 was not found. It might have been deleted since NuGet restore. Otherwise, NuGet restore might have only partially completed, which might have been due to maximum path length restrictions. [/source/globalapp.csproj]
------
Dockerfile:13
--------------------
  11 |     # Copy source code and publish app
  12 |     COPY --link . .
  13 | >>> RUN dotnet publish -a $TARGETARCH -o /app --no-restore
  14 |     
  15 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c dotnet publish -a $TARGETARCH -o /app --no-restore" did not complete successfully: exit code: 1

[richlander]

What I wrote above is wrong. There was some local weirdness on my machine. Adding RuntimeIdentifiers resolves the issue, in terms of the behavior.

That leaves us with two things:

• Documenting that RuntimeIdenifiers in the project file is the one true way
• Aligning the SelfContained and PublishSelfContained.

[baronfel]

I do agree that RuntimeIdentifiers in the project file is the best way, generally - but I'd still like to make the CLI-based approaches work too!

[lbussell]

@richlander I think this is the same as #36334