-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Ensure that Trimming and AOT scenarios trigger downloading of necessary runtime packs #51765
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
baronfel
wants to merge
8
commits into
dotnet:main
Choose a base branch
from
baronfel:processframeworkreferences-publishprops-restore
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+643
−55
Open
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
105627d
create MSBuild-Task-specific slnf to make it easier to work on tasks …
baronfel 98a10bb
scaffold out test scenarios to demonstrate gaps
baronfel 54b4095
extract out spec for behavior of ProcessFrameworkReferences
baronfel 6948352
Update ProcessFrameworkReferences so that trimming and AOT scenarios …
baronfel dcd5bbc
Update src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs
baronfel e21abc6
Update documentation/specs/ProcessFrameworkReferences-outputs.md
baronfel 9e36ac1
Update documentation/specs/ProcessFrameworkReferences-outputs.md
baronfel eb417d3
patch gap in publish-for-specific-rid scenario where SelfContained is…
baronfel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
345 changes: 345 additions & 0 deletions
345
documentation/specs/ProcessFrameworkReferences-outputs.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,345 @@ | ||
| # ProcessFrameworkReferences Task Output Specification | ||
|
|
||
| ## Overview | ||
|
|
||
| The `ProcessFrameworkReferences` MSBuild task is responsible for resolving framework references and determining which NuGet packages need to be downloaded to support compilation and publishing scenarios. This task produces several output item groups that represent different types of packages required for the build. | ||
|
|
||
| ## Output Item Groups | ||
|
|
||
| The task produces the following output collections: | ||
|
|
||
| - **`TargetingPacks`**: Reference assemblies used during compilation | ||
| - **`RuntimePacks`**: Runtime-specific implementations of framework libraries | ||
| - **`PackagesToDownload`**: Packages that need to be restored via direct PackageDownload mechanism | ||
| - **`RuntimeFrameworks`**: Framework dependencies to be written to runtimeconfig.json | ||
| - **`ImplicitPackageReferences`**: Build-time tool packages that need to be restored via PackageReference mechanism (ILLink, Crossgen2, ILCompiler) | ||
| - **`Crossgen2Packs`**: Ready-to-Run compilation tools | ||
| - **`HostILCompilerPacks`**: Native AOT compiler tools for the host platform | ||
| - **`TargetILCompilerPacks`**: Native AOT compiler tools for the target platform | ||
| - **`UnavailableRuntimePacks`**: Framework packs not available for the specified RID | ||
|
|
||
| ## Expected Outputs by Scenario | ||
|
|
||
| ### Base Scenario: Framework-Dependent Projects | ||
|
|
||
| For all projects that reference frameworks (via `<FrameworkReference>` items): | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One entry per framework reference per target framework | ||
| - Contains the targeting pack name and version | ||
| - Used for compile-time reference resolution | ||
| - ✅ `RuntimeFrameworks`: One entry per framework reference per target framework | ||
| - Written to runtimeconfig.json to specify runtime dependencies | ||
|
|
||
| **Example:** | ||
| ```xml | ||
| <FrameworkReference Include="Microsoft.NETCore.App" /> | ||
| <FrameworkReference Include="Microsoft.AspNetCore.App" /> | ||
| ``` | ||
|
|
||
| Results in: | ||
| - `TargetingPacks`: `Microsoft.NETCore.App.Ref`, `Microsoft.AspNetCore.App.Ref` | ||
| - `RuntimeFrameworks`: `Microsoft.NETCore.App`, `Microsoft.AspNetCore.App` | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 1: Self-Contained Deployment (`SelfContained=true`, `PublishSelfContained=true`) | ||
|
|
||
| When `RuntimeIdentifier` is specified explicitly: | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One per framework reference per TFM | ||
| - ✅ `RuntimePacks`: One per framework reference per TFM for the specified RID | ||
| - Contains RID-specific runtime libraries | ||
| - Example: `Microsoft.NETCore.App.Runtime.linux-x64` | ||
| - ✅ `PackagesToDownload`: Includes both targeting packs and runtime packs | ||
| - ✅ `RuntimeFrameworks`: One per framework reference per TFM | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <SelfContained>true</SelfContained> | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| ``` | ||
|
|
||
| **Behavior:** | ||
| - Runtime packs are resolved for the primary `RuntimeIdentifier` | ||
| - RID-specific assets are included in the publish output | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 2: Self-Contained with multiple RuntimeIdentifiers (`RuntimeIdentifiers` property) | ||
|
|
||
| When multiple RIDs are specified via `RuntimeIdentifiers`: | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One per framework reference per TFM | ||
| - ✅ `RuntimePacks`: | ||
| - For the primary RID (if `RuntimeIdentifier` is set): Full runtime pack metadata | ||
| - For additional RIDs: Runtime packs are downloaded but not added to `RuntimePacks` output | ||
| - ✅ `PackagesToDownload`: Runtime packs for ALL RIDs in `RuntimeIdentifiers` | ||
| - ✅ `RuntimeFrameworks`: One per framework reference per TFM | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <SelfContained>true</SelfContained> | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| <RuntimeIdentifiers>linux-x64;linux-arm64;win-x64</RuntimeIdentifiers> | ||
| ``` | ||
|
|
||
| **Behavior:** | ||
| - All runtime packs for all RIDs are downloaded to support multi-RID publishing | ||
| - Only the primary RID's runtime packs appear in the `RuntimePacks` output for consumption | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 3: Ready-to-Run Compilation (`ReadyToRunEnabled=true`, `ReadyToRunUseCrossgen2=true`) | ||
|
|
||
| For projects using Ready-to-Run (R2R) compilation: | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One per framework reference per TFM | ||
| - ✅ `RuntimePacks`: One per framework reference per TFM for the specified RID | ||
| - ✅ `Crossgen2Packs`: RID-specific Crossgen2 compiler for the host platform | ||
| - Example: `Microsoft.NETCore.App.Crossgen2.linux-x64` | ||
| - ✅ `PackagesToDownload`: Includes targeting packs, runtime packs (if applicable), and Crossgen2 pack | ||
| - ✅ `RuntimeFrameworks`: One per framework reference per TFM | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <SelfContained>true</SelfContained> | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| <ReadyToRunEnabled>true</ReadyToRunEnabled> | ||
| <ReadyToRunUseCrossgen2>true</ReadyToRunUseCrossgen2> | ||
| ``` | ||
|
|
||
| **Behavior:** | ||
| - Crossgen2 pack is resolved based on the SDK's runtime identifier (host RID) | ||
| - Enables ahead-of-time compilation of IL to native code for faster startup | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 4: Publish with Trimming (`PublishTrimmed=true`, `RequiresILLinkPack=true`) | ||
|
|
||
| For projects that trim unused code during publish: | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One per framework reference per TFM | ||
| - ✅ `RuntimePacks`: One per framework reference per TFM for the specified RID | ||
| - **Required even when `SelfContained=false`** because trimming requires RID-specific analysis | ||
| - ✅ `ImplicitPackageReferences`: `Microsoft.NET.ILLink.Tasks` | ||
| - Contains MSBuild targets and tasks for trimming | ||
| - ✅ `PackagesToDownload`: Includes targeting packs, runtime packs, and ILLink pack | ||
| - ✅ `RuntimeFrameworks`: One per framework reference per TFM | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| <PublishTrimmed>true</PublishTrimmed> | ||
| <RequiresILLinkPack>true</RequiresILLinkPack> | ||
| <EnableTrimAnalyzer>true</EnableTrimAnalyzer> | ||
| ``` | ||
|
|
||
| **Behavior:** | ||
| - Runtime packs are required for trimming analysis even in framework-dependent deployments | ||
| - ILLink pack provides the trimming engine and MSBuild integration | ||
| - For .NET 6+, trim analyzer warnings are enabled by default | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 5: Trim and AOT Analysis Support (`IsTrimmable=true`, `EnableTrimAnalyzer=true`, `IsAotCompatible=true`, `EnableAotAnalyzer=true`) | ||
|
|
||
| For libraries that want to support trimming and AOT: | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One per framework reference per TFM | ||
| - ✅ `ImplicitPackageReferences`: `Microsoft.NET.ILLink.Tasks` | ||
| - Provides analyzers for trim and AOT compatibility | ||
| - ✅ `PackagesToDownload`: Includes targeting packs and ILLink pack | ||
| - ✅ `RuntimeFrameworks`: One per framework reference per TFM | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <IsTrimmable>true</IsTrimmable> | ||
| <EnableTrimAnalyzer>true</EnableTrimAnalyzer> | ||
| <IsAotCompatible>true</IsAotCompatible> | ||
| <EnableAotAnalyzer>true</EnableAotAnalyzer> | ||
| <RequiresILLinkPack>true</RequiresILLinkPack> | ||
| ``` | ||
|
|
||
| **Behavior:** | ||
| - No RID specified, so no runtime packs are downloaded | ||
| - ILLink pack provides compile-time analyzers for library authors | ||
| - Warnings/errors guide developers to make code trim/AOT-compatible | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 6: Native AOT Publishing (`PublishAot=true`) | ||
|
|
||
| For projects using Native AOT compilation: | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks`: One per framework reference per TFM | ||
| - ✅ `RuntimePacks`: One per framework reference per TFM for the specified RID | ||
| - **Required even when `SelfContained=false`** because AOT requires RID-specific compilation | ||
| - ✅ `HostILCompilerPacks`: ILCompiler pack for the host/SDK RID | ||
| - Example: `runtime.linux-x64.Microsoft.DotNet.ILCompiler` | ||
| - Used to run the AOT compiler | ||
| - ✅ `TargetILCompilerPacks`: ILCompiler pack for the target RID (if different from host) | ||
| - Example: `runtime.linux-arm64.Microsoft.DotNet.ILCompiler` | ||
| - Contains target-specific compilation assets | ||
| - ✅ `ImplicitPackageReferences`: `Microsoft.NET.ILLink.Tasks` | ||
| - ILLink is used as part of the AOT compilation pipeline | ||
| - ✅ `PackagesToDownload`: Includes targeting packs, runtime packs, and ILCompiler packs | ||
| - ✅ `RuntimeFrameworks`: One per framework reference per TFM | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <RuntimeIdentifier>linux-arm64</RuntimeIdentifier> | ||
| <PublishAot>true</PublishAot> | ||
| <RequiresILLinkPack>true</RequiresILLinkPack> | ||
| ``` | ||
|
|
||
| **Behavior:** | ||
| - Runtime packs are required for AOT compilation regardless of `SelfContained` setting | ||
| - Host ILCompiler pack must match the SDK's RID to run the compiler | ||
| - Target ILCompiler pack must match the `RuntimeIdentifier` for cross-compilation scenarios | ||
| - Native AOT produces a single native executable with no runtime dependencies | ||
|
|
||
| --- | ||
|
|
||
| ### Scenario 7: Combined Scenarios | ||
|
|
||
| Projects can combine multiple publish features: | ||
|
|
||
| #### Self-Contained + Ready-to-Run + Trimming | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <SelfContained>true</SelfContained> | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| <ReadyToRunEnabled>true</ReadyToRunEnabled> | ||
| <ReadyToRunUseCrossgen2>true</ReadyToRunUseCrossgen2> | ||
| <PublishTrimmed>true</PublishTrimmed> | ||
| <RequiresILLinkPack>true</RequiresILLinkPack> | ||
| ``` | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks` | ||
| - ✅ `RuntimePacks` (for the specified RID) | ||
| - ✅ `Crossgen2Packs` (for R2R compilation) | ||
| - ✅ `ImplicitPackageReferences` (ILLink for trimming) | ||
| - ✅ `PackagesToDownload` (all of the above) | ||
|
|
||
| #### Multi-RID + Ready-to-Run + Trimming | ||
|
|
||
| **Key Properties:** | ||
| ```xml | ||
| <SelfContained>true</SelfContained> | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| <RuntimeIdentifiers>linux-x64;linux-arm64;win-x64</RuntimeIdentifiers> | ||
| <ReadyToRunEnabled>true</ReadyToRunEnabled> | ||
| <ReadyToRunUseCrossgen2>true</ReadyToRunUseCrossgen2> | ||
| <PublishTrimmed>true</PublishTrimmed> | ||
| <RequiresILLinkPack>true</RequiresILLinkPack> | ||
| ``` | ||
|
|
||
| **Expected Outputs:** | ||
| - ✅ `TargetingPacks` | ||
| - ✅ `RuntimePacks` (for the primary RID only) | ||
| - ✅ `Crossgen2Packs` (for the host RID) | ||
| - ✅ `ImplicitPackageReferences` (ILLink for trimming) | ||
| - ✅ `PackagesToDownload` (runtime packs for ALL RIDs, plus Crossgen2 and ILLink) | ||
|
|
||
| --- | ||
|
|
||
| ## Current Issues and Expected Fixes | ||
|
|
||
| ### Issue #51667: Runtime Packs Not Resolved for PublishTrimmed/PublishAot Without SelfContained | ||
|
|
||
| **Problem:** | ||
| Currently, when `PublishTrimmed=true` or `PublishAot=true` is set without `SelfContained=true`, the task does not resolve runtime packs. This causes publish failures because trimming and AOT require runtime-specific assets. | ||
|
|
||
| **Current Behavior:** | ||
| ```csharp | ||
| var runtimeRequiredByDeployment | ||
| = (SelfContained || ReadyToRunEnabled) && | ||
| !string.IsNullOrEmpty(EffectiveRuntimeIdentifier) && | ||
| !string.IsNullOrEmpty(selectedRuntimePack?.RuntimePackNamePatterns); | ||
| ``` | ||
|
|
||
| **Expected Behavior:** | ||
| Runtime packs should be resolved when: | ||
| - `SelfContained=true`, OR | ||
| - `ReadyToRunEnabled=true`, OR | ||
| - `PublishTrimmed=true`, OR | ||
| - `PublishAot=true` | ||
|
|
||
| **AND** a `RuntimeIdentifier` is specified. | ||
|
|
||
| **Proposed Fix:** | ||
| ```csharp | ||
| var runtimeRequiredByDeployment | ||
| = (SelfContained || ReadyToRunEnabled || PublishAot || RequiresILLinkPack) && | ||
| !string.IsNullOrEmpty(EffectiveRuntimeIdentifier) && | ||
| !string.IsNullOrEmpty(selectedRuntimePack?.RuntimePackNamePatterns); | ||
| --- | ||
|
|
||
| ## Implementation Details | ||
|
|
||
| ### Key Decision Points | ||
|
|
||
| The task uses the following logic to determine which packs to include: | ||
|
|
||
| 1. **Targeting Packs**: Always included for all framework references matching the target framework | ||
| 2. **Runtime Packs**: Included when `runtimeRequiredByDeployment` is true OR `RuntimePackAlwaysCopyLocal` is set | ||
| 3. **Tool Packs**: Included based on specific properties: | ||
| - Crossgen2: When `ReadyToRunEnabled && ReadyToRunUseCrossgen2` | ||
| - ILCompiler: When `PublishAot` | ||
| - ILLink: When `RequiresILLinkPack` | ||
|
|
||
| ### RID Resolution | ||
|
|
||
| The task uses the runtime graph (`RuntimeGraphPath`) to find the best matching RID from the available runtime packs: | ||
|
|
||
| 1. For the primary `RuntimeIdentifier`, full runtime pack metadata is generated | ||
| 2. For additional `RuntimeIdentifiers`, only download entries are created | ||
| 3. Portable RIDs are preferred over non-portable RIDs when appropriate for tool packs | ||
|
|
||
| ### Version Selection | ||
|
|
||
| Runtime framework versions follow this precedence: | ||
| 1. `RuntimeFrameworkVersion` metadata on `FrameworkReference` item | ||
| 2. `RuntimeFrameworkVersion` MSBuild property | ||
| 3. `LatestRuntimeFrameworkVersion` (if `TargetLatestRuntimePatch=true`) | ||
| 4. `DefaultRuntimeFrameworkVersion` (if `TargetLatestRuntimePatch=false`) | ||
|
|
||
| --- | ||
|
|
||
| ## Testing Scenarios | ||
|
|
||
| The following test scenarios validate the expected outputs: | ||
|
|
||
| 1. ✅ **Self-contained deployment** resolves runtime packs for the specified RID | ||
| 2. ❌ **PublishTrimmed without SelfContained** should resolve runtime packs (currently fails) | ||
| 3. ❌ **PublishAot without SelfContained** should resolve runtime packs (currently fails) | ||
| 4. ✅ **Multiple RuntimeIdentifiers** downloads runtime packs for all RIDs | ||
| 5. ✅ **Ready-to-Run** includes Crossgen2 packs | ||
| 6. ✅ **Combined scenarios** include all necessary packs | ||
|
|
||
| Tests are located in: `src/Tasks/Microsoft.NET.Build.Tasks.UnitTests/ProcessFrameworkReferencesTests.cs` | ||
|
|
||
| --- | ||
|
|
||
| ## Related Documentation | ||
|
|
||
| - [Runtime Identifier Catalog](https://learn.microsoft.com/dotnet/core/rid-catalog) | ||
| - [Framework-dependent vs Self-contained deployment](https://learn.microsoft.com/dotnet/core/deploying/) | ||
| - [Trim self-contained deployments](https://learn.microsoft.com/dotnet/core/deploying/trimming/trim-self-contained) | ||
| - [Native AOT deployment](https://learn.microsoft.com/dotnet/core/deploying/native-aot/) | ||
| - [ReadyToRun compilation](https://learn.microsoft.com/dotnet/core/deploying/ready-to-run) | ||
|
|
||
| --- | ||
|
|
||
| ## Revision History | ||
|
|
||
| - **2025-01-14**: Initial specification documenting expected outputs and identifying issue #51667 | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a minimum runtime pack version that the current .NET SDK is compatible with? Does the current .NET SDK still support publishing for .NET 5 in all flavors?
UseCrossgen2property was a thing in .NET 5 where crossgen2 was in preview and setting this property to true was used to opt-in into the preview. It has no purpose (it is always true) in .NET 6+.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The SDK does still support publishing net5 apps - though you get a warning that that runtime is out of support. The property is mostly mentioned here for completeness given that it does play into the runtime pack selection in the processframeworkreferences task today - I didn't want a reader to read this and then see an entirely new concept in the implementation and wonder if we missed something critical.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you have a deprecation schedule for when .NET SDK is going to drop support for runtimes that are out of support for a long time?
I do not think it is a good use of resources to carry this baggage around, and the support is likely broken in number of ways anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why currently have no timeline for this. The general approach has been to allow everything, but only explicitly test 2 LTSs back. So for example net 6 is the lower bound on a lot of our tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why currently have no timeline for this. The general approach has been to allow everything, but only explicitly test 2 LTSs back. So for example net 6 is the lower bound on a lot of our tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What's your confidence that the extra code required to allow everything actually works?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Surprisingly high - there are a pretty high number of SDK-repo scenario tests that cover .NET 5 at least. I do get what you're saying though - there is a cost to carrying this around. There's a tension that we face on the tooling level because we want to enable folks to use the most recent tooling regardless of the TFM they're actually targeting. We feel that this is the best strategy to ensure that folks are getting the improved tooling experiences, perf improvements in the tooling, etc. The downside to this is that the tooling then needs to keep all of this downlevel logic around, and the tests contribute to overall SDK PR times, etc etc.
My gut feel is that this cost isn't incredibly high right now, but I'd be in favor of pushing tests that target these out of support or more-niche scenarios to a kind of 'outerloop' test run that doesn't block or impact normal PR flow.