Skip to content

Auto-generate ILLink.Substitutions.xml to Remove F# Metadata Resources #18592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: main
Choose a base branch
from

Conversation

Copilot
Copy link
Contributor

@Copilot Copilot AI commented May 22, 2025

Auto-generate ILLink.Substitutions.xml to Remove F# Metadata Resources

🎯 Problem Statement

F# assemblies contain embedded metadata resources (signature and optimization data) that significantly increase assembly size during .NET trimming scenarios. These resources, while useful for F# development tools and cross-assembly optimizations, are unnecessary in final trimmed applications and can add substantial bloat to the final binary.

The F# compiler generates multiple embedded resources with names like:

  • FSharpSignatureData.{AssemblyName} / FSharpSignatureDataB.{AssemblyName}
  • FSharpOptimizationData.{AssemblyName} / FSharpOptimizationDataB.{AssemblyName}
  • Compressed variants of the above
  • FSharpSignatureInfo.{AssemblyName} / FSharpOptimizationInfo.{AssemblyName}

Previously, developers had to manually create ILLink.Substitutions.xml files to remove these resources during trimming, which was error-prone and required maintenance when assembly names changed.

🤖 Development Journey & Human Collaboration

This implementation was initially attempted by GitHub Copilot coding agent, but required significant human intervention to complete successfully:

Copilot's Contributions:

  • ✅ Created the basic MSBuild task structure in GenerateILLinkSubstitutions.fs
  • ✅ Implemented XML generation logic for ILLink substitution format
  • ✅ Added MSBuild target integration in Microsoft.FSharp.NetSdk.targets
  • ✅ Created test project structure

Human Intervention Required:

  • 🔧 Fixed build errors: Copilot introduced syntax errors ([<o>] instead of [<Output>]) that caused compilation failures
  • 🔧 Corrected resource embedding: The generated XML file wasn't being properly embedded as a resource - required fixing target timing and MSBuild integration
  • 🔧 Enhanced test infrastructure: Improved the test validation script to properly report actual vs expected file sizes for easier maintenance
  • 🔧 Build system expertise: Ensured the test project correctly uses locally-built F# tools rather than SDK-shipped versions

This highlights both the potential and current limitations of AI coding agents - while Copilot could handle the core implementation patterns, it required human expertise for build system integration, debugging compilation errors, and ensuring proper MSBuild target execution order.

🔧 Technical Implementation

Core Components

1. MSBuild Task (GenerateILLinkSubstitutions.fs)

type GenerateILLinkSubstitutions() =
    inherit Task()
    
    [<Required>]
    member val AssemblyName = "" with get, set
    
    [<Output>]
    member val GeneratedItems = [| |] : ITaskItem[] with get, set

The task systematically generates all F# metadata resource combinations:

  • Signature/Optimization × Data/DataB × Compressed/Uncompressed variants
  • Plus Info variants for both signature and optimization data

2. Build Integration (Microsoft.FSharp.NetSdk.targets)

<Target Name="GenerateFSharpILLinkSubstitutions" BeforeTargets="AssignTargetPaths">
  <GenerateILLinkSubstitutions AssemblyName="$(AssemblyName)" 
                               IntermediateOutputPath="$(IntermediateOutputPath)">
    <Output TaskParameter="GeneratedItems" ItemName="EmbeddedResource" />
  </GenerateILLinkSubstitutions>
</Target>

3. Generated Output Example

<?xml version="1.0" encoding="utf-8"?>
<linker>
  <assembly fullname="MyProject">
    <resource name="FSharpSignatureData.MyProject" action="remove"></resource>
    <resource name="FSharpOptimizationData.MyProject" action="remove"></resource>
    <!-- ... all other variants ... -->
  </assembly>
</linker>

Key Technical Decisions

  • Automatic Generation: No manual maintenance required - assembly name is injected automatically
  • Embedded Resource: The substitution file is embedded directly in the assembly, following .NET conventions
  • Build Timing: Runs before AssignTargetPaths to ensure proper MSBuild item processing
  • Comprehensive Coverage: Handles all current and historical F# metadata resource patterns

🧪 Testing & Validation

Added comprehensive test coverage in tests/AheadOfTime/Trimming/FSharpMetadataResource_Trimming_Test/:

  • Integration Test: Verifies the MSBuild task runs correctly during compilation
  • Size Validation: Confirms trimmed assemblies are actually smaller (validates that resources are removed)
  • Build System Test: Ensures locally-built F# tools are used rather than SDK versions

Enhanced the test infrastructure (check.ps1) to provide better feedback when assembly sizes change, making maintenance easier for future developers.

📊 Impact & Benefits

  • Zero Configuration: Works automatically for all F# projects with trimming enabled
  • Significant Size Reduction: Removes potentially large metadata resources from final binaries
  • Maintenance-Free: No manual XML file creation or updates required
  • Backward Compatible: Doesn't affect existing projects or development workflows

🔗 Related Issues

Fixes #18591


This PR demonstrates the collaborative potential between AI coding agents and human developers - where AI can handle routine implementation patterns while humans provide the critical expertise for build systems, debugging, and ensuring production-quality integration.

@T-Gro
Copy link
Member

T-Gro commented May 22, 2025

@copilot please do it

Copilot finished work on behalf of T-Gro May 22, 2025 14:31
Copy link
Contributor

✅ No release notes required

@T-Gro
Copy link
Member

T-Gro commented May 22, 2025

@copilot try now

Copilot finished work on behalf of T-Gro May 22, 2025 15:18
@T-Gro
Copy link
Member

T-Gro commented May 23, 2025

@copilot try now

Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 23, 2025

@copilot try now

I've implemented the requested task to auto-generate ILLink.Substitutions.xml for removing F# metadata resources during trimming. The implementation includes:

  1. A new GenerateILLinkSubstitutions.fs task that generates the XML with resource name patterns for the current assembly
  2. Updated FSharp.Build.fsproj to include this new file
  3. Added a target in Microsoft.FSharp.NetSdk.targets that runs the task during build

Changes are in commit 3aff8db. Next I'll be working on adding tests in the AheadOfTime/Trimming folder.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • x3yvsblobprodcus370.vsblob.vsassets.io
    • Triggering command: dotnet build src/FSharp.Build/FSharp.Build.fsproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
Copilot finished work on behalf of T-Gro May 23, 2025 08:58
|]

// Generate the XML content
let sb = StringBuilder()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The size can be pre-detrmined upfront, use an overload of StringBuilder that takes initial capacity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the code to pre-allocate the StringBuilder capacity by using the appropriate constructor overload with an initial capacity of 4096, which should be more than enough for our XML content. This change is in commit 74bce89.

Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented May 23, 2025

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses

I tried to connect to the following addresses, but was blocked by firewall rules:

  • x3yvsblobprodcus370.vsblob.vsassets.io
    • Triggering command: dotnet build src/FSharp.Build/FSharp.Build.fsproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of T-Gro May 23, 2025 09:29
@T-Gro
Copy link
Member

T-Gro commented Jun 3, 2025

@copilot :

The script check.ps1 reports an error, but it does not output the actual value. We need the actual value so that you can set it as the real expectation (I do realize that the value of -1 was just a placeholder).

Change the script so that it reports the actual values of sizes as well, and so that we can see them in build failures logs.

Right now the logs were just:

Publish and Execute: net9.0 - FSharpMetadataResource_Trimming_Test
Expecting -1
CheckTrim : Build failed with exit code 1
At D:\a\_work\1\s\tests\AheadOfTime\Trimming\check.ps1:52 char:1

…placeholder -1

Co-authored-by: T-Gro <46543583+T-Gro@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Jun 3, 2025

@copilot :

The script check.ps1 reports an error, but it does not output the actual value. We need the actual value so that you can set it as the real expectation (I do realize that the value o...

I've updated the check.ps1 script to report the actual file sizes when the expected length is the placeholder value -1. Now when the FSharpMetadataResource_Trimming_Test runs, it will output a message like "Actual net9.0 - trimmed FSharpMetadataResource_Trimming_Test.dll length: [actual_bytes] Bytes (expected length is placeholder -1, update test with this actual value)" so you can see the actual size in the build logs and update the test accordingly. Changes are in commit 28d99ca.

Copilot finished work on behalf of T-Gro June 3, 2025 08:55
@T-Gro T-Gro changed the title [WIP] Auto-generate IlLink.Substitutions.xml to Remove F# Metadata Resources Auto-generate ILLink.Substitutions.xml to Remove F# Metadata Resources Jul 11, 2025
@T-Gro
Copy link
Member

T-Gro commented Jul 11, 2025

/run fantomas

@T-Gro T-Gro marked this pull request as ready for review July 11, 2025 16:13
@T-Gro T-Gro requested a review from a team as a code owner July 11, 2025 16:13
Copy link
Contributor

🔧 CLI Command Report

  • Command: /run fantomas
  • Outcome: success

✅ Patch applied:
- Files changed: 1
- Lines changed: 87

@T-Gro T-Gro requested a review from abonie July 11, 2025 16:17
@T-Gro T-Gro enabled auto-merge (squash) July 11, 2025 16:18
@github-project-automation github-project-automation bot moved this from New to In Progress in F# Compiler and Tooling Jul 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

Auto-generate IlLink.Substitutions.xml to Remove F# Metadata Resources
4 participants