[net11.0] Support multiple R2R images for ios/tvos apps#25077
[net11.0] Support multiple R2R images for ios/tvos apps#25077rolfbjarne wants to merge 3 commits intonet11.0from
Conversation
Instead of linking all R2R .o files into a single large dylib/framework, create a separate framework (or dylib) for each R2R .o file. This way, when only one R2R input changes, only that module's framework needs to be relinked — significantly improving incremental build times. Each R2R .o file exports an RTR_HEADER symbol. To avoid collisions when multiple modules are loaded, each module's dylib uses the linker flags -Wl,-alias,_RTR_HEADER,_RTR_HEADER_<module> and -Wl,-unexported_symbol,_RTR_HEADER to export a uniquely-named alias. A new MSBuild task (GenerateR2RModuleRegistration) generates a native registration file (r2r_modules.mm) that maps module names to their header pointers. The file is compiled into the main executable and uses __attribute__((constructor)) to register the modules before main(). The runtime's xamarin_get_native_code_data callback now iterates the module table to find the correct R2R header for each owner_composite_name, with a fallback to the single xamarin_rtr_header for backward compat. Changes: - runtime/xamarin/main.h: Add struct xamarin_r2r_module and externs - runtime/runtime.m: Multi-module lookup in get_native_code_data - tools/common/Target.cs: Remove single RTR_HEADER from generated main.mm - msbuild/.../GenerateR2RModuleRegistration.cs: New task - dotnet/targets/Microsoft.Sdk.R2R.targets: Per-module framework/dylib creation with symbol renaming - dotnet/targets/Xamarin.Shared.Sdk.targets: Handle multiple R2R frameworks in post-processing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR changes the ReadyToRun (R2R) packaging/linking pipeline so iOS/tvOS (framework) and macOS/Mac Catalyst (dylib) builds produce one native image per R2R object file, enabling faster incremental relinks, and adds runtime/MSBuild plumbing to register and resolve per-module R2R headers safely.
Changes:
- Introduce a per-module R2R registration table (
xamarin_r2r_modules) and look it up at runtime when CoreCLR requests native code data. - Add an MSBuild task to generate a native registration source file (
r2r_modules.mm) and update targets to create per-module frameworks/dylibs withRTR_HEADERsymbol aliasing. - Update post-processing collection to handle multiple R2R frameworks.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/common/Target.cs | Removes legacy single-module RTR_HEADER extern/assignment from generated main. |
| runtime/xamarin/main.h | Adds xamarin_r2r_module struct and externs for module table + count. |
| runtime/runtime.m | Implements per-module lookup for xamarin_get_native_code_data with a single-module fallback path. |
| msbuild/Xamarin.MacDev.Tasks/Tasks/GenerateR2RModuleRegistration.cs | New MSBuild task to generate r2r_modules.mm registration source. |
| dotnet/targets/Microsoft.Sdk.R2R.targets | Updates R2R targets to create one framework/dylib per module and generate registration file. |
| dotnet/targets/Xamarin.Shared.Sdk.targets | Updates post-processing item collection to include per-module R2R frameworks. |
| // Multi-module: look up by owner_composite_name | ||
| if (xamarin_r2r_modules != NULL && xamarin_r2r_module_count > 0) { | ||
| for (int i = 0; i < xamarin_r2r_module_count; i++) { | ||
| if (strcmp (xamarin_r2r_modules [i].name, context->owner_composite_name) == 0) { | ||
| r2r_header = xamarin_r2r_modules [i].header; | ||
| break; | ||
| } | ||
| } | ||
| if (r2r_header == NULL) | ||
| return false; | ||
| } else { | ||
| // Single-module fallback for backward compatibility | ||
| r2r_header = xamarin_rtr_header; | ||
| } |
There was a problem hiding this comment.
When xamarin_r2r_modules is non-null, this returns false if no entry matches context->owner_composite_name, instead of falling back to the single-module xamarin_rtr_header as described in the PR. This makes the lookup brittle (any mismatch between owner_composite_name and the generated ModuleName causes R2R to be disabled) and also eliminates the intended backward-compat behavior. Consider falling back to xamarin_rtr_header (when non-NULL) if no module match is found, and/or emitting a diagnostic/assertion to aid troubleshooting.
| @@ -440,8 +435,6 @@ void GenerateMainImpl (StringWriter sw, Abi abi) | |||
| sw.WriteLine ("\txamarin_runtime_configuration_name = {0};", string.IsNullOrEmpty (app.RuntimeConfigurationFile) ? "NULL" : $"\"{app.RuntimeConfigurationFile}\""); | |||
| if (app.Registrar == RegistrarMode.ManagedStatic) | |||
| sw.WriteLine ("\txamarin_set_is_managed_static_registrar (true);"); | |||
There was a problem hiding this comment.
xamarin_rtr_header no longer appears to be assigned anywhere (Target.cs removed the generated xamarin_rtr_header = &RTR_HEADER;, and runtime/runtime.m only declares it). Since runtime/runtime.m still has a single-module fallback path that uses xamarin_rtr_header, this change can break any scenario where the module table isn't populated (or when building older single-module R2R outputs). Either keep setting xamarin_rtr_header for the single-module case, or ensure the generated registration file also populates xamarin_rtr_header appropriately for backward compatibility.
| sw.WriteLine ("\txamarin_set_is_managed_static_registrar (true);"); | |
| sw.WriteLine ("\txamarin_set_is_managed_static_registrar (true);"); | |
| if (app.XamarinRuntime == XamarinRuntime.NativeAOT) | |
| sw.WriteLine ("\txamarin_rtr_header = &RTR_HEADER;"); |
| <!-- Create frameworks for each R2R module (iOS/tvOS) --> | ||
| <Target Name="_CreateR2RModuleFrameworks" | ||
| Condition="'$(CreateR2RFramework)' == 'true'" | ||
| Outputs="%(_R2RModule.Identity)" | ||
| > | ||
|
|
||
| <!-- Create framework directory structure --> | ||
| <MakeDir | ||
| SessionId="$(BuildSessionId)" | ||
| Directories="%(_R2RModule.FrameworkPath)" |
There was a problem hiding this comment.
The target-level incremental build metadata looks incorrect: _CreateR2RModuleFrameworks declares Outputs="%(_R2RModule.Identity)" but has no Inputs, and the output value is the input item identity (the .r2r.dylib item), not an actual produced file (like %(FrameworkOutput) / %(InfoPlistPath)). This can cause the target to be skipped incorrectly on subsequent builds even when the underlying .o changes, and it also risks missing _FrameworkNativeReference item creation in builds where the target is skipped. Define proper per-module Inputs (e.g. %(ObjectFile) plus any properties that affect link flags) and Outputs (the framework binary / Info.plist), and ensure the items needed by downstream targets are created even when no relink is needed (move item creation to _PrepareR2RModules or a separate always-run target).
| <!-- Create dylibs for each R2R module (macOS/Mac Catalyst) --> | ||
| <Target Name="_CreateR2RModuleDylibs" | ||
| Condition="'$(CreateR2RDylib)' == 'true'" | ||
| Outputs="%(_R2RModule.Identity)" | ||
| > | ||
|
|
||
| <MakeDir | ||
| SessionId="$(BuildSessionId)" | ||
| Directories="$(_R2RFrameworkPath)" | ||
| /> | ||
|
|
||
| <LinkNativeCode | ||
| SessionId="$(BuildSessionId)" | ||
| ObjectFiles="@(_R2RFrameworkInputs)" | ||
| OutputFile="$(_R2RFrameworkOutput)" | ||
| LinkerFlags="@(_R2RFrameworkLinkerFlags)" | ||
|
|
||
| MinimumOSVersion="$(_MinimumOSVersion)" | ||
| SdkDevPath="$(_SdkDevPath)" | ||
| SdkIsSimulator="$(_SdkIsSimulator)" | ||
| SdkRoot="$(_SdkRoot)" | ||
| TargetArchitectures="$(TargetArchitectures)" | ||
| TargetFrameworkMoniker="$(_ComputedTargetFrameworkMoniker)" | ||
| /> | ||
| </Target> | ||
| <MakeDir | ||
| SessionId="$(BuildSessionId)" | ||
| Directories="$(_R2RModuleIntermediateOutputPath)" |
There was a problem hiding this comment.
Same incremental-build concern for _CreateR2RModuleDylibs: Outputs="%(_R2RModule.Identity)" with no Inputs means MSBuild can incorrectly skip relinking even if %(ObjectFile) changes, and the _FileNativeReference items will not exist in builds where this target is skipped. Use proper Inputs/Outputs (e.g. %(ObjectFile) -> %(DylibOutput)) and ensure downstream link/publish item groups are populated even when the target is up-to-date.
| foreach (var module in R2RModules) { | ||
| var symbolName = module.GetMetadata ("SymbolName"); | ||
| sb.AppendLine ($"extern void* {symbolName};"); | ||
| } | ||
|
|
||
| sb.AppendLine (); | ||
| sb.AppendLine ("static struct xamarin_r2r_module r2r_module_entries [] = {"); | ||
|
|
||
| foreach (var module in R2RModules) { | ||
| var moduleName = module.GetMetadata ("ModuleName"); | ||
| var symbolName = module.GetMetadata ("SymbolName"); | ||
| sb.AppendLine ($"\t{{ \"{moduleName}\", &{symbolName} }},"); | ||
| } |
There was a problem hiding this comment.
This task writes out C/C++ code using MSBuild item metadata without validating or escaping it. If ModuleName or SymbolName metadata is missing/empty, the generated file will contain invalid declarations/initializers, and if ModuleName contains characters that need escaping in a C string literal (e.g. \ or "), the generated source will not compile. Consider validating required metadata for every item (and logging an MSBuild error when missing) and C-escaping moduleName before embedding it into the string literal.
| Files="$(_R2RFrameworkStructureStampFile)" | ||
| AlwaysCreate="True" | ||
| /> | ||
| <!-- Link the .o file into a dylib inside the framework, renaming the R2R_HEADER symbol --> |
There was a problem hiding this comment.
@jkoritzinsky Instead of renaming the R2R_HEADER symbol, IMHO it makes more sense to name it correctly in the first place, i.e. pass the desired symbol name to the R2R compiler. Would this be possible?
There was a problem hiding this comment.
There's no good mechanism to communicate a different symbol name to the runtime, so any such feature would only work with custom hosts (ie the macios scenario). I think I can introduce some optional metadata that you could add in the _TouchR2ROutputs target (or another one that runs then) to get the SDK targets to work, but this would definitely be something that only works for your scenario.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This comment has been minimized.
This comment has been minimized.
✅ [CI Build #1e817d0] Build passed (Build packages) ✅Pipeline on Agent |
✅ [PR Build #1e817d0] Build passed (Detect API changes) ✅Pipeline on Agent |
✅ API diff for current PR / commitNET (empty diffs)✅ API diff vs stableNET (empty diffs)ℹ️ Generator diffGenerator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes) Pipeline on Agent |
✅ [CI Build #1e817d0] Build passed (Build macOS tests) ✅Pipeline on Agent |
This comment has been minimized.
This comment has been minimized.
🔥 [CI Build #1e817d0] Test results 🔥Test results❌ Tests failed on VSTS: test results 3 tests crashed, 16 tests failed, 131 tests passed. Failures❌ introspection tests3 tests failed, 6 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (iOS)4 tests failed, 11 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (MacCatalyst)4 tests failed, 15 tests passed.Failed tests
Html Report (VSDrops) Download ❌ monotouch tests (macOS)🔥 Failed catastrophically on VSTS: test results - monotouch_macos (no summary found). Html Report (VSDrops) Download ❌ monotouch tests (tvOS)4 tests failed, 11 tests passed.Failed tests
Html Report (VSDrops) Download ❌ sharpie tests🔥 Failed catastrophically on VSTS: test results - sharpie (no summary found). Html Report (VSDrops) Download ❌ windows tests1 tests failed, 2 tests passed.Failed tests
Html Report (VSDrops) Download ❌ xtro tests🔥 Failed catastrophically on VSTS: test results - xtro (no summary found). Html Report (VSDrops) Download Successes✅ cecil: All 1 tests passed. Html Report (VSDrops) Download macOS tests✅ Tests on macOS Sonoma (14): All 5 tests passed. Html Report (VSDrops) Download Linux Build VerificationPipeline on Agent |
Note
This PR was generated with the help of GitHub Copilot.
Instead of linking all R2R .o files into a single large dylib/framework, create a separate framework (or dylib) for each R2R .o file. This way, when only one R2R input changes, only that module's framework needs to be relinked — significantly improving incremental build times.
Each R2R .o file exports an RTR_HEADER symbol. To avoid collisions when multiple modules are loaded, each module's dylib uses the linker flags
-Wl,-alias,_RTR_HEADER,_RTR_HEADER_<module>and-Wl,-unexported_symbol,_RTR_HEADERto export a uniquely-named alias.A new MSBuild task (GenerateR2RModuleRegistration) generates a native registration file (r2r_modules.mm) that maps module names to their header pointers. The file is compiled into the main executable and uses
__attribute__((constructor))to register the modules before main().The runtime's xamarin_get_native_code_data callback now iterates the module table to find the correct R2R header for each owner_composite_name, with a fallback to the single xamarin_rtr_header for backward compat.
Changes:
Contributes to dotnet/runtime#126194
This is a recreation of #25072 from origin (due to CI requirements).