Add a managed static registrar. Fixes #17324. #18268

rolfbjarne · 2023-05-11T09:22:56Z

Go grab a ☕️ or a 🫖 first, this is a rather big PR 😉

Add a new version of the static registrar (called the managed static
registrar), which most notably doesn't use metadata tokens (because NativeAOT
doesn't support metadata tokens). In addition, the new registrar also takes
advantage of new features in both C# and the runtime, in order to be more
performant.

I won't go into detail about everything here, because it would be rather long,
but I've added documentation for the new registrar (the first commit, so start
reviewing there).

Fixes #17324.

This PR is best reviewed commit-by-commit. Note that each commit has a
descriptive commit message that describes that change.

There are no functional changes here, just code simplification.

…are warnings.

…iguration.Report method. Since LinkerConfiguration.Report uses the trimmer's API to report warnings and errors.

…turn exceptions. Without having to throw them.

…monotouch-test.

…r on. There are no functional changes here, just refactoring to make code easier to re-use.

This new mode is still considered a 'Static' registrar mode, it's just a variation of it.

This way we can ask the linker to inline the Runtime.IsManagedStaticRegistrar property, and remove any dead code paths.

…ManagedRegistrarLookupTablesStep.

…current registrar mode is 'ManagedStatic'.

…egistrar The managed static registrar will add code to the processed assemblies, which means it must run before the trimmer sweeps unused code. This means we have to split the current registrar logic in two: 1. First we process all the assemblies. 2. Then we write out the results. When not using the managed static registrar, these two steps happens right after oneanother (like they do now), while when using the managed static registrar, the processing is done before the trimmer sweeps (where we'll also generate all the new IL code), and then the generated native code will be done at the end of the build process (like for the old-school static registrar).

…ed static registrar step can access them. There are no functional changes here, just refactoring to make code easier to re-use.

Comparing -1 to 0xFFFFFFFF doesn't get the right result otherwise.

…ve-C method signatures. This is required when generating a cast of a function pointer to an Objective-C method signature (which the managed static registrar does).

…in generated method bodies.

…icable code paths.

…ime.GetNSObject We can't call the generic Runtime.GetNSObject<T> method when we don't know the final type of the returned object, which happens for open types. However, we must always call the generic Runtime.GetNSObject<T> when we know the final type, so that we have the correct object type on the evaluation stack. Fix this by properly detecting open types and only calling Runtime.GetNSObject in that case.

…voke. Keep better track of whether we're calling MethodBase.Invoke to call the target method, which allows us to detect certain error scenarios (and show a corresponding error).

… 'object' reference is left on the stack. Keep track of when we called MethodBase.Invoke and an 'object' reference is left on the stack, so that we can cast to the appropiate type later on when we know how the value is converted to the corresponding native type. This fixes a few cases of the managed static registrar creating unverifiable IL.

…able

…ypes. We can't create an instance of a generic type in the UnmanagedCallersOnly trampoline, so simplify the code to not even try, and instead just throw the final exception.

github-actions · 2023-05-18T10:12:08Z

⚠️ Your code has been reformatted. ⚠️

If this is not desired, add the actions-disable-autoformat label, and revert the reformatting commit.

If files unrelated to your change were modified, try reverting the reformatting commit + merging with the target branch (and push those changes).

vs-mobiletools-engineering-service2 · 2023-05-18T14:59:18Z

📚 [PR Build] Artifacts 📚

Packages generated

View packages

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2023-05-18T14:59:49Z

✅ API diff for current PR / commit

Legacy Xamarin (No breaking changes)

~~iOS~~ (no change detected)
~~tvOS~~ (no change detected)
~~watchOS~~ (no change detected)
~~macOS~~ (no change detected)

.NET (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
MacCatalyst: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)

✅ API diff vs stable

Legacy Xamarin (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
watchOS: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)

.NET (No breaking changes)

iOS: vsdrops gist (No breaking changes)
tvOS: vsdrops gist (No breaking changes)
MacCatalyst: vsdrops gist (No breaking changes)
macOS: vsdrops gist (No breaking changes)
Microsoft.iOS vs Microsoft.MacCatalyst: vsdrops gist

Legacy Xamarin (stable) vs .NET

iOS: vsdrops gist
tvOS: vsdrops gist
macOS: vsdrops gist

ℹ️ Generator diff

Generator Diff: vsdrops (html) vsdrops (raw diff) gist (raw diff) - Please review changes)

Pipeline on Agent
Hash: 65590ba8c0d82b49de7aa7312720fd7b7c42f09d [PR build]

vs-mobiletools-engineering-service2 · 2023-05-18T15:09:26Z

💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻

✅ All tests on macOS M1 - Mac Big Sur (11.5) passed.

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2023-05-18T15:12:22Z

💻 [PR Build] Tests on macOS M1 - Mac Ventura (13.0) passed 💻

✅ All tests on macOS M1 - Mac Ventura (13.0) passed.

Pipeline on Agent
Hash: [PR build]

vs-mobiletools-engineering-service2 · 2023-05-18T15:24:38Z

💻 [CI Build] Windows Integration Tests passed 💻

✅ All Windows Integration Tests passed.

Pipeline on Agent
Hash: 65590ba8c0d82b49de7aa7312720fd7b7c42f09d [PR build]

vs-mobiletools-engineering-service2 · 2023-05-18T18:06:15Z

🚀 [CI Build] Test results 🚀

Test results

✅ All tests passed on VSTS: simulator tests.

🎉 All 235 tests passed 🎉

Tests counts

✅ bcl: All 69 tests passed. Html Report (VSDrops) Download
✅ cecil: All 1 tests passed. Html Report (VSDrops) Download
✅ dotnettests: All 1 tests passed. Html Report (VSDrops) Download
✅ fsharp: All 7 tests passed. Html Report (VSDrops) Download
✅ framework: All 8 tests passed. Html Report (VSDrops) Download
✅ generator: All 2 tests passed. Html Report (VSDrops) Download
✅ interdependent_binding_projects: All 7 tests passed. Html Report (VSDrops) Download
✅ install_source: All 1 tests passed. Html Report (VSDrops) Download
✅ introspection: All 8 tests passed. Html Report (VSDrops) Download
✅ linker: All 65 tests passed. Html Report (VSDrops) Download
✅ mac_binding_project: All 1 tests passed. Html Report (VSDrops) Download
✅ mmp: All 2 tests passed. Html Report (VSDrops) Download
✅ mononative: All 12 tests passed. Html Report (VSDrops) Download
✅ monotouch: All 35 tests passed. Html Report (VSDrops) Download
✅ msbuild: All 2 tests passed. Html Report (VSDrops) Download
✅ mtouch: All 1 tests passed. Html Report (VSDrops) Download
✅ xammac: All 3 tests passed. Html Report (VSDrops) Download
✅ xcframework: All 8 tests passed. Html Report (VSDrops) Download
✅ xtro: All 2 tests passed. Html Report (VSDrops) Download

Pipeline on Agent
Hash: 65590ba8c0d82b49de7aa7312720fd7b7c42f09d [PR build]

tj-devel709

Things LGTM! Thank you for taking the time to split things by commit, helped make things more clear as reviewing!

rolfbjarne requested a review from mandel-macaque as a code owner May 11, 2023 09:22

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

rolfbjarne added 4 commits May 11, 2023 12:16

[docs] Document the managed static registrar.

527cba2

[dotnet-linker] Reduce a bit of code duplication.

7c7637e

There are no functional changes here, just code simplification.

[dotnet-linker] Don't fail trimming if all the exceptions we collect …

9e11297

…are warnings.

[dotnet-linker] Unify exception handling to go through the LinkerConf…

eb01507

…iguration.Report method. Since LinkerConfiguration.Report uses the trimmer's API to report warnings and errors.

This comment has been minimized.

Sign in to view

rolfbjarne and others added 9 commits May 11, 2023 12:21

[dotnet-linker] Add a way for ConfigurationAwareStep subclasses to re…

4351674

…turn exceptions. Without having to throw them.

[xharness] Add new variations using the managed static registrar for …

3ac9b8f

…monotouch-test.

[static registrar] Refactor code to make it easier to reuse code late…

872af5d

…r on. There are no functional changes here, just refactoring to make code easier to re-use.

[static registrar] Refactor code to make it easier to reuse code late…

e66f82d

…r on. There are no functional changes here, just refactoring to make code easier to re-use.

[static registrar] Refactor code to make it easier to reuse code late…

a1410ac

…r on. There are no functional changes here, just refactoring to make code easier to re-use.

[tools] Add a ManagedStatic registrar mode.

cfb248e

This new mode is still considered a 'Static' registrar mode, it's just a variation of it.

[dotnet] Add an 'IsManagedStaticRegistrar' feature to the linker.

34264cd

This way we can ask the linker to inline the Runtime.IsManagedStaticRegistrar property, and remove any dead code paths.

[dotnet-linker] Add the scaffolding for a ManagedRegistrarStep and a …

bf639a9

…ManagedRegistrarLookupTablesStep.

[dotnet-linker] Don't do anything in ManagedRegistrarStep unless the …

57d40d9

…current registrar mode is 'ManagedStatic'.

rolfbjarne force-pushed the msr branch from c28004c to 15383e9 Compare May 11, 2023 10:26

This comment was marked as outdated.

Sign in to view

This comment has been minimized.

Sign in to view

rolfbjarne and others added 4 commits May 11, 2023 13:10

[registrar] Make some API from the registrar public so that the manag…

a1e0e30

…ed static registrar step can access them. There are no functional changes here, just refactoring to make code easier to re-use.

[src] Fix comparison between signed and unsigned int.

43b88af

Comparing -1 to 0xFFFFFFFF doesn't get the right result otherwise.

[static registrar] Add support for generating block syntax in Objecti…

8f1fb22

…ve-C method signatures. This is required when generating a cast of a function pointer to an Objective-C method signature (which the managed static registrar does).

stephen-hawley approved these changes May 16, 2023

View reviewed changes

rolfbjarne and others added 9 commits May 18, 2023 11:54

[dotnet-linker] Make sure local variables are initialized before use …

643015e

…in generated method bodies.

[dotnet-linker] Ensure we call Runtime.CopyAndAutorelease in all appl…

799095e

…icable code paths.

[dotnet-linker] Remove dead code.

a7371cf

[dotnet-linker] Keep better track of when we're calling MethodBase.In…

ed7b4b6

…voke. Keep better track of whether we're calling MethodBase.Invoke to call the target method, which allows us to detect certain error scenarios (and show a corresponding error).

[dotnet-linker] Add a missing cast to make generated code more verifi…

d540050

…able

[dotnet-linker] Simplify generated code for constructors of generic t…

666ec96

…ypes. We can't create an instance of a generic type in the UnmanagedCallersOnly trampoline, so simplify the code to not even try, and instead just throw the final exception.

Auto-format source code

03488a2

This comment has been minimized.

Sign in to view

Merge remote-tracking branch 'origin/main' into msr

65590ba

rolfbjarne mentioned this pull request May 18, 2023

[DRAFT] Add a managed static registrar. Fixes #17324. #17828

Closed

filipnavara mentioned this pull request May 18, 2023

ILLink produces invalid IL metadata with -b switch and Save assembly mode dotnet/runtime#86462

Closed

tj-devel709 approved these changes May 18, 2023

View reviewed changes

filipnavara mentioned this pull request May 18, 2023

Linking an empty .NET MAUI app produces inconsistent binaries dotnet/runtime#86242

Closed

rolfbjarne merged commit 0bc12c8 into xamarin:main May 19, 2023
75 of 77 checks passed

rolfbjarne deleted the msr branch May 19, 2023 05:30

ivanpovazan mentioned this pull request May 23, 2023

Enable support for NativeAOT targeting iOS via opt-in feature dotnet/runtime#80905

Open

52 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a managed static registrar. Fixes #17324. #18268

Add a managed static registrar. Fixes #17324. #18268

rolfbjarne commented May 11, 2023 •

edited

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented May 18, 2023

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented May 18, 2023

vs-mobiletools-engineering-service2 commented May 18, 2023

vs-mobiletools-engineering-service2 commented May 18, 2023

vs-mobiletools-engineering-service2 commented May 18, 2023

vs-mobiletools-engineering-service2 commented May 18, 2023

vs-mobiletools-engineering-service2 commented May 18, 2023

tj-devel709 left a comment

Add a managed static registrar. Fixes #17324. #18268

Add a managed static registrar. Fixes #17324. #18268

Conversation

rolfbjarne commented May 11, 2023 • edited

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment was marked as outdated.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions bot commented May 18, 2023

⚠️ Your code has been reformatted. ⚠️

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

vs-mobiletools-engineering-service2 commented May 18, 2023

📚 [PR Build] Artifacts 📚

Packages generated

vs-mobiletools-engineering-service2 commented May 18, 2023

✅ API diff for current PR / commit

✅ API diff vs stable

ℹ️ Generator diff

vs-mobiletools-engineering-service2 commented May 18, 2023

💻 [PR Build] Tests on macOS M1 - Mac Big Sur (11.5) passed 💻

vs-mobiletools-engineering-service2 commented May 18, 2023

💻 [PR Build] Tests on macOS M1 - Mac Ventura (13.0) passed 💻

vs-mobiletools-engineering-service2 commented May 18, 2023

💻 [CI Build] Windows Integration Tests passed 💻

vs-mobiletools-engineering-service2 commented May 18, 2023

🚀 [CI Build] Test results 🚀

Test results

Tests counts

tj-devel709 left a comment

Choose a reason for hiding this comment

rolfbjarne commented May 11, 2023 •

edited