Expand test coverage for native AOT on android #121217

sbomer · 2025-10-30T20:47:29Z

Enables building native AOT smoke tests for android with one APK per test project
Adds marker files to set expected exit code when running through helix
Adds a runtime-androidemulator job for native AOT smoke tests
Expands libraries test coverage to include non-smoke tests
Sets up appcontext base directory in test host
Calls managed main through a pinvoke when running Exe project

Fixes #120715

- Expand native AOT smoke tests to all libraries tests - Add job for runtime smoke tests using native AOT on emulator

Move NDK compiler logic to a shared props file and import it in test and build projects to unify Android NativeAOT and Mono builds.

- Set ASSETS_DIR for crypto tests (using same pattern as existing crypto test) - Set AppContext.BaseDirectory We should probably change crypto tests to use just AppContext.BaseDirectory.

- Use ReferenceXUnitWrapperGenerator != false to define SINGLE_FILE_TEST_RUNNER (android & singlefile targets) instead of IsFunctionalTest - Condition SingleFileTestRunner compile and TrimmerRootAssembly (TestUtilities) on wrapper/generator and SkipTestUtilitiesReference - Add Android NativeAOT publish/trim settings and set NativeLib/CustomNativeMain for android nativeaot - Make DisableImplicitFrameworkReferences respect preexisting value and add explicit DisableImplicitFrameworkReferences=false in nativeaot tests dir - Import testing\tests.targets when TargetsAndroid is true - Update many NativeAOT smoke tests Main signatures to public static int Main(string[] args) to match runner invocation

- Ensure they have Main entry point - Tighten MSBuild conditions: - Only set NativeLib/CustomNativeMain for Android when OutputType == Exe - Only import testing/tests.targets for Android when using MicrosoftNETSdk and building executables - Make TestNativeAOT/UseNativeAOTRuntime defaults apply when TestBuildMode == nativeaot and only include monodroid-nativeaot.cs for executable/XUnit-wrapper cases

This reverts commit 4ed06ec.

- Avoid importing logic for SharedLibrary - Fix library filename - Set HasMergedInTests to ensure projects build

This reverts commit 37b7b4e.

This import is now covered by the IsMergedTestRunnerAssembly property. Fixes a warning about a double import. Fixes issue where mono runtime tests are getting incorrect assembly name due to this undesired import.

src/libraries/System.Runtime/tests/System.Reflection.Tests/AssemblyTests.cs

src/libraries/tests.proj

src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostApplicationBuilderTests.cs

.../Microsoft.Extensions.Hosting/tests/UnitTests/Microsoft.Extensions.Hosting.Unit.Tests.csproj

MichalStrehovsky · 2025-10-31T06:37:15Z

src/tests/nativeaot/SmokeTests/AttributeTrimming/AttributeTrimming.cs

 class Program
 {
    [Method]
-    static int Main()


This doesn't look sustainable. src\tests\nativeaot is not different from the rest of src\tests. We would have to do these changes to all of src\tests to ship Android.

I assume this is needed because monodroid-nativeaot.cs needs to call main?

Can we make this work with the CustomMain approach? See the src\tests\nativeaot\CustomMain test - in that test we start in native code in CustomMainWithStubExeNative.cpp and call a magic __managed__Main symbol to reach the managed main.

If we let the compiler generate __managed__Main for these, monodroid-nativeaot.cs could just do:

[DllImport("*" /* Magic name that hard-binds the symbol at linker time*/, EntryPoint="__managed__Main")] static extern int ManagedMain(int argc, void** argv);

And then p/invoke that (yep, it's weird, we're in managed code, we do a p/invoke transition, and the first thing managed_main will do is to transition back to managed; should be fine; the important part is that this gives us a very stable symbol name and signature).

This should match what iOS testing is doing: #87260, the only difference is that iOS testing calls managed__Main from native code and we would call it from managed code.

The trick at the project level should be:

OutputType = Exe

NativeLib = Shared (or Static would work too, but I think this is using Shared right now)

CustomNativeMain= true

Updated to use this approach for the runtime tests, thanks!

I already had to set CustomNativeMain to get around an error when using OutputType=Exe with NativeLib=Shared, and had it on my list to check if that would cause an unnecessary export. Now I know!

I was wondering how dotnet/android avoids the error - apparently they override OutputType so that it's always Library.

- Use ActiveIssue - Re-enable tests that are passing with AppContext.BaseDirectory fix - Disable tests failing after merge - Update issue link for System.Diagnostics.Debug.Tests

MichalStrehovsky · 2025-11-03T06:01:43Z

src/libraries/Microsoft.Bcl.Cryptography/tests/X509Certificates/TestFiles.cs

    internal static class TestFiles
    {
-        internal const string TestDataFolder = "TestData";
+        internal static readonly string? TestDataRoot = Environment.GetEnvironmentVariable("ASSETS_DIR") ?? AppContext.BaseDirectory;


Do we still need an adjustment with this in place?

runtime/src/tasks/AndroidAppBuilder/Templates/monodroid-nativeaot.cs

Lines 43 to 45 in d08f20b

var filesDir = env->GetStringUTFChars(j_files_dir) ?? string.Empty;

AppContext.SetData("APP_CONTEXT_BASE_DIRECTORY", filesDir);

Environment.CurrentDirectory = filesDir;

Nope, cleaned this up. I'm planning to clean up other uses of the TEST_ASSETS env var in a separate PR.

src/tests/nativeaot/SmokeTests/Exceptions/Exceptions.csproj

src/tests/nativeaot/SmokeTests/HardwareIntrinsics/Directory.Build.props

src/tests/nativeaot/SmokeTests/Preinitialization/Preinitialization.csproj

src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostApplicationBuilderTests.cs

- Clean up whitespace diffs - Clean up TEST_ASSETS reference - ifdef out test logic that fails on Android - Work around failure by not using libc++

sbomer · 2025-11-03T18:45:24Z

/azp run runtime-android

sbomer · 2025-11-03T18:45:31Z

/azp run runtime-androidemulator

azure-pipelines · 2025-11-03T18:45:33Z

Azure Pipelines successfully started running 1 pipeline(s).

azure-pipelines · 2025-11-03T18:45:41Z

Azure Pipelines successfully started running 1 pipeline(s).

Copilot

Pull Request Overview

This PR adds support for running NativeAOT tests on Android by implementing the necessary infrastructure changes. The changes enable Android NativeAOT tests to build, package as APKs, and run on Android devices/emulators through Helix.

Key Changes

Adds Android-specific build configuration for NativeAOT tests
Implements custom entry point handling for NativeAOT on Android via monodroid-nativeaot.cs template
Configures CI/CD pipelines to build and run NativeAOT tests on Android
Excludes several tests that are incompatible with Android NativeAOT

Reviewed Changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/tests/nativeaot/Directory.Build.props	Adds Android-specific build properties including library import generator settings and test runner configuration
src/tests/Directory.Build.props	Enables aggressive trimming for Android NativeAOT, similar to iOS/tvOS
src/tests/Directory.Build.targets	Adds logic to skip OutputType inference and handle DisableImplicitFrameworkReferences for Android NativeAOT
src/tests/build.proj	Imports Android NDK compiler properties and adds RuntimeFlavor resolution for Android builds
src/tests/Common/mergedrunner*.targets	Adds ExpectedExitCode file generation and conditions for mobile targets
src/tests/Common/helixpublishwitharcade.proj	Reads ExpectedExitCode from file for Android test payloads
src/tasks/AndroidAppBuilder/Templates/monodroid-nativeaot.cs	Implements entry point execution via `__managed__Main` for Android NativeAOT tests
eng/testing/Android/AndroidNdkCompiler.props	New file that centralizes Android NDK compiler/linker path configuration
src/mono/msbuild/android/build/AndroidBuild.props	Imports centralized AndroidNdkCompiler.props instead of defining CppCompilerAndLinker locally
eng/testing/tests.*.targets	Updates conditions to use ReferenceXUnitWrapperGenerator and adds NativeAOT-specific properties for Android
eng/pipelines/extra-platforms/*.yml	Adds new Android NativeAOT CI jobs and removes /p:RunSmokeTestsOnly=true
eng/pipelines/common/platform-matrix.yml	Parameterizes runtimeFlavor for Android platforms
src/tests/nativeaot/SmokeTests/*/?.csproj	Marks several smoke tests as unsupported on Android with explanatory comments
src/tests/issues.targets	Excludes nativeaot/SmokeTests/UnitTests from Android NativeAOT
src/libraries/tests.proj	Excludes multiple library test projects that fail on Android NativeAOT
src/libraries//tests/.cs	Adds ActiveIssue attributes for tests that fail on Android NativeAOT
src/libraries/Microsoft.Extensions.Hosting/tests/UnitTests/HostApplicationBuilderTests.cs	Removes trailing whitespace

sbomer added 30 commits October 13, 2025 17:21

Add more native AOT on android ci test coverage

3bf76d6

- Expand native AOT smoke tests to all libraries tests - Add job for runtime smoke tests using native AOT on emulator

Remove --cross argument, fix and disable some libs tests

432572c

Remove --cross argument, fix and disable some libs tests

214d1f9

Set runtimeFlavor to coreclr

a9eeff0

Parameterize runtimeFlavor in Android job definitions

5253ab2

Remove runtimeFlavor: coreclr from Android emulator pipeline

9d8577b

Fix typo in runtimeFlavor parameter name

fef0415

Add 'nativeaot' arg to testBuildArgs for Android emulator

c509f51

Increase timeout to 180

4cbbff5

Refactor Android NDK compiler selection for test builds

c1292f1

Move NDK compiler logic to a shared props file and import it in test and build projects to unify Android NativeAOT and Mono builds.

Increase timeout to 240

dfbf8ea

Pass RuntimeFlavor to AndroidAppBuilderTask

b8abeba

Check TestBuildMode instead

cc1d5c3

Fix a few test failures

2b7f252

- Set ASSETS_DIR for crypto tests (using same pattern as existing crypto test) - Set AppContext.BaseDirectory We should probably change crypto tests to use just AppContext.BaseDirectory.

Set Environment.CurrentDirectory to app files directory

1ee5f13

Exclude tests failing due to missing calli marshalling support

cce4d9a

Exclude tests failing due to missing crypto shim export

e6338fb

Disable directory apk build for smoke tests

eec6deb

Try using merged test runners

4ed06ec

Revert "Try using merged test runners"

cbcca7d

This reverts commit 4ed06ec.

Add Merged marker for smoke tests

37b7b4e

Try merged wrappers again

9e89dda

- Avoid importing logic for SharedLibrary - Fix library filename - Set HasMergedInTests to ensure projects build

Revert "Add Merged marker for smoke tests"

3a266b8

This reverts commit 37b7b4e.

Fix some issues with non-standalone build

a0c0cd6

Remove buildAllTestsAsStandalone

55b18f5

Set expected exit code

8e60575

Remove redundant import

9f6e630

This import is now covered by the IsMergedTestRunnerAssembly property. Fixes a warning about a double import. Fixes issue where mono runtime tests are getting incorrect assembly name due to this undesired import.

Fix mono test build

d4d4ad3

github-actions bot added the area-NativeAOT-coreclr label Oct 30, 2025

dotnet-policy-service bot assigned sbomer Oct 30, 2025

jkotas reviewed Oct 31, 2025

View reviewed changes

src/libraries/System.Runtime/tests/System.Reflection.Tests/AssemblyTests.cs Outdated Show resolved Hide resolved

jkotas reviewed Oct 31, 2025

View reviewed changes

src/libraries/tests.proj Outdated Show resolved Hide resolved

MichalStrehovsky reviewed Oct 31, 2025

View reviewed changes

MichalStrehovsky mentioned this pull request Oct 31, 2025

[naot][Android] Assembly.GetEntryAssembly() returns null #121209

Open

sbomer added 6 commits October 31, 2025 10:36

Use __managed__Main, move suppession to common targets

dce1301

Merge remote-tracking branch 'origin/main' into naot-android-coverage

64ec45a

Clean up disabled tests

18b1964

- Use ActiveIssue - Re-enable tests that are passing with AppContext.BaseDirectory fix - Disable tests failing after merge - Update issue link for System.Diagnostics.Debug.Tests

Revert SmokeTest entry point code changes

d0edc55

Skip/enable some more tests

52b22d5

Fix non-android build, disable test

aabde97

sbomer force-pushed the sbomer/naot-android-coverage branch from 20fbe86 to d08f20b Compare October 31, 2025 22:51

Re-enable passing tests

d08f20b

This was referenced Nov 1, 2025

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

Unable to pull image from mcr.microsoft.com #117164

Open

MichalStrehovsky reviewed Nov 3, 2025

View reviewed changes

dotnet deleted a comment from azure-pipelines bot Nov 3, 2025

Copilot AI review requested due to automatic review settings November 3, 2025 18:43

PR feedback

6ea7cb9

- Clean up whitespace diffs - Clean up TEST_ASSETS reference - ifdef out test logic that fails on Android - Work around failure by not using libc++

dotnet deleted a comment from azure-pipelines bot Nov 3, 2025

Copilot AI reviewed Nov 3, 2025

View reviewed changes

	var filesDir = env->GetStringUTFChars(j_files_dir) ?? string.Empty;
	AppContext.SetData("APP_CONTEXT_BASE_DIRECTORY", filesDir);
	Environment.CurrentDirectory = filesDir;

Expand test coverage for native AOT on android #121217

Are you sure you want to change the base?

Expand test coverage for native AOT on android #121217

Conversation

sbomer commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

MichalStrehovsky Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

sbomer Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

MichalStrehovsky Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

sbomer Nov 3, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sbomer commented Nov 3, 2025

Uh oh!

sbomer commented Nov 3, 2025

Uh oh!

azure-pipelines bot commented Nov 3, 2025

Uh oh!

azure-pipelines bot commented Nov 3, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes

Reviewed Changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sbomer commented Oct 30, 2025 •

edited

Loading