[ci-scan] Known Build Error: Android x86 Mono APP_CRASH in libmonosgen-2.0.so after Android API level bump

[simonrozsival]

Note

This issue was drafted with GitHub Copilot.

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=1418805&view=logs&j=f580a640-930a-547e-3e08-155527737d6c&t=462974e4-73a9-512f-970b-7525db7a3891&s=ff05ad62-bb9a-53b6-ce9f-72f329a63e7c
Build error leg or test failing: net11.0-android-Release-x86-Mono_Release-Ubuntu.2204.Amd64.Android.29.Open - Microsoft.Extensions.Primitives.Tests / System.Runtime.Tests

Error Message

{
  "ErrorMessage": ["<<XHARNESS_RESULT_START>>","\"exitCodeName\": \"APP_CRASH\"","\"platform\": \"android\"","\"helixWorkItemId\": \"Microsoft.Extensions.Primitives.Tests\"","workitems/Microsoft.Extensions.Primitives.Tests/console","\"architecture\": \"x86\""],
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Notes

This tracks recurring Android x86 Mono APP_CRASH failures in runtime-extra-platforms.

Observed failing work items:

• Microsoft.Extensions.Primitives.Tests
• System.Runtime.Tests

Representative signature:

--------- beginning of crash
04-25 03:59:25.518  5978  6003 F libc    : Fatal signal 11 (SIGSEGV), code 128 (SI_KERNEL), fault addr 0x0 in tid 6003 (.NET TP Worker), pid 5978 (rimitives.Tests)
04-25 03:59:25.536  1893  1919 E memtrack: Couldn't load memtrack module
04-25 03:59:25.536  1893  1919 W android.os.Debug: failed to get memory consumption info: -1
04-25 03:59:25.673  6008  6008 I crash_dump32: obtaining output fd from tombstoned, type: kDebuggerdTombstone
04-25 03:59:25.674  1782  1782 I /system/bin/tombstoned: received crash request for pid 6003
04-25 03:59:25.675  6008  6008 I crash_dump32: performing dump of process 5978 (target tid = 6003)
04-25 03:59:25.693  6008  6008 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
04-25 03:59:25.693  6008  6008 F DEBUG   : Build fingerprint: 'Android/sdk_phone_x86/generic_x86:10/QSR1.210820.001/7663313:userdebug/test-keys'
04-25 03:59:25.693  6008  6008 F DEBUG   : Revision: '0'
04-25 03:59:25.693  6008  6008 F DEBUG   : ABI: 'x86'
04-25 03:59:25.694  6008  6008 F DEBUG   : Timestamp: 2026-04-25 03:59:25+0000
04-25 03:59:25.694  6008  6008 F DEBUG   : pid: 5978, tid: 6003, name: .NET TP Worker  >>> net.dot.Microsoft.Extensions.Primitives.Tests <<<
04-25 03:59:25.694  6008  6008 F DEBUG   : uid: 10123
04-25 03:59:25.694  6008  6008 F DEBUG   : signal 11 (SIGSEGV), code 128 (SI_KERNEL), fault addr 0x0
04-25 03:59:25.694  6008  6008 F DEBUG   :     eax 00000000  ebx 7b40e5ac  ecx 00000010  edx 799b7604
04-25 03:59:25.694  6008  6008 F DEBUG   :     edi 00000000  esi 78b876fc
04-25 03:59:25.694  6008  6008 F DEBUG   :     ebp 8eda41f8  esp 78b8767c  eip 7b2db927
04-25 03:59:25.765  6008  6008 F DEBUG   : 
04-25 03:59:25.765  6008  6008 F DEBUG   : backtrace:
04-25 03:59:25.766  6008  6008 F DEBUG   :       #00 pc 00248927  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.766  6008  6008 F DEBUG   :       #01 pc 002748d5  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.766  6008  6008 F DEBUG   :       #02 pc 002747fc  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #03 pc 002860d9  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #04 pc 00287132  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #05 pc 00286cef  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #06 pc 00286d5a  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (mono_class_setup_vtable+26) (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #07 pc 002d9b46  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (mono_class_vtable_checked+454) (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #08 pc 002e184b  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #09 pc 0028e4d8  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (mono_exception_from_token+104) (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #10 pc 00203c54  /data/app/net.dot.Microsoft.Extensions.Primitives.Tests-L1SoeqUitlghwTnk-YwB4g==/lib/x86/libmonosgen-2.0.so (BuildId: ba9fe95f091776b9dfb98426935559db42ca45ca)
04-25 03:59:25.767  6008  6008 F DEBUG   :       #11 pc 00000fae  <anonymous:988d3000>

The crash is present on main and was first observed in the scanned daily range between:

• Last clean checked build: 1394339, commit c835d0fc4f8a
• First failing checked build: 1396313, commit 51cfc7a608e6

Recent main builds and PR build 1418805 show the same failure mode. This does not appear related to PR #124173.

Local build plus Helix execution points to #126838, "Bump Android minimum API level from 21 to 24", as the likely regression cause or trigger:

Experiment	Work item	Helix job	Result
Baseline upstream/main commit 553d88792317e028eff9703301d84fab46ba095d	Microsoft.Extensions.Primitives.Tests	3eb04ceb-819a-4cce-9f86-0913f4944dda	Failed with the same APP_CRASH / libmonosgen-2.0.so signature
Revert #125835	Microsoft.Extensions.Primitives.Tests	452b3530-79bc-47bf-90ff-b168e260413b	Still failed with the same signature
Revert #126838	Microsoft.Extensions.Primitives.Tests	8056938a-ead1-462e-8d2e-f0750bddee52	Passed
Revert #126838	System.Runtime.Tests	ee3751f3-cdcc-4bb6-8027-64710103b9e9	Passed

The #126838-reverted APKs were built locally and submitted to Ubuntu.2204.Amd64.Android.29.Open, matching the failing queue. The first #126838-revert rebuild hit a stale native CMake configuration issue after changing API level from 24 back to 21; removing the stale Android x86 native obj outputs fixed the local build:

rm -rf artifacts/obj/mono/android.x86.Release artifacts/obj/native/net11.0-android-Release-x86 artifacts/bin/mono/android.x86.Release artifacts/bin/native/net11.0-android-Release-x86
./build.sh mono+libs -os android -arch x86 -c Release

No exact existing GitHub issue was found for the combination of Android x86 Mono, these work items, XHarness APP_CRASH exit 80, Fatal signal 11 (SIGSEGV), code 128 (SI_KERNEL), and libmonosgen-2.0.so.

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1396313
Error message validated: [Fatal signal 11 \(SIGSEGV\), code 128 \(SI_KERNEL\), fault addr 0x0[^\n]*\n(?:[^\n]*\n){0,80}[^\n]*/data/app/net\.dot\.(?:Microsoft\.Extensions\.Primitives|System\.Runtime)\.Tests-[^\n]*/lib/x86/libmonosgen-2\.0\.so]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 5/14/2026 11:32:39 AM UTC

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=1396313
Error message validated: [signal 11 (SIGSEGV), code 128 (SI_KERNEL), fault addr 0x0 /lib/x86/libmonosgen-2.0.so >>> net.dot.Microsoft.Extensions.Primitives.Tests <<<]
Result validation: ❌ Known issue did not match with the provided build.
Validation performed at: 5/14/2026 11:51:18 AM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Known Issue Error Message

DO NOT USE JSON BELOW IF THIS IS A BUILD BREAK otherwise build analysis will allow pull requests to merge that break the build worse. For a build break, do not use this issue form. Make a regular new issue.

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

Known Issue Error Message

DO NOT USE JSON BELOW IF THIS IS A BUILD BREAK otherwise build analysis will allow pull requests to merge that break the build worse. For a build break, do not use this issue form. Make a regular new issue.

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

[dotnet-policy-service]

Tagging subscribers to this area: @akoeplinger, @matouskozak, @simonrozsival
See info in area-owners.md if you want to be subscribed.

[simonrozsival]

Note

This comment was drafted with GitHub Copilot.

Additional symbolication for the Microsoft.Extensions.Primitives.Tests tombstone from build 1396313.

Caveat: I could not find the exact CI libmonosgen-2.0.so.dbg for BuildId ba9fe95f091776b9dfb98426935559db42ca45ca in the public build artifacts; those contain log/binlog bundles only. I rebuilt the exact source commit 51cfc7a608e60e2e1d5b8e65c9f0ffe8a322fe1f locally for Android x86 Release. That produced BuildId c7abdd93d87b658c3f41354c05b67c3c9126215d, so direct addr2line was not valid.

The exported frames in the tombstone let us derive a constant local adjustment of +0x2dc0:

crash pc 0x00286d5a = mono_class_setup_vtable+26
local symbol mono_class_setup_vtable = 0x00289b00

crash pc 0x002d9b46 = mono_class_vtable_checked+454
local symbol mono_class_vtable_checked = 0x002dc740

crash pc 0x0028e4d8 = mono_exception_from_token+104
local symbol mono_exception_from_token = 0x00291230

Using local_pc = crash_pc + 0x2dc0, the native stack resolves to:

#00 monoeg_g_hash_table_new
    src/mono/mono/eglib/ghashtable.c:114

#01 collect_implemented_interfaces_aux
    src/mono/mono/metadata/class.c:1908

#02 mono_class_get_implemented_interfaces
    src/mono/mono/metadata/class.c:1934

#03 mono_class_setup_interface_offsets_internal
    src/mono/mono/metadata/class-setup-vtable.c:220

#04 mono_class_setup_vtable_general
    src/mono/mono/metadata/class-setup-vtable.c:1723

#05 mono_class_setup_vtable_full
    src/mono/mono/metadata/class-setup-vtable.c:968

#06 mono_class_setup_vtable
    src/mono/mono/metadata/class-setup-vtable.c:896

#07 mono_class_create_runtime_vtable
    src/mono/mono/metadata/object.c:2153

#08 mono_object_new_handle
    src/mono/mono/metadata/object.c:5369

#09 mono_exception_from_token
    src/mono/mono/metadata/exception.c:133

#10 mono_x86_throw_corlib_exception
    src/mono/mono/mini/exceptions-x86.c:506

#11 <anonymous JIT frame>

The interesting path appears to be: the x86 throw helper tries to create a corlib exception, mono_exception_from_token() allocates the exception object, that forces vtable creation, and the crash occurs while collecting implemented interfaces during interface-offset setup.