Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] signal 11 (SIGSEGV), code 1 (SEGV_MAPERR) #167

Open
neuronix opened this issue Dec 5, 2019 · 24 comments
Open

[BUG] signal 11 (SIGSEGV), code 1 (SEGV_MAPERR) #167

neuronix opened this issue Dec 5, 2019 · 24 comments

Comments

@neuronix
Copy link

@neuronix neuronix commented Dec 5, 2019

Ever since migrating to targetSDK="28" as required by Google, our apps are plagued with these errors, causing up to 9% of sessions with crashes on Android 9. The bug does sometimes occur on other versions but 95% of times, it's Android 9.

With exactly the same ANEs, setup & AIR SDK, we had just about none of these errors using targetSDK=19.

This is a bug that has been around in AIR for years now (not joking) but it has never been so bad. AIR 33 does not fix the issue either.

Some topics where devs have discussed this issue but failed to find a solution:

Unfortunately the stack traces are pretty poor:

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
strcmp

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000001cb70 /system/lib64/libc.so (strcmp+24)
#1 pc 00000000003b8ddc /data/app/air.com.company.application-1/lib/arm64/libCore.so
#2 pc 00000000003bc750 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#3 pc 00000000002d2a6c /data/app/air.com.company.application-1/lib/arm64/libCore.so
#4 pc 00000000002d3b10 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#5 pc 00000000002d64e8 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#6 pc 00000000002d9250 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#7 pc 00000000006f0308 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#8 pc 00000000006ef5a8 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#9 pc 0000000000271c38 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#10 pc 0000000000271ca8 /data/app/air.com.company.application-1/lib/arm64/libCore.so
#11 pc 00000000002719dc /data/app/air.com.company.application-1/lib/arm64/libCore.so
#12 pc 00000000000677d4 /system/lib64/libc.so (_ZL15__pthread_startPv+52)
#13 pc 000000000001edf0 /system/lib64/libc.so (__start_thread+16)`

I'd like to underline the urgence of this matter. 9% crash rate is really far far over the 1% Playstore bad behavior threshold, so this is hurty our apps very badly because Google reduces exposure of apps with poor crash / anr rates. We also have no workaround since we can no longer revert to a lower targetSDK.

Thanks for your support

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Dec 5, 2019

Hi - there may be a lot of different reasons for crashing with sigsegv but the above trace would likely allow us to eliminate at least that one! What build is this using?

Thanks

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Dec 5, 2019

Hi Andrew, thanks for the prompt answer! This particular app is .288 but we have the same issue with another one built with .315 and same issue with our apps on AIR32.

Here are a few other traces (and there are many in the starling forum topic)

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000001c828 /system/lib64/libc.so (memset+168)
#1 pc 0000000000c65450 /data/app/air.com.company.application-1/lib/arm64/libCore.so`

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 00000000003bc544 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#1 pc 00000000002d2aac /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#2 pc 00000000002d3b10 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#3 pc 00000000002d64e8 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#4 pc 00000000002d9250 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#5 pc 00000000006f0308 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#6 pc 00000000006ef5a8 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#7 pc 0000000000271c38 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#8 pc 0000000000271ca8 /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#9 pc 00000000002719dc /data/app/air.com.company.application-aRJQTUDq-WDNS_I-mY5odQ==/lib/arm64/libCore.so
#10 pc 00000000000671b8 /system/lib64/libc.so (_ZL15__pthread_startPv+36)
#11 pc 000000000001eee4 /system/lib64/libc.so (__start_thread+68)`

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 0000000000092de0 /system/lib64/libc.so (ifree+88)
#1 pc 0000000000093334 /system/lib64/libc.so (je_free+120)
#2 pc 0000000000a36280 /data/app/air.com.company.application-6OkQlTyusfNPJRyysaoWCQ==/lib/arm64/libCore.so
#3 pc 000000000052a6fc /data/app/air.com.company.application-6OkQlTyusfNPJRyysaoWCQ==/lib/arm64/libCore.so
#4 pc 0000000000368920 /data/app/air.com.company.application-6OkQlTyusfNPJRyysaoWCQ==/lib/arm64/libCore.so
#5 pc 0000000000271c38 /data/app/air.com.company.application-6OkQlTyusfNPJRyysaoWCQ==/lib/arm64/libCore.so
#6 pc 0000000000271ca8 /data/app/air.com.company.application-6OkQlTyusfNPJRyysaoWCQ==/lib/arm64/libCore.so
#7 pc 00000000002719dc /data/app/air.com.company.application-6OkQlTyusfNPJRyysaoWCQ==/lib/arm64/libCore.so
#8 pc 0000000000068560 /system/lib64/libc.so (_ZL15__pthread_startPv+36)
#9 pc 000000000001f9f4 /system/lib64/libc.so (__start_thread+68)`

This one on Android 8.0
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000001c568 /system/lib64/libc.so (memcpy+248)
#1 pc 0000000000019a9c /system/lib64/libopenjdk.so (newEntry+568)
#2 pc 0000000000019e0c /system/lib64/libopenjdk.so (ZIP_GetNextEntry+60)
#3 pc 0000000000621870 /system/framework/arm64/boot.oat (java.util.zip.ZipFile.getNextEntry [DEDUPED]+160)
#4 pc 00000000006bb6c0 /system/framework/arm64/boot.oat (java.util.zip.ZipFile$ZipEntryIterator.next+336)
#5 pc 000000000061ee88 /system/framework/arm64/boot.oat (java.util.jar.JarFile$JarEntryIterator.next [DEDUPED]+40)
#6 pc 00000000003c6968 /system/framework/arm64/boot.oat (java.security.cert.PKIXRevocationChecker.clone [DEDUPED]+40)
#7 pc 00000000000bc60c /data/app/air.com.company.application-FviBakdrTTh1W6Z3poGRpA==/oat/arm64/base.odex`

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000008e848 /system/lib64/libc.so (je_malloc+264)
#1 pc 000000000134dc40 /data/app/com.android.chrome-1/base.apk
#2 pc 00000000013e9138 /data/app/com.android.chrome-1/base.apk`

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 00000000000686e4 /system/lib64/libc.so (pthread_mutex_lock)
#1 pc 000000000007df60 /system/lib64/libc.so (je_arena_dalloc_large+44)
#2 pc 0000000000092abc /system/lib64/libc.so (je_free+120)
#3 pc 0000000000a3262c /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#4 pc 000000000052ab94 /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#5 pc 0000000000525030 /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#6 pc 0000000000523f6c /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#7 pc 0000000000330c18 /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#8 pc 0000000000330824 /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#9 pc 0000000000271c38 /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#10 pc 0000000000271ca8 /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#11 pc 00000000002719dc /data/app/air.com.company.application-QGaeK_EjyUYK8kCJT6aoqg==/lib/arm64/libCore.so
#12 pc 0000000000067ce8 /system/lib64/libc.so (_ZL15__pthread_startPv+36)
#13 pc 000000000001f324 /system/lib64/libc.so (__start_thread+68)`

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Dec 5, 2019

Looks like lots of different problems! The first one is in openssl .. we can look at adding some protection around this or see whether there's a bad call being made to it. I don't suppose these are reproducible at all..? it's always nicer if we can get confirmation that something had been crashing and then has been fixed..!

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Dec 5, 2019

Unfortunately we don't know how/when these errors occur, we just have the reports in the Google Playstore Console. But we will definitely see a drop in crash % once things are fixed :)

Here are a few more from another app with minor differences:

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application2 <<<

backtrace:
#00 pc 00000000000c05b4 /system/lib64/libc.so (je_tcache_bin_flush_small+376)
#1 pc 00000000000c2adc /system/lib64/libc.so (tcache_destroy+2688)
#2 pc 00000000000c2048 /system/lib64/libc.so (je_tcache_cleanup+28)
#3 pc 00000000000c402c /system/lib64/libc.so (je_tsd_cleanup+60)
#4 pc 00000000000c4de8 /system/lib64/libc.so (je_tsd_cleanup_wrapper+32)
#5 pc 00000000000906a4 /system/lib64/libc.so (pthread_key_clean_all()+128)
#6 pc 0000000000090104 /system/lib64/libc.so (pthread_exit+72)
#7 pc 000000000008ffcc /system/lib64/libc.so (__pthread_start(void*)+40)
#8 pc 0000000000023968 /system/lib64/libc.so (__start_thread+68)`

`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application2 <<<

backtrace:
#00 pc 000000000007db64 /system/lib64/libc.so (arena_dalloc_bin_locked_impl+364)
#1 pc 000000000009c8dc /system/lib64/libc.so (je_tcache_bin_flush_small+344)
#2 pc 000000000009d87c /system/lib64/libc.so (tcache_destroy+300)
#3 pc 000000000009d73c /system/lib64/libc.so (je_tcache_cleanup+28)
#4 pc 000000000009e008 /system/lib64/libc.so (je_tsd_cleanup+60)
#5 pc 000000000009ec50 /system/lib64/libc.so (je_tsd_cleanup_wrapper+32)
#6 pc 00000000000683ec /system/lib64/libc.so (_Z21pthread_key_clean_allv+120)
#7 pc 0000000000067ea0 /system/lib64/libc.so (pthread_exit+72)
#8 pc 0000000000067d60 /system/lib64/libc.so (_ZL15__pthread_startPv+292)
#9 pc 000000000001f348 /system/lib64/libc.so (__start_thread+68)`

There are many more different ones but then sometimes only one occurrence. So far the most "popular" one is the "strcmp" one, the one in my first post.

Here are some from another app that is still running on AIR 32. As I stated, the issues arise with targetSDK=28, we have other apps still running with targetSDK=19 on the same AIR 32 build and they don't have any major issues.

Android 9 & 8.1
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000001a70c /system/lib/libc.so (strcmp+108)
#1 pc 000000000060ec2d /data/app/air.com.company.application-GDDGFi_7XRTxN3glbvz8Gw==/lib/arm/libCore.so (VarStrEqual(char const*, char const*, bool)+12)
#2 pc 00000000f42c0068 `

Multiple Android versions
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 0000000000019aa4 /system/lib/libc.so (strcmp+107)
#1 pc 000000000060ec2d /data/app/air.com.company.application-Nii0Dt0mwJgDjA12Hzho6g==/lib/arm/libCore.so (_Z11VarStrEqualPKcS0_b+12)
#2 pc 00000000f42c0064 `

Android 9 & 8.1
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 0000000000612288 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (HashTable::LookupItem(void const*, void**)+43)
#1 pc 0000000000559403 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (AndroidAppDirectoryManager::IncrementHashCount(FlashFileString const&)+62)
#2 pc 000000000055a07d /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (AndroidAppDirectoryManager::InternalFileOpenAppDir(FlashFileString const&, FlashFileString const&)+24)
#3 pc 000000000055c9dd /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (PlatformFileManager::InternalFileOpen(FlashFileString const&, int, long long&, long long&)+56)
#4 pc 000000000055e915 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (PlatformFileManager::FileOpen(FlashFileString const&, unsigned int)+72)
#5 pc 0000000000873f69 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (avmplus::FileStreamAsync::ThreadProc(CoreFileManager*)+64)
#6 pc 0000000000873583 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (avmplus::FileStreamAsync::ThreadProc(void*)+50)
#7 pc 000000000050e3ed /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (TSafeThread::RunHelper()+92)
#8 pc 000000000050e443 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so (TSafeThread::Run(EThreadType)+6)
#9 pc 000000000050e1b7 /data/app/air.com.company.application-umYrZnf7iqjzC7nW92bj0Q==/lib/arm/libCore.so
#10 pc 0000000000048811 /system/lib/libc.so (__pthread_start(void*)+24)
#11 pc 000000000001b369 /system/lib/libc.so (__start_thread+32)`

Android 9 only
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000001a7c8 /system/lib/libc.so (strcmp+48)
#1 pc 000000000060ec2d /data/app/air.com.company.application-Pv6WZkBRZpm5PH_zVhtTRw==/lib/arm/libCore.so (VarStrEqual(char const*, char const*, bool)+12)
#2 pc 00000000006eaf5f /system/framework/boot-framework.vdex`

Multiple Android versions
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 0000000000612288 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN9HashTable10LookupItemEPKvPPv+43)
#1 pc 0000000000559403 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN26AndroidAppDirectoryManager18IncrementHashCountERK15FlashFileString+62)
#2 pc 000000000055a07d /data/app/air.com.company.application-2/lib/arm/libCore.so (ZN26AndroidAppDirectoryManager22InternalFileOpenAppDirERK15FlashFileStringS2+24)
#3 pc 000000000055c9dd /data/app/air.com.company.application-2/lib/arm/libCore.so (ZN19PlatformFileManager16InternalFileOpenERK15FlashFileStringiRxS3+56)
#4 pc 000000000055e915 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN19PlatformFileManager8FileOpenERK15FlashFileStringj+72)
#5 pc 0000000000873f69 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN7avmplus15FileStreamAsync10ThreadProcEP15CoreFileManager+64)
#6 pc 0000000000873583 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN7avmplus15FileStreamAsync10ThreadProcEPv+50)
#7 pc 000000000050e3ed /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN11TSafeThread9RunHelperEv+92)
#8 pc 000000000050e443 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN11TSafeThread3RunE11EThreadType+6)
#9 pc 000000000050e1b7 /data/app/air.com.company.application-2/lib/arm/libCore.so
#10 pc 00000000000479c3 /system/lib/libc.so (_ZL15__pthread_startPv+22)
#11 pc 0000000000019efd /system/lib/libc.so (__start_thread+6)`

Android 6 only (Starling forum topic started with issues on Android 6)
`*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.application <<<

backtrace:
#00 pc 000000000003085e /system/lib/libc.so (dlfree+1285)
#1 pc 0000000000609375 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN9HashTable15BucketArrayFreeEPPNS_6BucketE+8)
#2 pc 0000000000612247 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN9HashTable6ExpandEv+126)
#3 pc 00000000006121bd /data/app/air.com.company.application-2/lib/arm/libCore.so (ZN9HashTable10InsertItemEPvS0+132)
#4 pc 000000000055942b /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN26AndroidAppDirectoryManager18IncrementHashCountERK15FlashFileString+102)
#5 pc 000000000055a07d /data/app/air.com.company.application-2/lib/arm/libCore.so (ZN26AndroidAppDirectoryManager22InternalFileOpenAppDirERK15FlashFileStringS2+24)
#6 pc 000000000055c9dd /data/app/air.com.company.application-2/lib/arm/libCore.so (ZN19PlatformFileManager16InternalFileOpenERK15FlashFileStringiRxS3+56)
#7 pc 000000000055e915 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN19PlatformFileManager8FileOpenERK15FlashFileStringj+72)
#8 pc 0000000000873f69 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN7avmplus15FileStreamAsync10ThreadProcEP15CoreFileManager+64)
#9 pc 0000000000873583 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN7avmplus15FileStreamAsync10ThreadProcEPv+50)
#10 pc 000000000050e3ed /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN11TSafeThread9RunHelperEv+92)
#11 pc 000000000050e443 /data/app/air.com.company.application-2/lib/arm/libCore.so (_ZN11TSafeThread3RunE11EThreadType+6)
#12 pc 000000000050e1b7 /data/app/air.com.company.application-2/lib/arm/libCore.so
#13 pc 0000000000041813 /system/lib/libc.so (_ZL15__pthread_startPv+30)
#14 pc 0000000000019315 /system/lib/libc.so (__start_thread+6)`

@marcanw

This comment has been minimized.

Copy link

@marcanw marcanw commented Dec 12, 2019

Yep same issue Signal 11 or Signal 5
And I confirm majority of crashes occur on android 9

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Feb 13, 2020

Hi @ajwfrost

Is there any news on this topic ? I don't see anything in the release notes regarding this. It really is plaguing our apps with insane crash rates: ~6% of sessions.. ! :(

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Feb 14, 2020

Hi - sorry for the delay. We had a change go in to a build recently that may have improved things, a lot of these issues on Android 9 appear to be due to a change in the workflow and the fact we're not getting a valid activity/context from some of our calls (e.g. when in the background). Looking at it, this was in a 33.1 beta release but not in the current 33.0 branch.. the plan is to release the next production version at the end of next week.

The calls stacks you have above seem to fall into one of three camps:

_ZN9HashTable10LookupItemEPKvPPv or VarStrEqual: this appears to be due to a corrupt (or null) string being used for a filename, where it's trying to get the application directory (which is likely to rely on the context..)

ZIP_GetNextEntry: this may be slightly different if it's accessing resources within the main APK file itself, there were some ANRs relating to this too I think so we were looking to check how we access these and ensure we clean up properly in case of any exceptions on the Java side.

je_free: this is most likely caused by string corruption and is probably linked to the first of the issues here.

There's always another possibility so please keep the call stacks coming (and any register/memory information that you might get as part of these). It's also really helpful if you're able to identify the exact runtime used in each of the crash logs, we trace this out during startup but I'm guessing you don't actually get the full logcat output from Google Play...

thanks

Andrew

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Mar 12, 2020

Hi Andrew,

We updated an app with 33.1.1.63 and unfortunately the issue has not changed much, please find more stack traces here:

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
strcmp

Huawei HUAWEI P30 lite (HWMAR), 3840MB RAM, Android 9
Report 1
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 000000000001e748  /system/lib64/libc.so (strcmp+136)
  #01  pc 00000000003bad8c  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #02  pc 00000000003be700  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #03  pc 00000000002d38a0  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #04  pc 00000000002d4944  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #05  pc 00000000002d731c  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #06  pc 00000000002da084  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #07  pc 00000000006f4ac8  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #08  pc 00000000006f3d68  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #09  pc 0000000000272960  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #10  pc 00000000002729d0  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #11  pc 0000000000272704  /data/app/air.com.company.app-s53aJSh7H_JH1I25gz1MSA==/lib/arm64/libCore.so
  #12  pc 0000000000083588  /system/lib64/libc.so (__pthread_start(void*)+36)
  #13  pc 00000000000241dc  /system/lib64/libc.so (__start_thread+68)
Samsung Galaxy S8 (dreamlte), 3840MB RAM, Android 9
Report 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 000000000001e2d8  /system/lib64/libc.so (strcmp+24)
  #01  pc 00000000003bad8c  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #02  pc 00000000003be504  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #03  pc 00000000002d38e0  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #04  pc 00000000002d4944  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #05  pc 00000000002d731c  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #06  pc 00000000002da084  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #07  pc 00000000006f4ac8  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #08  pc 00000000006f3d68  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #09  pc 0000000000272960  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #10  pc 00000000002729d0  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #11  pc 0000000000272704  /data/app/air.com.company.app-IRUfmMRZHqSAgDbfYiWlKQ==/lib/arm64/libCore.so
  #12  pc 0000000000084e00  /system/lib64/libc.so (__pthread_start(void*)+208)
  #13  pc 0000000000023a8c  /system/lib64/libc.so (__start_thread+68)
Google Pixel 2 (walleye), 3840MB RAM, Android 10
Reports

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 000000000007e858  /apex/com.android.runtime/lib64/bionic/libc.so (strcmp+24)
  #01  pc 00000000003bad8c  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #02  pc 00000000003be700  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #03  pc 00000000002d38a0  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #04  pc 00000000002d4944  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #05  pc 00000000002d731c  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #06  pc 00000000002da084  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #07  pc 00000000006f4ac8  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #08  pc 00000000006f3d68  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #09  pc 0000000000272960  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #10  pc 00000000002729d0  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #11  pc 0000000000272704  /data/app/air.com.company.app-XZjr6fKxmHt2iUMgWbfb0Q==/lib/arm64/libCore.so
  #12  pc 00000000000e2364  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+36)
  #13  pc 0000000000083d98  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
libCore.so

Huawei HUAWEI P30 lite (HWMAR), 3840MB RAM, Android 9
Reports

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 00000000003be6f0  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #01  pc 00000000002d38a0  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #02  pc 00000000002d4944  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #03  pc 00000000002d731c  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #04  pc 00000000002da084  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #05  pc 00000000006f4ac8  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #06  pc 00000000006f3d68  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #07  pc 0000000000272960  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #08  pc 00000000002729d0  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #09  pc 0000000000272704  /data/app/air.com.company.app-T7O2dUiLEHNqTwzrTWKD5g==/lib/arm64/libCore.so
  #10  pc 0000000000083588  /system/lib64/libc.so (__pthread_start(void*)+36)
  #11  pc 00000000000241dc  /system/lib64/libc.so (__start_thread+68)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
arena_run_reg_alloc

GE LG X charge (mlv7), 2048MB RAM, Android 7.1
Reports
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 0000000000054282  /system/lib/libc.so (arena_run_reg_alloc+101)
  #01  pc 00000000000548a9  /system/lib/libc.so (je_arena_malloc_hard+76)
  #02  pc 0000000000061c0f  /system/lib/libc.so (je_malloc+122)
  #03  pc 0000000000048835  /system/lib/libc++.so (operator new(unsigned int)+16)
  #04  pc 000000000002618d  /system/lib/libandroidfw.so (OpenArchive+24)
  #05  pc 0000000000022a25  /system/lib/libandroidfw.so (android::ZipFileRO::open(char const*)+20)
  #06  pc 0000000000016777  /system/lib/libandroidfw.so (android::AssetManager::SharedZip::SharedZip(android::String8 const&, long)+70)
  #07  pc 0000000000016837  /system/lib/libandroidfw.so (android::AssetManager::SharedZip::get(android::String8 const&, bool)+146)
  #08  pc 0000000000015049  /system/lib/libandroidfw.so (android::AssetManager::ZipSet::getZipResourceTableAsset(android::String8 const&)+52)
  #09  pc 0000000000013b69  /system/lib/libandroidfw.so (android::AssetManager::appendPathToResTable(android::AssetManager::asset_path const&, bool) const+200)
  #10  pc 0000000000013887  /system/lib/libandroidfw.so (android::AssetManager::addAssetPath(android::String8 const&, int*, bool, bool)+302)
  #11  pc 000000000009b993  /system/lib/libandroid_runtime.so
  #12  pc 0000000001d8a215  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.content.res.AssetManager.addAssetPathNative+104)
  #13  pc 0000000001d8a13b  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.content.res.AssetManager.addAssetPathInternal+70)
  #14  pc 0000000001d8ba43  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.content.res.AssetManager.addAssetPath+46)
  #15  pc 0000000001c2721b  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ResourcesManager.createAssetManager+302)
  #16  pc 0000000001c24ebb  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ResourcesManager.createResourcesImpl+118)
  #17  pc 0000000001c256bf  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ResourcesManager.getOrCreateResources+482)
  #18  pc 0000000001c27bd7  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ResourcesManager.getResources+282)
  #19  pc 0000000001b81153  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ActivityThread.getTopLevelResources+150)
  #20  pc 0000000001c04ce1  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.LoadedApk.getResources+84)
  #21  pc 0000000001bba39d  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ContextImpl.<init>+768)
  #22  pc 0000000001bbe229  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.app.ContextImpl.createApplicationContext+268)
  #23  pc 0000000001af665b  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.content.ContextWrapper.createApplicationContext+46)
  #24  pc 00000000024b2fa9  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebViewFactory.getWebViewContextAndSetProvider+796)
  #25  pc 00000000024b2775  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebViewFactory.getProviderClass+104)
  #26  pc 00000000024b246b  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebViewFactory.getProvider+182)
  #27  pc 00000000024ab631  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.getFactory+52)
  #28  pc 00000000024ab481  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.ensureProviderCreated+68)
  #29  pc 00000000024af9d5  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.setOverScrollMode+64)
  #30  pc 00000000023a3733  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.view.View.<init>+486)
  #31  pc 00000000023a3987  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.view.View.<init>+66)
  #32  pc 00000000023f9761  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.view.ViewGroup.<init>+76)
  #33  pc 00000000024a9e9d  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.widget.AbsoluteLayout.<init>+64)
  #34  pc 00000000024aae5f  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.<init>+74)
  #35  pc 00000000024aaddd  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.<init>+80)
  #36  pc 00000000024aad55  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.<init>+64)
  #37  pc 00000000024aace1  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.<init>+60)
  #38  pc 00000000024aac73  /system/framework/arm/boot-framework.oat (offset 0x1af3000) (android.webkit.WebView.<init>+46)
  #39  pc 00000000006b42ff  /data/user_de/0/com.google.android.gms/app_chimera/m/000000d9/oat/arm/dl-AdsFdrDynamite.integ_21207000006.odex (offset 0x4b0000)

tgkill

Huawei Mate 10 lite (HWRNE), 3840MB RAM, Android 8.0
Reports
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 000000000006a808  /system/lib64/libc.so (tgkill+8)
  #01  pc 000000000001db50  /system/lib64/libc.so (abort+88)
  #02  pc 000000000002532c  /system/lib64/libc.so (__libc_fatal+116)
  #03  pc 0000000000091cf0  /system/lib64/libc.so (ifree+812)
  #04  pc 0000000000091f70  /system/lib64/libc.so (je_free+120)
  #05  pc 000000000035d2c0  /system/lib64/libart.so (void art::JNI::ReleasePrimitiveArray<_jcharArray*, unsigned short, art::mirror::PrimitiveArray<unsigned short> >(_JNIEnv*, _jcharArray*, unsigned short*, int)+652)
  #06  pc 0000000000027ec8  /system/lib64/libjavacore.so (NativeConverter_decode(_JNIEnv*, _jclass*, long, _jbyteArray*, int, _jcharArray*, int, _jintArray*, unsigned char)+872)
  #07  pc 00000000003149cc  /system/framework/arm64/boot-core-libart.oat (offset 0xd6000) (libcore.icu.NativeConverter.decode [DEDUPED]+268)
  #08  pc 0000000000308460  /system/framework/arm64/boot-core-libart.oat (offset 0xd6000) (java.nio.charset.CharsetDecoderICU.implFlush+288)
  #09  pc 00000000006b9ee8  /system/framework/arm64/boot.oat (offset 0x1dc000) (java.util.zip.ZipCoder.toString+1160)
  #10  pc 0000000000620240  /system/framework/arm64/boot.oat (offset 0x1dc000) (java.util.zip.ZipFile.getZipEntry+592)
  #11  pc 00000000006bab10  /system/framework/arm64/boot.oat (offset 0x1dc000) (java.util.zip.ZipFile$ZipEntryIterator.next+1120)
  #12  pc 000000000061d588  /system/framework/arm64/boot.oat (offset 0x1dc000) (java.util.jar.JarFile$JarEntryIterator.next [DEDUPED]+40)
  #13  pc 00000000003c50b8  /system/framework/arm64/boot.oat (offset 0x1dc000) (java.security.cert.PKIXRevocationChecker.clone [DEDUPED]+40)
  #14  pc 0000000000005198  /dev/ashmem/dalvik-jit-code-cache (deleted)
@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Mar 13, 2020

Hi @ajwfrost

Please find a new batch of traces here from another app built with .63. It seems like some of these are more detailed and maybe more useful:


signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
strcmp


Huawei P20 lite (HWANE), 3840MB RAM, Android 9
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 000000000001e748  /system/lib64/libc.so (strcmp+136)
  #01  pc 00000000003bad8c  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #02  pc 00000000003be700  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #03  pc 00000000002d38a0  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #04  pc 00000000002d4944  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #05  pc 00000000002d731c  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #06  pc 00000000002da084  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #07  pc 00000000006f4ac8  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #08  pc 00000000006f3d68  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #09  pc 0000000000272960  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #10  pc 00000000002729d0  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #11  pc 0000000000272704  /data/app/air.com.company.app2-4hNERi4AQP7Gw9VlYL-frg==/lib/arm64/libCore.so
  #12  pc 0000000000083588  /system/lib64/libc.so (__pthread_start(void*)+36)
  #13  pc 00000000000241dc  /system/lib64/libc.so (__start_thread+68)
  
Samsung Galaxy A6 (a6lte), 2816MB RAM, Android 8.0
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 0000000000019a8c  /system/lib/libc.so (strcmp+107)
  #01  pc 00000000001e1e6d  /data/app/air.com.company.app2-H2qOn1114nQ3KxB3brV35w==/lib/arm/libCore.so
  #02  pc 00000000f42c0064  <unknown>

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
arena_dalloc_bin_locked_impl


Huawei honor 10 Lite (HWHRY-H), 2816MB RAM, Android 10
Report 1

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 0000000000044838  /apex/com.android.runtime/lib64/bionic/libc.so (arena_dalloc_bin_locked_impl+360)
  #01  pc 00000000000658d8  /apex/com.android.runtime/lib64/bionic/libc.so (je_tcache_bin_flush_small+340)
  #02  pc 000000000006681c  /apex/com.android.runtime/lib64/bionic/libc.so (tcache_destroy+304)
  #03  pc 00000000000666d8  /apex/com.android.runtime/lib64/bionic/libc.so (je_tcache_cleanup+24)
  #04  pc 0000000000066f84  /apex/com.android.runtime/lib64/bionic/libc.so (je_tsd_cleanup+64)
  #05  pc 0000000000067dd8  /apex/com.android.runtime/lib64/bionic/libc.so (je_tsd_cleanup_wrapper+32)
  #06  pc 00000000000ce850  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_key_clean_all()+116)
  #07  pc 00000000000ce2b4  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_exit+72)
  #08  pc 00000000000ce194  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+40)
  #09  pc 0000000000070ba8  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
pthread_mutex_lock


Huawei HUAWEI MediaPad T3 10 (HWAGS-Q), 2048MB RAM, Android 8.0
Report 1 of 4

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 0000000000075c20  /system/lib64/libc.so (pthread_mutex_lock)
  #01  pc 00000000000b5084  /system/lib64/libc.so (je_tcache_bin_flush_small+216)
  #02  pc 00000000000b7998  /system/lib64/libc.so (tcache_destroy+2676)
  #03  pc 00000000000b6f10  /system/lib64/libc.so (je_tcache_cleanup+28)
  #04  pc 00000000000b8a9c  /system/lib64/libc.so (je_tsd_cleanup+60)
  #05  pc 00000000000b9590  /system/lib64/libc.so (je_tsd_cleanup_wrapper+32)
  #06  pc 000000000007588c  /system/lib64/libc.so (pthread_key_clean_all()+128)
  #07  pc 0000000000075340  /system/lib64/libc.so (pthread_exit+72)
  #08  pc 00000000000751f8  /system/lib64/libc.so (__pthread_start(void*)+40)
  #09  pc 000000000001f83c  /system/lib64/libc.so (__start_thread+68)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
art::gc::collector::ConcurrentCopying::ProcessThreadLocalMarkStacks(bool, art::Closure*)


Huawei P20 lite (HWANE), 3840MB RAM, Android 9
Report 1 of 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 00000000001c9bfc  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ProcessThreadLocalMarkStacks(bool, art::Closure*)+324)
  #01  pc 00000000001c97c4  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ProcessMarkStackOnce()+960)
  #02  pc 00000000001c93ec  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ProcessMarkStack()+28)
  #03  pc 00000000001c2908  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::MarkingPhase()+1756)
  #04  pc 00000000001c0d68  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::RunPhases()+1176)
  #05  pc 00000000001d7018  /system/lib64/libart.so (art::gc::collector::GarbageCollector::Run(art::gc::GcCause, bool)+320)
  #06  pc 00000000001fcde8  /system/lib64/libart.so (art::gc::Heap::CollectGarbageInternal(art::gc::collector::GcType, art::gc::GcCause, bool)+3404)
  #07  pc 000000000020e404  /system/lib64/libart.so (art::gc::Heap::ConcurrentGC(art::Thread*, art::gc::GcCause, bool)+128)
  #08  pc 0000000000213aec  /system/lib64/libart.so (art::gc::Heap::ConcurrentGCTask::Run(art::Thread*)+40)
  #09  pc 0000000000235e9c  /system/lib64/libart.so (art::gc::TaskProcessor::RunAllTasks(art::Thread*)+68)
  #10  pc 0000000000092a6c  /system/framework/arm64/boot-core-libart.oat (offset 0x90000) (dalvik.system.VMRuntime.clampGrowthLimit [DEDUPED]+124)
  #11  pc 00000000001d79d4  /system/framework/arm64/boot-core-libart.oat (offset 0x90000) (java.lang.Daemons$HeapTaskDaemon.runInternal+196)
  #12  pc 000000000015978c  /system/framework/arm64/boot-core-libart.oat (offset 0x90000) (java.lang.Daemons$Daemon.run+76)
  #13  pc 00000000002c1038  /system/framework/arm64/boot.oat (offset 0x13b000) (java.lang.Thread.run+72)
  #14  pc 0000000000571d88  /system/lib64/libart.so (art_quick_invoke_stub+584)
  #15  pc 00000000000d4d2c  /system/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+200)
  #16  pc 0000000000475d3c  /system/lib64/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+104)
  #17  pc 0000000000476df8  /system/lib64/libart.so (art::InvokeVirtualOrInterfaceWithJValues(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, jvalue*)+424)
  #18  pc 00000000004a23ec  /system/lib64/libart.so (art::Thread::CreateCallback(void*)+1120)
  #19  pc 0000000000083588  /system/lib64/libc.so (__pthread_start(void*)+36)
  #20  pc 00000000000241dc  /system/lib64/libc.so (__start_thread+68)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
je_arena_malloc_hard


Wiko W_C200SN (W_C200SN), 1024MB RAM, Android 8.1
Report 1 of 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 0000000000057756  /system/lib/libc.so (je_arena_malloc_hard+201)
  #01  pc 0000000000065e8b  /system/lib/libc.so (a0ialloc+58)
  #02  pc 00000000000668bd  /system/lib/libc.so (je_malloc+320)
  #03  pc 0000000000081781  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libc++_shared.so (operator new(unsigned int)+24)
  #04  pc 00000000000817b1  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libc++_shared.so (operator new(unsigned int, std::nothrow_t const&)+4)
  #05  pc 0000000000520d8b  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libCore.so
  #06  pc 00000000005212a1  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libCore.so
  #07  pc 00000000000ed2e1  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libCore.so
  #08  pc 00000000000ed343  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libCore.so
  #09  pc 00000000000ed0b7  /data/app/air.com.company.app2-HOIf_QkuhUUyKg8FzcqIMw==/lib/arm/libCore.so
  #10  pc 0000000000047627  /system/lib/libc.so (__pthread_start(void*)+22)
  #11  pc 000000000001af9d  /system/lib/libc.so (__start_thread+32)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
arena_run_reg_alloc


Samsung Galaxy Tab A (2016) (gtaxllte), 2048MB RAM, Android 6.0
Report 1 of 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 0000000000049948  /system/lib/libc.so (arena_run_reg_alloc+87)
  #01  pc 000000000004b70f  /system/lib/libc.so (je_arena_tcache_fill_small+94)
  #02  pc 000000000005ca4b  /system/lib/libc.so (je_tcache_alloc_small_hard+18)
  #03  pc 000000000005753d  /system/lib/libc.so (je_malloc+1112)
  #04  pc 000000000051d269  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #05  pc 00000000000e25e7  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #06  pc 00000000002a3b0f  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #07  pc 000000000042cfe9  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #08  pc 000000000042dac3  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #09  pc 000000000042cf9b  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #10  pc 00000000000ed2ed  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #11  pc 00000000000ed343  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #12  pc 00000000000ed0b7  /data/app/air.com.company.app2-1/lib/arm/libCore.so
  #13  pc 000000000003fc73  /system/lib/libc.so (__pthread_start(void*)+30)
  #14  pc 000000000001a3ab  /system/lib/libc.so (__start_thread+6)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
ifree


Huawei P20 lite (HWANE), 3840MB RAM, Android 9
Report 1 of 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 00000000000a0954  /system/lib64/libc.so (ifree+88)
  #01  pc 00000000000a0edc  /system/lib64/libc.so (je_free+120)
  #02  pc 00000000007f874c  /data/app/air.com.company.app2-cHIQ0tuoGdkdpAdylEvgfQ==/lib/arm64/libCore.so
  #03  pc 00000000007f8b80  /data/app/air.com.company.app2-cHIQ0tuoGdkdpAdylEvgfQ==/lib/arm64/libCore.so
  #04  pc 00000000007b6b20  /data/app/air.com.company.app2-cHIQ0tuoGdkdpAdylEvgfQ==/lib/arm64/libCore.so
  #05  pc 00000000007b694c  /data/app/air.com.company.app2-cHIQ0tuoGdkdpAdylEvgfQ==/lib/arm64/libCore.so
  #06  pc 00000000007b6774  /data/app/air.com.company.app2-cHIQ0tuoGdkdpAdylEvgfQ==/lib/arm64/libCore.so
  #07  pc 00000000007b61a0  /data/app/air.com.company.app2-cHIQ0tuoGdkdpAdylEvgfQ==/lib/arm64/libCore.so
  #08  pc 00000000000051b4  <anonymous>

signal 11 (SIGSEGV), code 2 (SEGV_ACCERR)
art::JNIEnvExt::JNIEnvExt(art::Thread*, art::JavaVMExt*, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator>*)


Huawei FIG-LX1 (HWFIG-H), 2816MB RAM, Android 9
Report 1 of 2

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app2 <<<

backtrace:
  #00  pc 000000000030ae34  /system/lib64/libart.so (art::JNIEnvExt::JNIEnvExt(art::Thread*, art::JavaVMExt*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*)+28)
  #01  pc 000000000030adb4  /system/lib64/libart.so (art::JNIEnvExt::Create(art::Thread*, art::JavaVMExt*, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>*)+52)
  #02  pc 00000000004a2ac8  /system/lib64/libart.so (art::Thread::Init(art::ThreadList*, art::JavaVMExt*, art::JNIEnvExt*)+344)
  #03  pc 00000000004a4db8  /system/lib64/libart.so (art::Thread::Attach(char const*, bool, _jobject*, bool)+452)
  #04  pc 000000000047c7bc  /system/lib64/libart.so (art::Runtime::AttachCurrentThread(char const*, bool, _jobject*, bool)+112)
  #05  pc 00000000002f4708  /system/lib64/libart.so (art::JII::AttachCurrentThreadInternal(_JavaVM*, _JNIEnv**, void*, bool)+244)
  #06  pc 000000000026ef00  /data/app/air.com.company.app2-j_F-pNDIrZBq7F2J0M5-1w==/lib/arm64/libCore.so
  #07  pc 0000000000272958  /data/app/air.com.company.app2-j_F-pNDIrZBq7F2J0M5-1w==/lib/arm64/libCore.so
  #08  pc 00000000002729d0  /data/app/air.com.company.app2-j_F-pNDIrZBq7F2J0M5-1w==/lib/arm64/libCore.so
  #09  pc 0000000000272704  /data/app/air.com.company.app2-j_F-pNDIrZBq7F2J0M5-1w==/lib/arm64/libCore.so
  #10  pc 0000000000083588  /system/lib64/libc.so (__pthread_start(void*)+36)
  #11  pc 00000000000241dc  /system/lib64/libc.so (__start_thread+68)
@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Mar 14, 2020

Wow, thanks for all the details there! we'll go through some of those to try to see what we can do to protect from the crashes..

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Mar 25, 2020

Hi @ajwfrost this issue is getting out of hand. The last app we updated now has a 9.5% crash rate! This really needs to be resolved urgently. It's compromising all our marketing efforts.

Please understand that most of your clients like us are long term AIR users. We don't care about features, we don't need any new features, ANEs can handle those. We need stability, we need bug fixes and we need store guideline deadlines to be met. Please focus on these points, AIR 33.1 is making us nervous.. :(

Last version AIR 33.1.0.63

image

image

Previous version AIR 32.0.0.116

image

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Mar 25, 2020

Here is the crash report for the most regular crash (strcmp):

image

Here is the entire developer console crash list, if there are any ones in here you'd like me to detail, please ask:

image

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Mar 25, 2020

Hi

Thanks for this - we've been looking into this and can see where it's crashing - but we can't actually see why there would be a problem. The code all looks fine and has null-checks and appears thread-safe. There is obviously something happening here which is causing a problem but we could do with being able to reproduce it ourselves so that we can dig deeper..

From the logs and call stacks we can see what sort of scenarios are happening when the crashes happen - so I'm assuming you're doing a lot of accesses to resources that are obtained via an "app://" URL (i.e. from the application folder) which are asynchronous. Am just wondering whether any of them are also synchronous accesses.. but I can't see how a mixture of the two would be a problem. However our working assumption currently is that there is something wrong with the memory accesses when the elements have been created in various threads and are added into a 'master' cache. We'll keep investigating, but unfortunately I don't think there's an immediate fix coming.

Was there an earlier version where the rate was lower? 32.0.0.116 still looks quite a bad crash rate to me..

thanks

Andrew

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Mar 25, 2020

Hi Andrew,

Thanks for your prompt reply!

Indeed this is the most resource demanding app we've ever made, loading 100+ atfs, sounds and other files. It has a huge scrollable map with lots and lots of images loaded from the application/storage directories. ATFs are the most loaded file type and we are reading files asynchronously to load them since it is a bit faster and less resource intensive than using urlloaders. There are also lots of File.exists checks to cancel the load if the file is missing and I guess the exists() is synchronous.

Our largest app still runs on 32.0.0.100 (we have been delaying updating to avoid this issue) and has a 0.13% crash rate for the last 30 days which is negligible. Something happened after 32.0.0.100 (we tried many 32 versions before staying on 116) that introduced these crashes.

One remark though, there was no big change in the number of resources loaded between our two app versions (5.27 on 32.0.0.116 and 5.39 on 33.1.1.63) so the increase in crash rate is weird if it's a async/sync file access issue.

We will try skipping the File.exists checks in next release and report back our findings!

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Mar 25, 2020

Okay thanks, that's an interesting clue as there was a fix that went into the async file handling code around that time - we can go back through the changes to see whether there's anything that should impact this (from memory it was mostly around synchronisation where there had been some deadlock in a background thread that prevented some of the progress events when loading large files.. so not really in the same area as where these crashes are happening, but it's suspicious enough..)

thanks

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Mar 26, 2020

Hi @ajwfrost

Please find some more stack traces from another of our apps which is smaller and has a very reasonable amount of files to load (probably 20-30 all file types combined). Also built with 33.1.1.63 and we also noticed an increase in the crash rate moving from 5.47% (not shown on the screenshot) to now up to 11.21% of sessions....

image

The strcmp error is also the most frequent but I don't see anything new in the trace. Here are some new traces though.

Please let me know if there is anything else we could provide you with. I know I keep on adding things here, I'm not trying to be annoying, just adding the info as it comes :)

signal 11 (SIGSEGV), code 2 (SEGV_ACCERR)
je_tcache_bin_flush_small

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 000000000006588c  /apex/com.android.runtime/lib64/bionic/libc.so (je_tcache_bin_flush_small+264)
  #01  pc 000000000006681c  /apex/com.android.runtime/lib64/bionic/libc.so (tcache_destroy+304)
  #02  pc 00000000000666d8  /apex/com.android.runtime/lib64/bionic/libc.so (je_tcache_cleanup+24)
  #03  pc 0000000000066f84  /apex/com.android.runtime/lib64/bionic/libc.so (je_tsd_cleanup+64)
  #04  pc 0000000000067dd8  /apex/com.android.runtime/lib64/bionic/libc.so (je_tsd_cleanup_wrapper+32)
  #05  pc 00000000000ce850  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_key_clean_all()+116)
  #06  pc 00000000000ce2b4  /apex/com.android.runtime/lib64/bionic/libc.so (pthread_exit+72)
  #07  pc 00000000000ce194  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+40)
  #08  pc 0000000000070ba8  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+64) 

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
je_arena_tcache_fill_small

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 0000000000057dee  /system/lib/libc.so (je_arena_tcache_fill_small+281)
  #01  pc 0000000000078141  /system/lib/libc.so (je_tcache_alloc_small_hard+16)
  #02  pc 000000000006922f  /system/lib/libc.so (je_malloc+630)
  #03  pc 0000000000010d0b  /system/lib/libopenjdk.so (newEntry+190)
  #04  pc 0000000000010fc7  /system/lib/libopenjdk.so (ZIP_GetNextEntry+32)
  #05  pc 000000000000eda3  /system/lib/libopenjdk.so (ZipFile_getNextEntry+6)
  #06  pc 000000000052e4b1  /system/framework/arm/boot.oat (offset 0x1cc000) (java.util.zip.ZipFile.getNextEntry [DEDUPED]+112)
  #07  pc 00000000005ab795  /system/framework/arm/boot.oat (offset 0x1cc000) (java.util.zip.ZipFile$ZipEntryIterator.next+284)
  #08  pc 000000000052c2ef  /system/framework/arm/boot.oat (offset 0x1cc000) (java.util.jar.JarFile$JarEntryIterator.next [DEDUPED]+38)
  #09  pc 0000000000357e27  /system/framework/arm/boot.oat (offset 0x1cc000) (java.security.cert.PKIXRevocationChecker.clone [DEDUPED]+38)
  #10  pc 0000000000095153  /data/app/air.com.company.app-Fw2-AH2cwlekNkuyQK43cA==/oat/arm/base.odex (offset 0x86000)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
pthread_mutex_lock

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 0000000000084020  /system/lib64/libc.so (pthread_mutex_lock)
  #01  pc 00000000000aaf08  /system/lib64/libc.so (je_tcache_bin_flush_small+176)
  #02  pc 00000000000abe08  /system/lib64/libc.so (tcache_destroy+300)
  #03  pc 00000000000abcc8  /system/lib64/libc.so (je_tcache_cleanup+28)
  #04  pc 00000000000ac5d0  /system/lib64/libc.so (je_tsd_cleanup+60)
  #05  pc 00000000000ad15c  /system/lib64/libc.so (je_tsd_cleanup_wrapper+32)
  #06  pc 0000000000083c38  /system/lib64/libc.so (pthread_key_clean_all()+120)
  #07  pc 00000000000836a8  /system/lib64/libc.so (pthread_exit+72)
  #08  pc 000000000008358c  /system/lib64/libc.so (__pthread_start(void*)+40)
  #09  pc 00000000000241dc  /system/lib64/libc.so (__start_thread+68)

signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)
art::gc::collector::ConcurrentCopying::ProcessThreadLocalMarkStacks(bool, art::Closure
)
*

*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 00000000001c6e98  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ProcessThreadLocalMarkStacks(bool, art::Closure*)+320)
  #01  pc 00000000001c6a58  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ProcessMarkStackOnce()+968)
  #02  pc 00000000001c6674  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::ProcessMarkStack()+28)
  #03  pc 00000000001bfa64  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::MarkingPhase()+1748)
  #04  pc 00000000001bde54  /system/lib64/libart.so (art::gc::collector::ConcurrentCopying::RunPhases()+1340)
  #05  pc 00000000001d44ac  /system/lib64/libart.so (art::gc::collector::GarbageCollector::Run(art::gc::GcCause, bool)+316)
  #06  pc 00000000001f73d4  /system/lib64/libart.so (art::gc::Heap::CollectGarbageInternal(art::gc::collector::GcType, art::gc::GcCause, bool)+3428)
  #07  pc 0000000000208d00  /system/lib64/libart.so (art::gc::Heap::ConcurrentGC(art::Thread*, art::gc::GcCause, bool)+128)
  #08  pc 000000000020e450  /system/lib64/libart.so (art::gc::Heap::ConcurrentGCTask::Run(art::Thread*)+40)
  #09  pc 000000000023136c  /system/lib64/libart.so (art::gc::TaskProcessor::RunAllTasks(art::Thread*)+68)
  #10  pc 000000000007b56c  /system/framework/arm64/boot-core-libart.oat (offset 0x79000) (dalvik.system.VMRuntime.clampGrowthLimit [DEDUPED]+124)
  #11  pc 00000000001845d4  /system/framework/arm64/boot-core-libart.oat (offset 0x79000) (java.lang.Daemons$HeapTaskDaemon.runInternal+196)
  #12  pc 0000000000111e7c  /system/framework/arm64/boot-core-libart.oat (offset 0x79000) (java.lang.Daemons$Daemon.run+76)
  #13  pc 000000000025e068  /system/framework/arm64/boot.oat (offset 0x115000) (java.lang.Thread.run+72)
  #14  pc 0000000000561f88  /system/lib64/libart.so (art_quick_invoke_stub+584)
  #15  pc 00000000000d0140  /system/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+200)
  #16  pc 00000000004682e0  /system/lib64/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+104)
  #17  pc 00000000004693a8  /system/lib64/libart.so (art::InvokeVirtualOrInterfaceWithJValues(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, jvalue*)+424)
  #18  pc 0000000000494ca0  /system/lib64/libart.so (art::Thread::CreateCallback(void*)+1120)
  #19  pc 0000000000084dc0  /system/lib64/libc.so (__pthread_start(void*)+208)
  #20  pc 0000000000023a4c  /system/lib64/libc.so (__start_thread+68)

signal 7 (SIGBUS), code 1 (BUS_ADRALN)
libGLESv2_adreno.so


*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
pid: 0, tid: 0 >>> air.com.company.app <<<

backtrace:
  #00  pc 000000000039c1d8  /vendor/lib64/egl/libGLESv2_adreno.so
  #01  pc 00000000001d90d0  /vendor/lib64/egl/libGLESv2_adreno.so
  #02  pc 000000000039c0b4  /vendor/lib64/egl/libGLESv2_adreno.so
  #03  pc 00000000001f1538  /vendor/lib64/egl/libGLESv2_adreno.so
  #04  pc 00000000001c7f5c  /vendor/lib64/egl/libGLESv2_adreno.so
  #05  pc 0000000000153348  /vendor/lib64/egl/libGLESv2_adreno.so
  #06  pc 000000000033c5c4  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #07  pc 000000000033c710  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #08  pc 000000000033c77c  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #09  pc 000000000032de94  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #10  pc 0000000000341160  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #11  pc 0000000000264840  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #12  pc 0000000000336d68  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #13  pc 00000000002bbd10  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #14  pc 00000000002bc9c4  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #15  pc 00000000002bb404  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #16  pc 00000000002bcb08  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #17  pc 00000000002c058c  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #18  pc 00000000003634b0  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #19  pc 0000000000363f50  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #20  pc 0000000000363e6c  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #21  pc 00000000003600f0  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #22  pc 000000000048f0f0  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #23  pc 00000000004b0ffc  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #24  pc 00000000004b1be4  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #25  pc 000000000025fdc0  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #26  pc 0000000000263104  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #27  pc 0000000000266828  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/lib/arm64/libCore.so
  #28  pc 00000000000960bc  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/oat/arm64/base.odex (art_jni_trampoline+140)
  #29  pc 00000000000eca14  /data/app/air.com.company.app-vEzQ8-QcG0nPRPKthFtgLg==/oat/arm64/base.odex (com.adobe.air.customHandler.handleMessage+68)
  #30  pc 0000000000736764  /system/framework/arm64/boot-framework.oat (android.os.Handler.dispatchMessage+180)
  #31  pc 0000000000739db4  /system/framework/arm64/boot-framework.oat (android.os.Looper.loop+1748)
  #32  pc 00000000004c6c24  /system/framework/arm64/boot-framework.oat (android.app.ActivityThread.main+756)
  #33  pc 00000000001375b8  /apex/com.android.runtime/lib64/libart.so (art_quick_invoke_static_stub+568)
  #34  pc 000000000014600c  /apex/com.android.runtime/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+276)
  #35  pc 00000000004b0d28  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+104)
  #36  pc 00000000004b28cc  /apex/com.android.runtime/lib64/libart.so (art::InvokeMethod(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jobject*, _jobject*, unsigned long)+1480)
  #37  pc 000000000043db64  /apex/com.android.runtime/lib64/libart.so (art::Method_invoke(_JNIEnv*, _jobject*, _jobject*, _jobjectArray*)+52)
  #38  pc 00000000000c2b54  /system/framework/arm64/boot.oat (art_jni_trampoline+180)
  #39  pc 00000000009b0118  /system/framework/arm64/boot-framework.oat (com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run+136)
  #40  pc 00000000009b7cec  /system/framework/arm64/boot-framework.oat (com.android.internal.os.ZygoteInit.main+2076)
  #41  pc 00000000001375b8  /apex/com.android.runtime/lib64/libart.so (art_quick_invoke_static_stub+568)
  #42  pc 000000000014600c  /apex/com.android.runtime/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+276)
  #43  pc 00000000004b0d28  /apex/com.android.runtime/lib64/libart.so (art::(anonymous namespace)::InvokeWithArgArray(art::ScopedObjectAccessAlreadyRunnable const&, art::ArtMethod*, art::(anonymous namespace)::ArgArray*, art::JValue*, char const*)+104)
  #44  pc 00000000004b0918  /apex/com.android.runtime/lib64/libart.so (art::InvokeWithVarArgs(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, std::__va_list)+408)
  #45  pc 00000000003bb5a8  /apex/com.android.runtime/lib64/libart.so (art::JNI::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, std::__va_list)+624)
  #46  pc 00000000000bf99c  /system/lib64/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+116)
  #47  pc 00000000000c282c  /system/lib64/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vector<android::String8> const&, bool)+776)
  #48  pc 00000000000035b0  /system/bin/app_process64 (main+1376)
  #49  pc 000000000007e898  /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+108)

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Apr 3, 2020

Hi @ajwfrost

I have new data on this topic:

  1. One of our test devices has crashed with several signal 11 and a signal 6, I will email you the log file today. Hopefully this will be useful!

  2. Some new information on our app with lots of files to load: the higher crash rates seems to mostly impact the 64 bit build, it's seems it's 2x worse for 64bit:

Version 5.39.1 is 64bit
Version 5.39 is 32bit

image

Here are screenshots of the crash report lists:

64bit:
64

32bit:
32

@Dima107

This comment has been minimized.

Copy link

@Dima107 Dima107 commented Apr 3, 2020

Hi. We have similar problems with our app.
Have you any recipe how to minimize ANR rate?

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Apr 3, 2020

Hi @Dima107 so far, we have not found any workaround.
If you have such errors occur on one of your devices, please send the log files to adobe.support@harman.com so that @ajwfrost can investigate.
Otherwise, if you have any detailed stack traces in your Developer Console that are not listed here, please add them :)

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Apr 5, 2020

Thanks for all the inputs! So the main thing still seems to be the strcmp .. wondering whether you'll see an improvement from the change we put into the 33.1.1.86 release but if this is GC-related then that may not help, we need to do some more investigations to be able to reproduce this here.

Wondering too, if anyone is using shared bytearrays between workers, there is an ARMv8-specific problem with that currently. That may explain the difference in overall instabilities between 32-bit and 64-bit..? If you are seeing this, please try to avoid changing the length (or calling clear) on any shared bytearray..

thanks

@neuronix

This comment has been minimized.

Copy link
Author

@neuronix neuronix commented Apr 5, 2020

Hi @ajwfrost we will release with .1.86 in a week or so, so i'll be able to provide data in 2-3 weeks time.

Hopefully we might have sooner results with our crashing test device, will report ASAP! Did you receive the log file by email? Is it more useful than the stack traces?

Regarding workers and shared bytearrays we do not use them at all as far as I know, The only code we have with bytearrays is when loading binary files asynchronously (ATFs) and the are some in the Starling internals for loading shaders unless i'm mistaken.

Thanks for your support on this issue!

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Apr 5, 2020

Great @neuronix thanks - yes, received the log file and having the register information for the crash is really helpful; if you're able to pull the tombstone files off the device as well it would be great as they provide more context with memory dumps around the locations of each register..

cheers

@marcanw

This comment has been minimized.

Copy link

@marcanw marcanw commented Apr 5, 2020

Here is mine
image

@ajwfrost

This comment has been minimized.

Copy link
Collaborator

@ajwfrost ajwfrost commented Apr 6, 2020

@neuronix so with your debug file we can see what's going wrong when we hit the strcmp issue (and the other native stacks that are similar) and it appears to be because there's a hash table being expanded at the same time as another thread is accessing it. The synchronisation that we added into .86 should then solve this.

There's another issue when it's reading a zip file via the Java APIs, we're not clear on why this may be a problem yet unless Java is trying to be efficient and sharing resources across threads..

@marcanw can you please confirm which sort of sensor events you're using? This is crashing when removing a sensor event listener but the variable that's invalid here shouldn't ever be invalid! Do you have any other details on this one (register values or a tombstone dump)?

thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.