Crash in JNI method CatalystInstanceImpl:: getJSCallInvokerHolder()

[johanblumenberg]

Description

We see this crash now and then:

2023-02-18T05:19:41.587Z 21829 21829 I crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstone
2023-02-18T05:19:41.596Z   336   336 I tombstoned: received crash request for pid 21824
2023-02-18T05:19:41.607Z 21829 21829 I crash_dump64: performing dump of process 21792 (target tid = 21824)
2023-02-18T05:19:41.618Z 21829 21829 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
2023-02-18T05:19:41.618Z 21829 21829 F DEBUG   : Build fingerprint: 'google/sdk_gphone_x86_64/generic_x86_64_arm64:11/RSR1.201211.001/7027799:user/release-keys'
2023-02-18T05:19:41.618Z 21829 21829 F DEBUG   : Revision: '0'
2023-02-18T05:19:41.618Z 21829 21829 F DEBUG   : ABI: 'x86_64'
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   : Timestamp: 2023-02-18 05:19:41+0000
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   : pid: 21792, tid: 21824, name: pool-2-thread-1  >>> me.veritru <<<
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   : uid: 10195
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   : signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   : Abort message: 'JNI ERROR (app bug): attempt to use stale Global 0x2d5a (should be 0x2d52)'
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   :     rax 0000000000000000  rbx 0000000000005520  rcx 0000776fb09662a8  rdx 0000000000000006
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   :     r8  0000000000000000  r9  0000000000000000  r10 0000776cc5ec41b0  r11 0000000000000246
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   :     r12 0000000000000002  r13 0000776cc5ec4280  r14 0000776cc5ec41a8  r15 0000000000005540
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   :     rdi 0000000000005520  rsi 0000000000005540
2023-02-18T05:19:41.619Z 21829 21829 F DEBUG   :     rbp 0000776ddfae5470  rsp 0000776cc5ec4198  rip 0000776fb09662a8
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   : backtrace:
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #00 pc 000000000005a2a8  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+24) (BuildId: 3707c39fc397eeaa328142d90b50a973)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #01 pc 000000000005d212  /apex/com.android.runtime/lib64/bionic/libc.so (abort+194) (BuildId: 3707c39fc397eeaa328142d90b50a973)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #02 pc 0000000000634927  /apex/com.android.art/lib64/libart.so (art::Runtime::Abort(char const*)+2375) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #03 pc 0000000000019cfc  /system/lib64/libbase.so (android::base::SetAborter(std::__1::function<void (char const*)>&&)::$_3::__invoke(char const*)+60) (BuildId: 7101d4565a51dea09dc23901546cbb64)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #04 pc 00000000000192a0  /system/lib64/libbase.so (android::base::LogMessage::~LogMessage()+368) (BuildId: 7101d4565a51dea09dc23901546cbb64)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #05 pc 0000000000385ffa  /apex/com.android.art/lib64/libart.so (art::IndirectReferenceTable::AbortIfNoCheckJNI(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&)+218) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #06 pc 00000000004297f2  /apex/com.android.art/lib64/libart.so (art::IndirectReferenceTable::GetChecked(void*) const+482) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #07 pc 00000000004244e2  /apex/com.android.art/lib64/libart.so (art::JavaVMExt::DecodeGlobal(void*)+18) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #08 pc 000000000069cf42  /apex/com.android.art/lib64/libart.so (art::Thread::DecodeJObject(_jobject*) const+162) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #09 pc 0000000000775137  /apex/com.android.art/lib64/libart.so (art::JniMethodEndWithReferenceHandleResult(_jobject*, unsigned int, art::Thread*)+55) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #10 pc 000000000005770e  /data/app/~~aPdyXeiSPGvKTmWyxXHvaQ==/me.veritru-k7vTjxpVB5bNGcC3vuGs2A==/oat/x86_64/base.odex (art_jni_trampoline+174)
2023-02-18T05:19:41.919Z 21829 21829 F DEBUG   :       #11 pc 00000000001713dc  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #12 pc 00000000003ebb74  /data/app/~~aPdyXeiSPGvKTmWyxXHvaQ==/me.veritru-k7vTjxpVB5bNGcC3vuGs2A==/base.apk (com.facebook.react.bridge.CatalystInstanceImpl.getJSCallInvokerHolder)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #13 pc 0000000000171f7a  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #14 pc 00000000002df486  /data/app/~~aPdyXeiSPGvKTmWyxXHvaQ==/me.veritru-k7vTjxpVB5bNGcC3vuGs2A==/base.apk (offset 0x8a3000) (com.mrousavy.camera.frameprocessor.FrameProcessorRuntimeManager.<init>+42)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #15 pc 0000000000170d5d  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #16 pc 00000000002d9d78  /data/app/~~aPdyXeiSPGvKTmWyxXHvaQ==/me.veritru-k7vTjxpVB5bNGcC3vuGs2A==/base.apk (offset 0x8a3000) (com.mrousavy.camera.CameraViewModule.initialize$lambda-0+40)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #17 pc 00000000001700d1  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #18 pc 00000000002d9b38  /data/app/~~aPdyXeiSPGvKTmWyxXHvaQ==/me.veritru-k7vTjxpVB5bNGcC3vuGs2A==/base.apk (offset 0x8a3000) (com.mrousavy.camera.CameraViewModule.$r8$lambda$YiRnUrObfbO8oH48lnR0UOsrros)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #19 pc 00000000001700d1  /apex/com.android.art/lib64/libart.so (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #20 pc 00000000002d813c  /data/app/~~aPdyXeiSPGvKTmWyxXHvaQ==/me.veritru-k7vTjxpVB5bNGcC3vuGs2A==/base.apk (offset 0x8a3000) (com.mrousavy.camera.CameraViewModule$$ExternalSyntheticLambda2.run+4)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #21 pc 00000000002e681f  /apex/com.android.art/javalib/x86_64/boot.oat (java.util.concurrent.ThreadPoolExecutor.runWorker+1103) (BuildId: 063a75624879be7d4274ea340748b04abbd86b8d)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #22 pc 00000000002e3433  /apex/com.android.art/javalib/x86_64/boot.oat (java.util.concurrent.ThreadPoolExecutor$Worker.run+67) (BuildId: 063a75624879be7d4274ea340748b04abbd86b8d)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #23 pc 0000000000177d81  /apex/com.android.art/javalib/x86_64/boot.oat (java.lang.Thread.run+81) (BuildId: 063a75624879be7d4274ea340748b04abbd86b8d)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #24 pc 0000000000178cb4  /apex/com.android.art/lib64/libart.so (art_quick_invoke_stub+756) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #25 pc 000000000020ba92  /apex/com.android.art/lib64/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+242) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #26 pc 000000000062a1be  /apex/com.android.art/lib64/libart.so (art::JValue art::InvokeVirtualOrInterfaceWithJValues<art::ArtMethod*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, art::ArtMethod*, jvalue const*)+478) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #27 pc 000000000068d843  /apex/com.android.art/lib64/libart.so (art::Thread::CreateCallback(void*)+1411) (BuildId: 7fbaf2a1a3317bd634b00eb90e32291e)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #28 pc 00000000000c7d2a  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+58) (BuildId: 3707c39fc397eeaa328142d90b50a973)
2023-02-18T05:19:41.920Z 21829 21829 F DEBUG   :       #29 pc 000000000005f0c7  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+55) (BuildId: 3707c39fc397eeaa328142d90b50a973)

I interpet this message as there is a stale global object reference that was used. Perhaps it has been deallocated.

JNI ERROR (app bug): attempt to use stale Global 0x2d5a (should be 0x2d52)

This is the interesting parts of the stack trace:

libc.so (abort+194)
libart.so (art::Runtime::Abort(char const*)+2375)
libbase.so (android::base::SetAborter()
libbase.so (android::base::LogMessage::~LogMessage()+368)
libart.so (art::IndirectReferenceTable::AbortIfNoCheckJNI()+218)
libart.so (art::IndirectReferenceTable::GetChecked() const+482)
libart.so (art::JavaVMExt::DecodeGlobal()+18)
libart.so (art::Thread::DecodeJObject(_jobject*) const+162)
libart.so (art::JniMethodEndWithReferenceHandleResult(_jobject*)+55)
base.apk (com.facebook.react.bridge.CatalystInstanceImpl.getJSCallInvokerHolder)
base.apk (com.mrousavy.camera.frameprocessor.FrameProcessorRuntimeManager.<init>+42)

FrameProcessorRuntimeManager constructor calls getJSCallInvokerHolder() which is a native method on CatalystInstanceImpl.
JniMethodEndWithReferenceHandleResult() indicates that this method is trying to return a reference to an object.
There is something wrong with this object to be returned, and the JVM calls abort().

This is the getJSCallInvokerHolder() method:

react-native/ReactAndroid/src/main/jni/react/jni/CatalystInstanceImpl.cpp

Line 374 in bcec590

CatalystInstanceImpl::getJSCallInvokerHolder() {

CatalystInstanceImpl::getJSCallInvokerHolder() {
  if (!jsCallInvokerHolder_) {
    auto runtimeScheduler = getRuntimeScheduler();
    auto runtimeSchedulerCallInvoker =
        std::make_shared<RuntimeSchedulerCallInvoker>(
            runtimeScheduler->cthis()->get());
    jsCallInvokerHolder_ = jni::make_global(
        CallInvokerHolder::newObjectCxxArgs(runtimeSchedulerCallInvoker));
  }
  return jsCallInvokerHolder_;
}

jsCallInvokerHolder_ is a smart pointer holding a reference to what looks to be a singleton.
If two threads call this method at the same time, there is a race and possibly a crash. Both threads will find that the reference is null, and both threads will go on and create an object. Then both threads will assign the global jsCallInvokerHolder_ reference, the later will override the first. The first object will be deallocated, but the first thread will continue and try to return the deallocated object to the caller.

The calling code is located in react-native-vision-camera: https://github.com/mrousavy/react-native-vision-camera/blob/06cbb742fbab331bfec3f6fce83d4751fb8dc2bc/android/src/main/java/com/mrousavy/camera/CameraViewModule.kt#L66

It executes some initialization code on a thread pool, that calls CatalystInstanceImpl.getJSCallInvokerHolder().

React Native Version

0.70.6

Output of npx react-native info

info Fetching system and libraries information...
System:
OS: macOS 13.2.1
CPU: (8) arm64 Apple M1
Memory: 58.53 MB / 16.00 GB
Shell: 5.8.1 - /bin/zsh
Binaries:
Node: 16.13.0 - ~/.nvm/versions/node/v16.13.0/bin/node
Yarn: 1.22.19 - ~/.yvm/shim/yarn
npm: 8.1.0 - ~/.nvm/versions/node/v16.13.0/bin/npm
Watchman: Not Found
Managers:
CocoaPods: Not Found
SDKs:
iOS SDK:
Platforms: DriverKit 22.2, iOS 16.2, macOS 13.1, tvOS 16.1, watchOS 9.1
Android SDK:
API Levels: 30, 31, 33
Build Tools: 30.0.2, 30.0.3, 31.0.0
System Images: android-30 | Google Play ARM 64 v8a, android-33 | Google Play ARM 64 v8a
Android NDK: Not Found
IDEs:
Android Studio: Not Found
Xcode: 14.2/14C18 - /usr/bin/xcodebuild
Languages:
Java: 17.0.1 - /usr/bin/javac
npmPackages:
@react-native-community/cli: Not Found
react: Not Found
react-native: Not Found
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found

Steps to reproduce

N/A

Snack, code example, screenshot, or link to a repository

N/A

[github-actions]

⚠️

Newer Version of React Native is Available!

ℹ️

You are on a supported minor version, but it looks like there's a newer patch available. Please upgrade to the highest patch for your minor or latest and verify if the issue persists (alternatively, create a new project and repro the issue in it). If it does not repro, please let us know so we can close out this issue. This helps us ensure we are looking at issues that still exist in the most recent releases.

[cortinico]

The calling code is located in react-native-vision-camera

Have you reporeted this also to react-native-vision-camera issue tracker (cc @mrousavy)?

[johanblumenberg]

The calling code is located in react-native-vision-camera

Have you reporeted this also to react-native-vision-camera issue tracker (cc @mrousavy)?

No, since the crash is caused by code in react-native I did not report any bug to react-native-vision-camera. Is there any problem with how react-native-vision-camera uses react-native?

[cortinico]

No, since the crash is caused by code in react-native I did not report any bug to react-native-vision-camera. Is there any problem with how react-native-vision-camera uses react-native?

I'm unsure but crashes that happens on specific libraries, should be reported to library authors. They have a better understanding on how the library interacts with the core of React Native and can point back to us, if the issue is inside the core.

[johanblumenberg]

@cortinico I'm quite sure that the code that I pointed to in react-native will crash now and then for any library that calls getJSCallInvokerHolder(), because the code that constructs the singleton is obviously not thread safe.

[johanblumenberg]

Some more details about this bug:

The method com.mrousavy.camera.frameprocessor.FrameProcessorRuntimeManager() is calling getJSCallInvokerHolder() on thread pool-2-thread-1:
https://github.com/mrousavy/react-native-vision-camera/blob/06cbb742fbab331bfec3f6fce83d4751fb8dc2bc/android/src/main/java/com/mrousavy/camera/CameraViewModule.kt#L66

The method com.swmansion.reanimated.NativeProxy() calling getJSCallInvokerHolder() on thread mqt_js:
https://github.com/software-mansion/react-native-reanimated/blob/7029f160220d37164c6da15ba943cc6bb48252e6/android/src/paper/java/com/swmansion/reanimated/NativeProxy.java#L25

Since these two invocations are both done at app start and on different threads, there is nothing that prevents the race condition in getJSCallInvokerHolder(), and this will crash every now and then. The same thing will happen for any app or library that calls getJSCallInvokerHolder(), if the app contains multiple libraries that try to call getJSCallInvokerHolder().