-
-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Reanimated v3 support in RNVisionCamera v2 #1759
Conversation
…lude_directories` in CMakeLists.txt
…radle Sync issue
LGTM!!!!!!!! @tomekzaw ❤️ 🚀 |
Thank you! |
hey @tomekzaw, fantastic work, thank you. I'm using rean@3.5.2 and I get note 1:
otherwise I get this error: note 2: Let me know if you want me to open an issue. |
Yes, @tomekzaw thanks for all your work. 🙂 Unfortunately, I’m unable to build on Android after this change (v2.15.6 works as intended). First, had to add
After that, the build still failed:
Didn’t try iOS. Likewise to the above comment, please let me know if you’d like me to open an issue. |
Hey @efstathiosntonas and @short-dsb, thanks for testing out the changes! Could you please check if these PRs resolve your issues? |
@tomekzaw works as a charm on iOS, on Android I get this: (rn@0.72.4, kotlinVersion: 1.8.22)
edit: If I disable |
@tomekzaw after commenting out these lines it builts successfully on Android (at least on my setup): |
Ah whoops yea I forgot that this should still be optional, my bad. Do we not need the kotlin extensions? Lemme try to build without |
Hi! @tomekzaw and @mrousavy thank you for this PR. Unfortunately I'm getting the following error Am I doing something wrong? Do you guys have any idea how to solve this?
|
@vadbertalan Thanks for reporting this problem. Just out of curiosity, could you please check if the issue persists when Hermes is enabled (instead of JSC)? |
@tomekzaw I can confirm that this is a JSC only crash. By enabling Hermes, the frame processor is providing the frames as intended. |
Hi! After cleaning the Android project, we get the next error: Execution failed for task ':react-native-vision-camera:buildCMakeDebug[arm64-v8a]'.
> com.android.ide.common.process.ProcessException: ninja: Entering directory `/Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vision-camera/android/.cxx/Debug/1t3p5vh3/arm64-v8a'
[1/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JHashMap.cpp.o
[2/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JFrameProcessorPlugin.cpp.o
[3/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JImageProxy.cpp.o
[4/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/VisionCameraScheduler.cpp.o
[5/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/VisionCamera.cpp.o
[6/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/FrameHostObject.cpp.o
[7/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o
[8/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/CameraView.cpp.o
[9/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/JSIJNIConversion.cpp.o
[10/10] Linking CXX shared library ../../../../build/intermediates/cxx/Debug/1t3p5vh3/obj/arm64-v8a/libVisionCamera.so
FAILED: ../../../../build/intermediates/cxx/Debug/1t3p5vh3/obj/arm64-v8a/libVisionCamera.so
: && However, if we change the Reanimated version to a different one (e.g., from 3.5.3 to 3.5.4) after encountering the build failure and then rebuild the project, the app compiles successfully. We are using: "react-native": "^0.72.5",
"react-native-vision-camera": "2.16.1",
"react-native-worklets-core": "^0.2.0",
"vision-camera-code-scanner": "^0.2.0",
"react-native-reanimated": "3.5.3", @tomekzaw @mrousavy , do you have any insights into what might be causing this issue? |
@denisbevilacqua What's the actual error? It seems like there should be some more info after "Linking CXX shared library" |
@tomekzaw Yes, I'm sorry, the complete error is: > Task :react-native-vision-camera:configureCMakeDebug[arm64-v8a]
C/C++: CMake Warning at CMakeLists.txt:216 (message):
C/C++: VisionCamera linking: FOR_HERMES=true
> Task :react-native-vision-camera:buildCMakeDebug[arm64-v8a] FAILED
C/C++: ninja: Entering directory `/Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vision-camera/android/.cxx/Debug/93z355m5/arm64-v8a'
C/C++: ld: error: undefined symbol: reanimated::extractWorkletRuntime(facebook::jsi::Runtime&, facebook::jsi::Value const&)
C/C++: ld: error: undefined symbol: reanimated::extractShareableOrThrow(facebook::jsi::Runtime&, facebook::jsi::Value const&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)
C/C++: ld: error: undefined symbol: typeinfo for reanimated::Shareable
C/C++: ld: error: undefined symbol: typeinfo for reanimated::ShareableWorklet
C/C++: clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
w: Detected multiple Kotlin daemon sessions at build/kotlin/sessions
Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.
You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.
See https://docs.gradle.org/8.0.1/userguide/command_line_interface.html#sec:command_line_warnings
229 actionable tasks: 222 executed, 7 up-to-date
info 💡 Tip: Make sure that you have set up your development environment correctly, by running react-native doctor. To read more about doctor command visit: https://github.com/react-native-community/cli/blob/main/packages/cli-doctor/README.md#doctor
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: /Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-svg/android/src/main/java/com/horcrux/svg/VirtualView.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: /Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vector-icons/android/src/newarch/java/com/oblador/vectoricons/VectorIconsModule.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':react-native-vision-camera:buildCMakeDebug[arm64-v8a]'.
> com.android.ide.common.process.ProcessException: ninja: Entering directory `/Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vision-camera/android/.cxx/Debug/93z355m5/arm64-v8a'
[1/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JHashMap.cpp.o
[2/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JImageProxy.cpp.o
[3/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JFrameProcessorPlugin.cpp.o
[4/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/VisionCameraScheduler.cpp.o
[5/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/VisionCamera.cpp.o
[6/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/FrameHostObject.cpp.o
[7/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/CameraView.cpp.o
[8/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o
[9/10] Building CXX object CMakeFiles/VisionCamera.dir/src/main/cpp/JSIJNIConversion.cpp.o
[10/10] Linking CXX shared library ../../../../build/intermediates/cxx/Debug/93z355m5/obj/arm64-v8a/libVisionCamera.so
FAILED: ../../../../build/intermediates/cxx/Debug/93z355m5/obj/arm64-v8a/libVisionCamera.so
: && /Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang++ --target=aarch64-none-linux-android26 --sysroot=/Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -fPIC -fexceptions -frtti -std=c++1y -DONANDROID -DFOR_HERMES=1 -g -fno-limit-debug-info -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--fatal-warnings -Qunused-arguments -Wl,--no-undefined -shared -Wl,-soname,libVisionCamera.so -o ../../../../build/intermediates/cxx/Debug/93z355m5/obj/arm64-v8a/libVisionCamera.so CMakeFiles/VisionCamera.dir/src/main/cpp/VisionCamera.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/JSIJNIConversion.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/FrameHostObject.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/CameraView.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/VisionCameraScheduler.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JFrameProcessorPlugin.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JImageProxy.cpp.o CMakeFiles/VisionCamera.dir/src/main/cpp/java-bindings/JHashMap.cpp.o /Users/denisbevilacqua/.gradle/caches/transforms-3/6e906ae11a9e62f178ac1fed272cf94e/transformed/jetified-hermes-android-0.72.5-debug/prefab/modules/libhermes/libs/android.arm64-v8a/libhermes.so /Users/denisbevilacqua/.gradle/caches/transforms-3/18e790de7d60dccd07e3a88da530f5b8/transformed/jetified-react-android-0.72.5-debug/prefab/modules/folly_runtime/libs/android.arm64-v8a/libfolly_runtime.so /Users/denisbevilacqua/.gradle/caches/transforms-3/18e790de7d60dccd07e3a88da530f5b8/transformed/jetified-react-android-0.72.5-debug/prefab/modules/glog/libs/android.arm64-v8a/libglog.so /Users/denisbevilacqua/.gradle/caches/transforms-3/18e790de7d60dccd07e3a88da530f5b8/transformed/jetified-react-android-0.72.5-debug/prefab/modules/jsi/libs/android.arm64-v8a/libjsi.so /Users/denisbevilacqua/.gradle/caches/transforms-3/18e790de7d60dccd07e3a88da530f5b8/transformed/jetified-react-android-0.72.5-debug/prefab/modules/reactnativejni/libs/android.arm64-v8a/libreactnativejni.so /Users/denisbevilacqua/.gradle/caches/transforms-3/34c2120d0d3cd8dc97826fc02c9e8fc4/transformed/jetified-fbjni-0.3.0/prefab/modules/fbjni/libs/android.arm64-v8a/libfbjni.so /Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/aarch64-linux-android/26/liblog.so -landroid -latomic -lm && :
ld: error: undefined symbol: reanimated::extractWorkletRuntime(facebook::jsi::Runtime&, facebook::jsi::Value const&)
>>> referenced by FrameProcessorRuntimeManager.cpp:105 (/Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vision-camera/android/src/main/cpp/FrameProcessorRuntimeManager.cpp:105)
>>> CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o:(vision::FrameProcessorRuntimeManager::setFrameProcessor(facebook::jsi::Runtime&, int, facebook::jsi::Value const&, facebook::jsi::Value const&))
ld: error: undefined symbol: reanimated::extractShareableOrThrow(facebook::jsi::Runtime&, facebook::jsi::Value const&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&)
>>> referenced by Shareables.h:163 (/Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-reanimated/android/build/prefab-headers/reanimated/Shareables.h:163)
>>> CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o:(std::__ndk1::shared_ptr<reanimated::ShareableWorklet> reanimated::extractShareableOrThrow<reanimated::ShareableWorklet>(facebook::jsi::Runtime&, facebook::jsi::Value const&, std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&))
ld: error: undefined symbol: typeinfo for reanimated::Shareable
>>> referenced by memory:0 (/Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1/memory:0)
>>> CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o:(std::__ndk1::enable_if<(!(is_array<reanimated::ShareableWorklet>::value)) && (!(is_array<reanimated::Shareable>::value)), std::__ndk1::shared_ptr<reanimated::ShareableWorklet> >::type std::__ndk1::dynamic_pointer_cast<reanimated::ShareableWorklet, reanimated::Shareable>(std::__ndk1::shared_ptr<reanimated::Shareable> const&))
>>> referenced by memory:0 (/Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1/memory:0)
>>> CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o:(std::__ndk1::enable_if<(!(is_array<reanimated::ShareableWorklet>::value)) && (!(is_array<reanimated::Shareable>::value)), std::__ndk1::shared_ptr<reanimated::ShareableWorklet> >::type std::__ndk1::dynamic_pointer_cast<reanimated::ShareableWorklet, reanimated::Shareable>(std::__ndk1::shared_ptr<reanimated::Shareable> const&))
ld: error: undefined symbol: typeinfo for reanimated::ShareableWorklet
>>> referenced by memory:0 (/Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1/memory:0)
>>> CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o:(std::__ndk1::enable_if<(!(is_array<reanimated::ShareableWorklet>::value)) && (!(is_array<reanimated::Shareable>::value)), std::__ndk1::shared_ptr<reanimated::ShareableWorklet> >::type std::__ndk1::dynamic_pointer_cast<reanimated::ShareableWorklet, reanimated::Shareable>(std::__ndk1::shared_ptr<reanimated::Shareable> const&))
>>> referenced by memory:0 (/Users/denisbevilacqua/Library/Android/Sdk/ndk/23.1.7779620/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/c++/v1/memory:0)
>>> CMakeFiles/VisionCamera.dir/src/main/cpp/FrameProcessorRuntimeManager.cpp.o:(std::__ndk1::enable_if<(!(is_array<reanimated::ShareableWorklet>::value)) && (!(is_array<reanimated::Shareable>::value)), std::__ndk1::shared_ptr<reanimated::ShareableWorklet> >::type std::__ndk1::dynamic_pointer_cast<reanimated::ShareableWorklet, reanimated::Shareable>(std::__ndk1::shared_ptr<reanimated::Shareable> const&))
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
C++ build system [build] failed while executing:
/Users/denisbevilacqua/Library/Android/Sdk/cmake/3.22.1/bin/ninja \
-C \
/Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vision-camera/android/.cxx/Debug/93z355m5/arm64-v8a \
VisionCamera
from /Users/denisbevilacqua/Desktop/nationsbenefits/nb-flexpay-mobile/node_modules/react-native-vision-camera/android
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 1m 57s |
@tomekzaw This issue was resolved for me by deleting the |
* Update Podfile.lock * Install REA v3 * Upgrade project to RN 0.71 * Update build.gradle * Update packages * Add VisionCamera pod again * Update dev deps * Extend TSConfig from RN * Fix Linting setup * fix tsconfig * Update project.pbxproj * Update Info.plist * Use `JSRuntimeHelper` (temporary workaround) * Wrap in GestureHandlerRootView * Use new console.log * fix: Update CameraRoll * It works! * Print _WORKLET in frameProcessor * Enforce shareable type * It works again! * console.log works * Setup console in Reanimated * Adapt to WorkletRuntime * Update Podfile.lock * Update CameraPage.tsx * Android works! * Fix iOS * Remove paths to Reanimated headers from node_modules from `target_include_directories` in CMakeLists.txt * Bump `com.android.tools.build:gradle` from 7.3.1 to 7.4.2 fix issue Gradle Sync issue * Use `@tomekzaw/prefab` branch instead of `link:` * Remove unnecessary header * Consume Reanimated from commit * Remove dependency on Reanimated Scheduler on iOS * Go back * Remove dependency on Reanimated Scheduler on Android * Pass job as runnable * Remove `makeJSIRuntime` * Use nightly * Remove react-native-reanimated from `watchFolders` in metro.config.js * Fix crash on reload * Fix import * Barcode detection works * Use latest commit * Use `runGuarded` instead of `callWithThis` * Remove private field * Remove unused type * fix: Optionally require Reanimated * Update .gitignore * chore: Revert example app changes * chore: Remove unused packages from example app --------- Co-authored-by: Marc Rousavy <me@mrousavy.com> Co-authored-by: Marc Rousavy <marcrousavy@hotmail.com>
What
This PR makes it possible to use RNVisionCamera v2 with Reanimated v3 (3.5.0+) 🥳
I've branched off from @mrousavy's PR (#1623) which bumps react-native version in the example app as well as does some general cleanup and integrated Reanimated v3 on top of it.
Changes
Reanimated.createWorkletRuntime
WorkletRuntime::runGuarded
which already supports error handlingVisionCameraScheduler
from Reanimated internalsDemo
iOS demo: https://twitter.com/tomekzaw_/status/1697302708879765983
IMG_3318.mp4
Android demo: https://twitter.com/tomekzaw_/status/1697604643365278077
android.mp4
Tested on
Related issues