🐛 Error while trying to take a photo or use frame processors and using React Native New Renderer Interop Layer

[rayronvictor]

What's happening?

The app crashes when trying to take a photo or use frame processors using the New Renderer Interop Layer.

Reproduceable Code

// react-native.config.js
module.exports = {
    project: {
        android: {
            unstable_reactLegacyComponentNames: ["CameraView"]
        },
        ios: {
            unstable_reactLegacyComponentNames: ["CameraView"]
        }
    },
};

// expo app.config.js
export default {
  // ...
  plugins: [
    [
      "expo-build-properties",
      {
        ios: {
          newArchEnabled: true
        },
        android: {
          newArchEnabled: true
        }
      }
    ],
    [
      "react-native-vision-camera",
      {
        "cameraPermissionText": "$(PRODUCT_NAME) needs access to your Camera."
      }
    ]
  ]
}

export default function CameraScreen() {
    const camera = useRef<Camera>(null)
    const [isCameraInitialized, setIsCameraInitialized] = useState(false)

    const device = useCameraDevice('back')

    const frameProcessor = useFrameProcessor((frame) => {
      'worklet'
      console.log(`Frame processed...`)
    }, [])

    const onInitialized = useCallback(() => {
        setIsCameraInitialized(true)
    }, [])

    const onMediaCaptured = useCallback(
        (media: PhotoFile | VideoFile, type: 'photo' | 'video') => {
            console.log(`Media captured! ${JSON.stringify(media)}`)
        }, []
    )

    if (device == null) return null;

    return (
        <View style={styles.container}>
            <Camera
                ref={camera}
                frameProcessor={frameProcessor}
                style={StyleSheet.absoluteFill}
                device={device}
                isActive={true}
                photo={true}
                enableZoomGesture={false}
                exposure={0}
                onInitialized={onInitialized}
            />

            <View style={styles.captureButton}>
                <Button
                    title="Capture"
                    disabled={!isCameraInitialized}
                    onPress={async () => {
                        try {
                            if (camera.current == null) throw new Error('Camera ref is null!')

                            console.log('Taking photo...')
                            const photo = await camera.current.takePhoto()
                            onMediaCaptured(photo, 'photo')
                        } catch (e) {
                            console.error('Failed to take photo!', e)
                        }
                    }}
                />
            </View>
        </View>
    );
}

const styles = StyleSheet.create({
    container: {
        flex: 1,
        alignItems: 'center',
        justifyContent: 'center',
    },
    captureButton: {
        position: 'absolute',
        alignSelf: 'center',
        bottom: 30,
    },
});

Relevant log output

# When using frame processors
com.facebook.react.uimanager.IllegalViewOperationException: Trying to resolve view with tag 116 which doesnt exist
at com.facebook.react.uimanager.NativeViewHierarchyManager.resolveView(NativeViewHierarchyManager.java:102)
at com.facebook.react.uimanager.UlManagerModule.resolveView(UlManagerModule.java:870)
at com.mrousavy.camera.frameprocessor.VisionCameraProxy.findCameraViewByld(VisionCameraProxy.kt:43)
at com.mrousavy.camera.frameprocessor.VisionCameraProxy.setFrameProcessor$lambda$0(VisionCameraProxy.kt:52)
at com.mrousavy.camera.frameprocessor.VisionCameraProxy.$r8$lambdaSILGATtBMqfunZ4A7y30ByA5WOoQ(Unknown Source:0)
at com.mrousavy.camera.frameprocessor.VisionCameraProxy$$ExternalSyntheticLambda0.run(Unknown Source:6)
at android.os.Handler.handleCallback(Handler.java:958)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loopOnce(Looper.java:222)
at android.os.Looper.loop(Looper.java:314)
at android.app.ActivityThread.main(ActivityThread.java)


# When taking a photo
com.facebook.react.uimanager.IllegalViewOperationException: Trying to resolve view with tag 116 which doesnt exist
at com.facebook.react.uimanager.NativeViewHierarchyManager.resolveView(NativeViewHierarchyManag er.java:102)
at com.facebook.react.uimanager.UlManagerModule.resolveView(UlManagerModule.java:870)
at com.mrousavy.camera.CameraViewModule$findCameraView$2$1.run(CameraViewModule.kt:62)
at android.os.Handler.handleCallback(Handler.java:958)
at android.os.Handler.dispatchMessageHandler.java:99)
at android.os.Looper.loopOnce(Looper.java:222)
at android.os.Looper.loop(Looper.java:314)
at android.app.ActivityThread.main(ActivityThread.java:8592)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.Runtimelnit$MethodAndArgsCaller.run(Runtimelnit.java:565)
at com.android.internal.os.Zygotelnit.main(Zygotelnit.java:1081)

Camera Device

{
  "formats": [],
  "sensorOrientation": "landscape-left",
  "hardwareLevel": "full",
  "maxZoom": 10,
  "minZoom": 1,
  "maxExposure": 24,
  "supportsLowLightBoost": false,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "supportsFocus": true,
  "supportsRawCapture": true,
  "isMultiCam": false,
  "minFocusDistance": 10,
  "minExposure": -24,
  "name": "BACK (0)",
  "hasFlash": true,
  "hasTorch": true,
  "position": "back",
  "id": "0"
}

Device

Xiaomi POCO X6 Pro

VisionCamera Version

3.9.0

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[mrousavy]

Yea I'm using this code to find the Camera View:

UIManagerHelper.getUIManager(
  reactApplicationContext,
  viewId
)?.resolveView(viewId) as CameraView?

..where viewId is from JavaScript:

private get handle(): number {
  const nodeHandle = findNodeHandle(this.ref.current)
  if (nodeHandle == null || nodeHandle === -1) {
    throw new CameraRuntimeError(
      'system/view-not-found',
      "Could not get the Camera's native view tag! Does the Camera View exist in the native view-tree?",
    )
  }

  return nodeHandle
}

Apparently the new Renderer interop layer doesn't have support for that yet, so I'd need to manually migrate that to Fabric/TurboModules.

[koktaildotcom]

I also have this issue when using ^3.9.1.