🐛 [4.0.0-beta.15] - Recording video with useSkiaFrameProcessor freezes my app

[Ballonek]

What's happening?

When I try to record video during frame processing using the new useSkiaFrameProcessor everything works fine, the problem occurs when I want to stop the video recording. At that point my application freezes and in some cases crashes.

All of this did work until I updated to beta.15 so i asume that is caused by this update.

I tried to reproduce it on example app but getting error: Error: Unable to resolve module @babel/runtime/helpers/interopRequireDefault from /Users/XYZ/Git/react-native-vision-camera/package/src/Camera.tsx: @babel/runtime/helpers/interopRequireDefault could not be found within the project.

Reproduceable Code

const frameProcessor = useSkiaFrameProcessor(frame => {
  'worklet';
  const detectedPose = poseDetection(
    frame,
    frameNumber.current,
  ) as SkeletonPose;
  frame.render();
  if (isRecording.current) {
    if (Object.values(detectedPose).length > 0) {
      const poseArray = getFramePoseData(
        detectedPose,
        frame.width,
        frame.height,
      );

      const pointsPaint = Skia.Paint();
      pointsPaint.setColor(Skia.Color('green'));
      pointsPaint.setAntiAlias(true);
      pointsPaint.setStyle(PaintStyle.Fill);
      for (let i = 0; i < poseArray.length; i++) {
        frame.drawCircle(
          poseArray[i].x * frame.width,
          poseArray[i].y * frame.height,
          10,
          pointsPaint,
        );
      }
    
      frame.restore();
      frameNumber.current++;
    } else {
      frame.render();
    }
  }
}, []);

<ReanimatedCamera
  ref={cameraRef}
  frameProcessor={frameProcessor}
  style={StyleSheet.absoluteFill}
  device={device}
  video={true}
  pixelFormat="rgb"
  isActive
  fps={30}
  format={format}
  preview={false}
/>

Relevant log output

15:35:41.555: [info] 📸 VisionCamera.startRecording(options:onVideoRecorded:onError:): Starting Video recording...
15:35:41.561: [info] 📸 VisionCamera.startRecording(options:onVideoRecorded:onError:): Will record to temporary file: /private/var/mobile/Containers/Data/Application/A47FCB4D-79DE-4162-98E5-A12594FFE8FF/tmp/ReactNative/61032842-6B19-4449-84E1-4DF712515D5D.mov
15:35:41.568: [info] 📸 VisionCamera.initializeVideoWriter(withSettings:): Initializing Video AssetWriter with settings: ["AVVideoCompressionPropertiesKey": {
    AllowFrameReordering = 1;
    AllowOpenGOP = 1;
    AverageBitRate = 4847616;
    ExpectedFrameRate = 30;
    MaxAllowedFrameQP = 41;
    MaxKeyFrameIntervalDuration = 1;
    MinAllowedFrameQP = 15;
    MinimizeMemoryUsage = 1;
    Priority = 80;
    ProfileLevel = "HEVC_Main_AutoLevel";
    RealTime = 1;
    RelaxAverageBitRateTarget = 1;
}, "AVVideoCodecKey": hvc1, "AVVideoHeightKey": 1280, "AVVideoWidthKey": 720]
15:35:41.578: [info] 📸 VisionCamera.initializeVideoWriter(withSettings:): Initialized Video AssetWriter.
15:35:41.578: [info] 📸 VisionCamera.start(clock:): Starting Asset Writer(s)...
15:35:45.966: [info] 📸 VisionCamera.start(clock:): Asset Writer(s) started!
15:35:45.967: [info] 📸 VisionCamera.start(clock:): Started RecordingSession at time: 188924.772228166
15:35:45.967: [info] 📸 VisionCamera.startRecording(options:onVideoRecorded:onError:): RecordingSesssion started in 4411.938375ms!

nw_socket_handle_socket_event [C20.1.1:1] Socket SO_ERROR 54
nw_protocol_socket_reset_linger [C20.1.1:1] setsockopt SO_LINGER failed 22

15:35:49.452: [info] 📸 VisionCamera.stop(clock:): Requesting stop at 188928.258044666 seconds for AssetWriter with status "writing"...
nw_socket_handle_socket_event [C21.1.1:1] Socket SO_ERROR 54
nw_protocol_socket_reset_linger [C21.1.1:1] setsockopt SO_LINGER failed 22
SocketStream read error [0x301425d60]: 1 54
15:35:53.563: [error] 📸 VisionCamera.stop(clock:): Waited 4.0 seconds but no late Frames came in, aborting capture...
15:35:53.563: [info] 📸 VisionCamera.finish(): Stopping AssetWriter with status "writing"...
15:35:53.587: [info] 📸 VisionCamera.startRecording(options:onVideoRecorded:onError:): RecordingSession finished with status completed.

Error occurred: The operation couldn’t be completed. (OSStatus error -50.)

(lldb)

Camera Device

{
  "hasTorch": true,
  "hardwareLevel": "full",
  "maxZoom": 123.75,
  "isMultiCam": false,
  "position": "back",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "supportsRawCapture": false,
  "supportsFocus": true,
  "neutralZoom": 1,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "minZoom": 1,
  "name": "Back Camera",
  "hasFlash": true,
  "minExposure": -8,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "sensorOrientation": "landscape-right",
  "minFocusDistance": 12,
  "formats": []
}

Device

iPhone 12 Mini (iOS 17.4.1)

VisionCamera Version

4.0.0-beta.15

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[mrousavy]

good bug report, will look into this soon

[mrousavy]

I think this should fix your issue: #2754 although I haven't tested it yet.

Wanna pull in the changes (e.g. via patch-package) and try if it works? You can ignore android/ changes if you only test iOS obviously.

[Ballonek]

Thank you, I will try to patch that :)

[mrousavy]

fixed in #2754.