🐛 using { iso: 'max' } formatFilter crashes the iPhone 15 pro #2640

stevengoldberg · 2024-03-07T19:55:55Z

What's happening?

When max ISO is specified in the format filter, the iPhone 15 Pro crashes. This does not happen on the iPhone 14 Pro. The crash only occurs when max iso is specified.

This is the camera format that causes the crash:

{"autoFocusSystem": "phase-detection", "fieldOfView": 103.62532043457031, "maxFps": 30, "maxISO": 12320, "maxZoom": 189, "minFps": 1, "minISO": 55, "photoHeight": 3024, "photoWidth": 4032, "pixelFormats": ["yuv", "yuv", "rgb", "unknown", "yuv", "unknown", "yuv", "unknown", "rgb"], "supportsDepthCapture": true, "supportsPhotoHdr": false, "supportsVideoHdr": false, "videoHeight": 3024, "videoStabilizationModes": ["auto", "off"], "videoWidth": 4032}

Reproduceable Code

const formatFilter = [
        {
            photoAspectRatio: 4 / 3,
        },
        { photoResolution: 'max' },
        { videoResolution: { width: 1024, height: 768 } },
        { photoHdr: true },
    ]
    if (photoQualitySetting === 3) {
        formatFilter.unshift({ iso: 'max' })
    }

Relevant log output

Your app just crashed. See the error below.
NSInvalidArgumentException: *** -[AVCaptureDevice setActiveDepthDataMinFrameDuration:] activeDepthDataMinFrameDuration cannot be set when activeDepthDataFormat is nil.
  0   CoreFoundation                      0x000000019966ab34 7A70D5D4-0550-38DC-AB33-71B72AFF1F5F + 969524
  1   libobjc.A.dylib                     0x00000001914b6f78 objc_exception_throw + 60
  2   AVFCapture                          0x00000001b6495c5c 86041496-AB5E-3852-BD13-FF79DDF7875C + 146524
  3   AVFCapture                          0x00000001b64980f4 86041496-AB5E-3852-BD13-FF79DDF7875C + 155892
  4   Phomo                               0x0000000102ee7bf0 $s12VisionCamera0B7SessionC15configureFormat13configuration6deviceyAA0B13ConfigurationC_So15AVCaptureDeviceCtKF + 1080
  5   Phomo                               0x0000000102ede920 $s12VisionCamera0B7SessionC9configureyyyAA0B13ConfigurationCKcFyyYbcfU_ + 2092
  6   Phomo                               0x00000001022539e0 $sIegh_IeyBh_TR + 48
  7   libdispatch.dylib                   0x00000001a147913c 7835E829-0652-3CB5-99D8-2CE521476DC0 + 8508
  8   libdispatch.dylib                   0x00000001a147add4 7835E829-0652-3CB5-99D8-2CE521476DC0 + 15828
  9   libdispatch.dylib                   0x00000001a1482400 7835E829-0652-3CB5-99D8-2CE521476DC0 + 46080
  10  libdispatch.dylib                   0x00000001a1482f30 7835E829-0652-3CB5-99D8-2CE521476DC0 + 48944
  11  libdispatch.dylib                   0x00000001a148dcb4 7835E829-0652-3CB5-99D8-2CE521476DC0 + 93364
  12  libdispatch.dylib                   0x00000001a148d528 7835E829-0652-3CB5-99D8-2CE521476DC0 + 91432
  13  libsystem_pthread.dylib             0x00000001f5568f20 _pthread_wqthread + 288
  14  libsystem_pthread.dylib             0x00000001f5568fc0 start_wqthread + 8

Camera Device

{
  "minFocusDistance": 2,
  "position": "back",
  "hardwareLevel": "full",
  "isMultiCam": true,
  "sensorOrientation": "landscape-right",
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:7",
  "supportsLowLightBoost": false,
  "maxExposure": 8,
  "minZoom": 1,
  "supportsFocus": true,
  "neutralZoom": 2,
  "supportsRawCapture": false,
  "physicalDevices": [
    "ultra-wide-angle-camera",
    "wide-angle-camera",
    "telephoto-camera"
  ],
  "formats": [],
  "minExposure": -8,
  "hasFlash": true,
  "name": "Back Triple Camera",
  "hasTorch": true,
  "maxZoom": 123.75
}

Device

iPhone 15 Pro iOS 17.4

VisionCamera Version

3.9.1

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

I am using Expo
I have enabled Frame Processors (react-native-worklets-core)
I have read the Troubleshooting Guide
I agree to follow this project's Code of Conduct
I searched for similar issues in this repository and found none.

The text was updated successfully, but these errors were encountered:

stevengoldberg · 2024-03-08T01:16:36Z

Update — the crash only happens when I'm using a multi-cam, e.g.

const physicalDevices = [
                  'ultra-wide-angle-camera',
                  'wide-angle-camera',
                  'telephoto-camera',
              ]

If I pass only wide-angle-camera as the physicalDevices, it doesn't crash. Here are the full xcode logs when the crash occurs:

{ minFps: 1,
  videoWidth: 4032,
  minISO: 55,
  photoHeight: 3024,
  maxFps: 30,
  fieldOfView: 103.62532043457031,
  videoHeight: 3024,
  autoFocusSystem: 'phase-detection',
  supportsVideoHdr: false,
  supportsPhotoHdr: false,
  photoWidth: 4032,
  supportsDepthCapture: true,
  maxISO: 12320,
  pixelFormats: 
   [ 'yuv',
     'yuv',
     'rgb',
     'unknown',
     'yuv',
     'unknown',
     'yuv',
     'unknown',
     'rgb' ],
  maxZoom: 189,
  videoStabilizationModes: [ 'auto', 'off' ] }
 This method can cause UI unresponsiveness if invoked on the main thread. Instead, consider waiting for the `-locationManagerDidChangeAuthorization:` callback and checking `authorizationStatus` first.
{ minFps: [Getter/Setter],
  videoWidth: [Getter/Setter],
  minISO: [Getter/Setter],
  photoHeight: [Getter/Setter],
  maxFps: [Getter/Setter],
  fieldOfView: [Getter/Setter],
  videoHeight: [Getter/Setter],
  autoFocusSystem: [Getter/Setter],
  supportsVideoHdr: [Getter/Setter],
  supportsPhotoHdr: [Getter/Setter],
  photoWidth: [Getter/Setter],
  supportsDepthCapture: [Getter/Setter],
  maxISO: [Getter/Setter],
  pixelFormats: [Getter/Setter],
  maxZoom: [Getter/Setter],
  videoStabilizationModes: [Getter/Setter] }
VisionCamera.didSetProps(_:): Updating 21 props: [onInitialized, cameraId, enableBufferCompression, onCodeScanned, collapsable, photoHdr, onStarted, isActive, pixelFormat, onViewReady, onError, onStopped, enableHighQualityPhotos, enableFrameProcessor, format, orientation, flex, width, exposure, zoom, photo]
VisionCamera.configure(_:): configure { ... }: Waiting for lock...
VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: true, outputsChanged: true, videoStabilizationChanged: true, orientationChanged: true, formatChanged: true, sidePropsChanged: true, torchChanged: true, zoomChanged: true, exposureChanged: true, audioSessionChanged: true)
VisionCamera.configureDevice(configuration:): Configuring Input Device...
VisionCamera.configureDevice(configuration:): Configuring Camera com.apple.avfoundation.avcapturedevice.built-in_video:7...
VisionCamera.configureDevice(configuration:): Successfully configured Input Device!
VisionCamera.configureOutputs(configuration:): Configuring Outputs...
VisionCamera.configureOutputs(configuration:): Adding Photo output...
VisionCamera.configureOutputs(configuration:): Successfully configured all outputs!
VisionCamera.configureFormat(configuration:device:): Configuring Format (4032x3024 | 4032x3024@30.0 (ISO: 55.0..12320.0, Pixel Formats: [VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.rgb, VisionCamera.PixelFormat.unknown, VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.unknown, VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.unknown, VisionCamera.PixelFormat.rgb]))...
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[AVCaptureDevice setActiveDepthDataMinFrameDuration:] activeDepthDataMinFrameDuration cannot be set when activeDepthDataFormat is nil.'
*** First throw call stack:
(0x19966ab28 0x1914b6f78 0x1b6495c5c 0x1b64980f4 0x100f33bf0 0x100f2a920 0x10029f9e0 0x106f88b98 0x106f8a7bc 0x106f9266c 0x106f93408 0x106fa0404 0x106f9fa38 0x1f5568f20 0x1f5568fc0)
DevLauncher tries to handle uncaught exception: *** -[AVCaptureDevice setActiveDepthDataMinFrameDuration:] activeDepthDataMinFrameDuration cannot be set when activeDepthDataFormat is nil.
Stack Trace: (
    "0   CoreFoundation                      0x000000019966ab34 7A70D5D4-0550-38DC-AB33-71B72AFF1F5F + 969524",
    "1   libobjc.A.dylib                     0x00000001914b6f78 objc_exception_throw + 60",
    "2   AVFCapture                          0x00000001b6495c5c 86041496-AB5E-3852-BD13-FF79DDF7875C + 146524",
    "3   AVFCapture                          0x00000001b64980f4 86041496-AB5E-3852-BD13-FF79DDF7875C + 155892",
    "4   Phomo                               0x0000000100f33bf0 $s12VisionCamera0B7SessionC15configureFormat13configuration6deviceyAA0B13ConfigurationC_So15AVCaptureDeviceCtKF + 1080",
    "5   Phomo                               0x0000000100f2a920 $s12VisionCamera0B7SessionC9configureyyyAA0B13ConfigurationCKcFyyYbcfU_ + 2092",
    "6   Phomo                               0x000000010029f9e0 $sIegh_IeyBh_TR + 48",
    "7   libdispatch.dylib                   0x0000000106f88b98 _dispatch_call_block_and_release + 32",
    "8   libdispatch.dylib                   0x0000000106f8
CGAffineTransformInvert: singular matrix.
libc++abi: terminating due to uncaught exception of type NSException

mrousavy · 2024-03-11T12:49:36Z

[AVCaptureDevice setActiveDepthDataMinFrameDuration:] activeDepthDataMinFrameDuration cannot be set when activeDepthDataFormat is nil

Are you sure this can be reproduced in the example app? It sounds like you're rolling a custom patch with some changes (activeDepthDataMinFrameDuration)

stevengoldberg · 2024-03-11T13:08:25Z

I’ll try running it in the example app, but I haven’t made any changes like that — the exact same code works correctly on an iPhone 14 Pro, but throws this error on a 15 Pro. I only encountered it because I got a new phone.

stevengoldberg · 2024-03-13T18:46:48Z

@mrousavy I have been unable to build the example project, but I created a minimal repro of the crash here:

https://github.com/stevengoldberg/vision-camera-crash/

It crashes the same way every time, only on the iPhone 15 Pro. Repro instructions in the readme.

iamtommcc · 2024-03-25T05:22:44Z

Confirming that I too have been able to replicate this bug on a physical iPhone 15 Pro on iOS 17.4

In addition to iso, it also crashes if I try setting videoResolution to max as well.

stevengoldberg · 2024-03-25T13:05:46Z

FWIW I updated to 4.0 beta 8 and the crash doesn’t happen there.

mrousavy · 2024-04-22T11:25:47Z

Hey - I think this issue has been fixed in VisionCamera 4.0.0. 🥳

Please try V4 and let me know if you still experience this issue;

if not, please consider 💖 sponsoring me on GitHub 💖 to support the development of VisionCamera and thank me for my time spent on fixing bugs and building new features.
if you still see this issue, please comment and we can re-open this. But please update your native logs with the native (Xcode/Android Studio) logs from running VisionCamera V4 so I can investigate this.

stevengoldberg added the 🐛 bug Something isn't working label Mar 7, 2024

mrousavy closed this as completed Apr 22, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🐛 using { iso: 'max' } formatFilter crashes the iPhone 15 pro #2640

🐛 using { iso: 'max' } formatFilter crashes the iPhone 15 pro #2640

stevengoldberg commented Mar 7, 2024 •

edited by mrousavy

Loading

stevengoldberg commented Mar 8, 2024

mrousavy commented Mar 11, 2024

stevengoldberg commented Mar 11, 2024

stevengoldberg commented Mar 13, 2024

iamtommcc commented Mar 25, 2024

stevengoldberg commented Mar 25, 2024

mrousavy commented Apr 22, 2024

🐛 using { iso: 'max' } formatFilter crashes the iPhone 15 pro #2640

🐛 using { iso: 'max' } formatFilter crashes the iPhone 15 pro #2640

Comments

stevengoldberg commented Mar 7, 2024 • edited by mrousavy Loading

What's happening?

Reproduceable Code

Relevant log output

Camera Device

Device

VisionCamera Version

Can you reproduce this issue in the VisionCamera Example app?

Additional information

stevengoldberg commented Mar 8, 2024

mrousavy commented Mar 11, 2024

stevengoldberg commented Mar 11, 2024

stevengoldberg commented Mar 13, 2024

iamtommcc commented Mar 25, 2024

stevengoldberg commented Mar 25, 2024

mrousavy commented Apr 22, 2024

stevengoldberg commented Mar 7, 2024 •

edited by mrousavy

Loading