💭 Capturing iPhone photos in max resolution

[stevengoldberg]

Question

Not sure if this is a bug, a feature request, or a question, sorry!

1. On an iPhone 14 Pro or 15 Pro, I can enable resolution control in the camera settings. This allows me to take 24 or 48MP photos in HEIF or JPEG using the wide angle camera. But when I pass an array of physical devices in vision camera, like this:

const physicalDevices =
        cameraPosition === 'back'
            ? [
                  'ultra-wide-angle-camera',
                  'wide-angle-camera',
                  'telephoto-camera',
              ]
            : ['wide-angle-camera']
    const device = useCameraDevice(cameraPosition, { physicalDevices })

There is no format available with a resolution higher than 3024x4032. However, when I include only wide-angle-camera in the physicalDevices array, more formats are available, including some with 6048x8064 resolution. Is this expected? Is it possible to use a multi-camera device but still use the 48MP resolution when the wide angle camera is active?

2. When I pass only wide-angle-camera and select the highest resolution format, which looks like this:

autoFocusSystem: "phase-detection"
fieldOfView: 72.044113
maxFps: 30
maxISO: 12320
maxZoom: 189
minFps: 1
minISO: 55
photoHeight: 6048
photoWidth: 8064
pixelFormats: (9) ['yuv', 'yuv', 'rgb', 'unknown', 'yuv', 'unknown', 'yuv', 'unknown', 'rgb']
supportsDepthCapture: false
supportsPhotoHdr: false
supportsVideoHdr: false
videoHeight: 3024
videoStabilizationModes: (2) ['auto', 'off']
videoWidth: 4032

xcode logs this when I initialize the camera:

VisionCamera.configureFormat(configuration:device:): Configuring Format (8064x6048 | 4032x3024@30.0 (ISO: 55.0..12320.0, Pixel Formats: [VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.rgb, VisionCamera.PixelFormat.unknown, VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.unknown, VisionCamera.PixelFormat.yuv, VisionCamera.PixelFormat.unknown, VisionCamera.PixelFormat.rgb]))...
VisionCamera.configureFormat(configuration:device:): Successfully configured Format!
VisionCamera.onSessionInitialized(): Camera initialized!

But the result of Camera.takePhoto (taken at neutral zoom) only has 3024x4032 resolution. Is this expected? Is it possible to take photos in higher resolution?

Here are the xcode logs when I take the photo:

VisionCamera.takePhoto(options:promise:): Capturing photo...
VisionCamera.didSetProps(_:): Updating 1 props: [isActive]
VisionCamera.configure(_:): configure { ... }: Waiting for lock...
VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: false, outputsChanged: false, videoStabilizationChanged: false, orientationChanged: false, formatChanged: false, sidePropsChanged: false, torchChanged: false, zoomChanged: false, exposureChanged: false, audioSessionChanged: false)
CGAffineTransformInvert: singular matrix.
{ width: 4032,
  metadata: 
   { Orientation: 6,
     DPIWidth: 72,
     '{Exif}': 
      { DateTimeOriginal: '2024:03:07 20:05:21',
        ExposureTime: 0.011111111111111112,
        FNumber: 1.7799999713897705,
        LensSpecification: 
         [ 6.764999866485596,
           6.764999866485596,
           1.7799999713897705,
           1.7799999713897705 ],
        ExposureBiasValue: 0,
        ColorSpace: 1,
        OffsetTimeDigitized: '-05:00',
        BrightnessValue: 3.2894448964615726,
        ExposureMode: 0,
        LensModel: 'iPhone 15 Pro back camera 6.765mm f/1.78',
        SceneType: 1,
        ExposureProgram: 2,
        SensingMethod: 2,
        ShutterSpeedValue: 6.48383958192882,
        SubjectArea: [ 2013, 1511, 2116, 1330 ],
        ApertureValue: 1.6637544360060017,
        SubsecTimeDigitized: '879',
        LensMake: 'Apple',
        FocalLength: 6.764999866485596,
        SubsecTimeOriginal: '879',
        PixelYDimension: 3024,
        ISOSpeedRatings: [ 125 ],
        WhiteBalance: 0,
        DateTimeDigitized: '2024:03:07 20:05:21',
        OffsetTimeOriginal: '-05:00',
        ExifVersion: '0232',
        PixelXDimension: 4032,
        OffsetTime: '-05:00',
        Flash: 16,
        CompositeImage: 2,
        FocalLenIn35mmFilm: 24,
        MeteringMode: 5 },
     DPIHeight: 72,
     '{MakerApple}': 
      { '1': 14,
        '2': null,
        '3': 
         { timescale: 1000000000,
           epoch: 0,
           value: 13640298836833,
           flags: 1 },
        '4': 1,
        '5': 187,
        '6': 184,
        '7': 1,
        '8': 
         [ 0.001691519864834845,
           -0.9887068271636963,
           -0.14766597747802734 ],
        '12': [ 0.4609375, 1.0625 ],
        '13': 18,
        '14': 0,
        '16': 1,
        '20': 10,
        '23': 5251108,
        '25': 2,
        '31': 0,
        '32': 'E996A6DC-C68B-4668-81BA-5E3D1C7BEF66',
        '33': 1.0099999904632568,
        '35': [ 56, 268435550 ],
        '37': 4238,
        '38': 3,
        '39': 43.61827850341797,
        '40': 1,
        '43': '134578D5-C74C-4431-B39F-B3CF69916EC5',
        '45': 3367,
        '46': 1,
        '47': 117,
        '48': 0.009671603329479694,
        '51': 4096,
        '52': 4,
        '53': 3,
        '54': 654,
        '55': 8,
        '56': 44,
        '57': 0,
        '58': 0,
        '59': 0,
        '60': 4,
        '61': 93,
        '63': 1,
        '64': { '0': 1, '1': 0, '2': 0 },
        '65': false,
        '66': false,
        '67': 0,
        '68': 0,
        '69': 0,
        '70': 0,
        '72': 0,
        '73': 0,
        '74': 2,
        '77': 43.896392822265625,
        '78': 
         { '1': 1,
           '2': 
            [ { '2.1': 89.58319091796875, '2.2': 65408 },
              { '2.1': 0, '2.2': 16 } ] },
        '79': 0,
        '82': 1,
        '83': 1 },
     '{TIFF}': 
      { ResolutionUnit: 2,
        Software: '17.4',
        Make: 'Apple',
        DateTime: '2024:03:07 20:05:21',
        XResolution: 72,
        HostComputer: 'iPhone 15 Pro',
        Model: 'iPhone 15 Pro',
        YResolution: 72 } },
  height: 3024,
  isMirrored: false,
  thumbnail: null,
  path: '/private/var/mobile/Containers/Data/Application/18A5B58A-7804-4581-8673-36FF41BA6D59/tmp/ReactNative/13D5CB20-1E4B-42CC-A0F0-1C68EDA9988A.jpeg',
  isRawPhoto: false,
  orientation: 'landscape-right' }

What I tried

No response

VisionCamera Version

3.9.1

Additional information

• I am using Expo
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar questions in the issues page as well as in the discussions page and found none.

[mrousavy]

High res would be RAW, I think.

[stevengoldberg]

Not a high-priority issue IMO, but I don't believe that's the case. The iPhone 15 can capture 24 or 48 MP photos as HEIF or JPEG. I just checked with vision camera 4.0.0-beta.8 and everything is still as I reported it above.

[mrousavy]

48 MP is HEIF - 24 MP is JPEG.
Currently only JPEG is supported, HEIF isn't.
Then it's not RAW but rather HEIF that needs to be supported.
Can you create a feature request?