🐛 Photo is not what is in the preview

[inssein]

What's happening?

I just upgraded from 2.x to 4.x to test the new release (not upgrading to new architecture yet).

My camera app intentionally has a view with a 4:3 aspect ratio, but when I execute takePhoto, I seem to get a portrait photo back.

Reproduceable Code

takePhoto({
  flash: "auto",
})

Relevant log output

 LOG  CONTAINER SIZE {"height": 573.3333333333333, "left": 0, "top": 0, "width": 430}
 LOG  Taken photo. {"height": 2376, "isMirrored": false, "isRawPhoto": false, "metadata": {"DPIHeight": 72, "DPIWidth": 72, "Orientation": 6, "kCGImageDestinationICCProfile": null, "{Exif}": {"ApertureValue": 1.6637544823833461, "BrightnessValue": -1.5472978893633422, "ColorSpace": 65535, "CustomRendered": 1, "DateTimeDigitized": "2024:05:05 19:15:47", "DateTimeOriginal": "2024:05:05 19:15:47", "ExifVersion": "0232", "ExposureBiasValue": 1, "ExposureMode": 0, "ExposureProgram": 2, "ExposureTime": 0.025, "FNumber": 1.78, "Flash": 16, "FocalLenIn35mmFilm": 25, "FocalLength": 6.86, "ISOSpeedRatings": [Array], "LensMake": "Apple", "LensModel": "iPhone 14 Pro Max back dual camera 6.86mm f/1.78", "LensSpecification": [Array], "MeteringMode": 5, "OffsetTime": "-07:00", "OffsetTimeDigitized": "-07:00", "OffsetTimeOriginal": "-07:00", "PixelXDimension": 4224, "PixelYDimension": 2376, "SceneType": 1, "SensingMethod": 2, "ShutterSpeedValue": 5.321870388239852, "SubjectArea": [Array], "SubsecTimeDigitized": "827", "SubsecTimeOriginal": "827", "WhiteBalance": 0}, "{MakerApple}": {"1": 14, "12": [Array], "13": 0, "14": 0, "15": 3, "16": 1, "2": null, "20": 1, "23": 132, "25": 0, "3": [Object], "31": 0, "32": "283938DE-16AF-4F3D-9FC0-99607618010C", "35": [Array], "37": 76, "38": 3, "39": 29.460046768188477, "4": 0, "43": "DECA7309-8409-46C7-88C9-D5D5B090F735", "45": 7460, "46": 1, "47": 222, "5": 264, "51": 0, "52": 1, "53": 0, "54": 28, "55": 8, "56": 15, "57": 0, "58": 0, "59": 0, "6": 254, "60": 4, "61": 9, "63": 0, "64": [Object], "65": false, "66": false, "67": 0, "68": 0, "69": 0, "7": 1, "70": 0, "72": 15, "73": 100, "74": 2, "77": 32.3538818359375, "78": [Object], "79": 0, "8": [Array], "82": -2}, "{TIFF}": {"DateTime": "2024:05:05 19:15:47", "HostComputer": "iPhone 14 Pro Max", "Make": "Apple", "Model": "iPhone 14 Pro Max", "ResolutionUnit": 2, "Software": "17.4.1", "XResolution": 72, "YResolution": 72}}, "orientation": "landscape-right", "path": "file:///private/var/mobile/Containers/Data/Application/202A0BF3-2371-415D-88EE-D0A80CC8883E/tmp/D295619E-783D-44AD-9E24-FE7EFA6293F6.jpeg", "thumbnail": null, "width": 4224}

Camera Device

All devices on my iPhone 14 Pro Max

Device

iPhone 14 Pro Max

VisionCamera Version

4.0.1

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• I am using Expo
• I have enabled Frame Processors (react-native-worklets-core)
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[inssein]

I tried scanning through the codebase and there doesn't seem to be any resizing happening - was this just a feature of 2.x that is no longer carried over?

[zzz08900]

Would mind clarify what is actually happening (and what is wrong there)?
Is the photo rotated by 90 degrees or?

Are you sure you are selecting the right format for your desired output, not just resizing the camera preview?

[mrousavy]

Hey-

1. There's no log output here, which is exactly what I need for these kinds of issues. I don't know how else to explain it than to give Xcode/Android Studio run instructions.
   
2. You didn't try in the example app.

Are you using useCameraFormat? That's what selects a format for you.

[inssein]

@mrousavy: I added logs via editing, you may have been looking at the email. That said, I'm not quite sure it's the logs you want. You are right about the example app, I can give it a shot later today.

I also haven't tried useCameraFormat yet, because I wasn't sure that was the problem.

If I had to re-describe the problem: I set up the camera with a 4:3 aspect ratio "preview", but when I take the photo, I get something as if the preview was my whole camera screen.

Here is what the preview looks like:

Here is what the photo ends up looking like:

You can see that in the preview, my webcam was not there, but the photo contains my webcam.

[mrousavy]

Those are not native logs - I need the native logs (from Xcode or Android Studio) - those will tell me what format actually gets selected.

But yes I can answer that question right away then;

I also haven't tried useCameraFormat yet, because I wasn't sure that was the problem.

If I had to re-describe the problem: I set up the camera with a 4:3 aspect ratio "preview", but when I take the photo, I get something as if the preview was my whole camera screen.

The preview view has nothing to do with the photo output - previously in V2 it might have had something to do with it, but now this is fully decoupled (preview can even be disabled with preview={false}). So you want to use useCameraFormat(...) to select a format that has the photo resolution you desire.

Maybe try something like:

const format = useCameraFormat(device, [
  { photoAspectRatio: 4 / 3 }, // or 3 / 4, can't remember which way, sorry
  { photoResolution: 'max' }
])

[mrousavy]

btw nice mic, is that the blue jeti? I have the same

[inssein]

Ah okay, in V2 the preview is exactly what you got when you took the photo, so those two being disconnected concepts is exactly the issue. I tried your tip with the photoAspectRatio, and while it's pretty close, I seem to be getting extra content on all 4 sides (compared to the preview).

Any thoughts on how I'd go about ensuring that the bounds of the preview are exactly the same as the photo I take?

And yeah, that's a Blue Yeti. We had some in the office before the pandemic and I was able to take one home. I hate wearing headphones all day, and the mic does a surprising job of not picking up the speaker output to cancel out echo.

[mrousavy]

ensuring that the bounds of the preview are exactly the same as the photo I take?

Well yea, make sure your preview is exactly the same aspect ratio as one of the photo sizes.

The photoWidth/photoHeight (or photoSize) values are fixed and given by the Camera hardware. It cannot be flexed/changed on the fly - so you always shoot in those specific formats. V2 wasn't exactly the same as the preview, it might've been a good coincidence that the aspect ratio exactly matched though.

But what you wanna do is figure out which format you want to use, then resize your preview accordingly. or just account for slight aspect ratio mismatches, which is what pretty much every camera app does.

[Zoxive]

I have a similar issue after upgrading to v4 from v3.

I have my format set to "photoHeight": 1512, "photoWidth": 2688, making it 16:9.

I have a react native view set with its aspect ratio of 9 / 16 for the Preview.

But my preview (set with resizeMode='contain') has blackbars in v4, and when the photo is taken i see extra padding just like what @inssein is noticing.

(This on a pixel 6 pro, "react-native-vision-camera": "^4.0.3")

[mrousavy]

Preview uses a video stream. So video resolution is what counts.