feat: Implement Orientation

[mrousavy]

What

Implements orientation for VisionCamera.

1. A new prop is introduced (outputOrientation), which specifies how VisionCamera should treat orientation:
   • device; use physical device orientation, even if screen is locked
   • preview; use preview orientation, respects screen-lock
   • portrait, landscape-left, landscape-right, portrait-upside-down; locked to a specific orientation
2. The current orientation prop has been removed to avoid confusion. (it never really worked anyways)
3. Adds documentation for Orientation to explain how it works
4. Make Frame.orientation actually reflect it's relative orientation.
5. Stops rotating CMSampleBuffers on iOS. This makes the video and frame processor pipeline much more lightweight and use less energy, as it now no longer physically rotates pixels in a buffer.
   • For Videos, rotation will be passed to the AVAssetWriter (EXIF flag)
   • For Frame Processors, Frame.orientation now represents it's rotation relative to the target rotation (like on Android)
6. In a previous PR, Skia Frame Processors now properly rotate their transform matrix to support orientation (fix: Fix Skia Frame Processor transformation matrix #2931)

Changes

Tested on

Related issues

• Resolves ✨ Implement Orientation ($3,000) #1891

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
react-native-vision-camera	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 6, 2024 3:15pm

[xulihang]

Stops rotating CMSampleBuffers on iOS. This makes the video and frame processor pipeline much more lightweight and use less energy, as it now no longer physically rotates pixels in a buffer.

This seems like a breaking change for frame processors. For text recognition, the frame processor has to rotate the image to match the preview in portrait mode if it does not accept passing the orientation.

Maybe adding an option to enable rotation is a good idea to make old frame processors compatible with the new version.

[mrousavy]

Hm, I am not sure - I think for portrait nothing changed, but whereas frame processors previously didnt work when the phone is rotated, they now work.

Not entirely sure though, need to test it...

[laxminarayana-capgemini]

Hi @mrousavy, we trying to record the video in Landscape, but the output video is in Portrait. Whatever the orientation we are recording it is in Portrait only. We are running react-native-vision-camera in Android application, but it is not working. Hereby I have attached a sample code. Kindly help us with a solution.
<Camera style={StyleSheet.absoluteFill} device={device} isActive={isAppForeground} video={true} audio={true} ref={camera} format={format} exposure={0} zoom={0} orientation='landscape-left' />

[pke]

thanks @mrousavy! It seems onCodeScanned of useCodeScanner has not had its frame.bounds fixed with this PR? The bounds should be aligned in React space always at 0,0 being top left of the detected code.

[jkaufman]

@laxminarayana-capgemini The name of the orientation prop changed with this PR.

[laxminarayana-capgemini]

@mrousavy and @jkaufman. thanks, the issue is resolved.