🐛 Frame is sideways on android portrait orientation

[jslok]

What were you trying to do?

When using frame processor on android in portrait mode, the frame is rotated to the left. When rotated to landscape mode, the frame is the correct orientation.

Orientation prop from #715 does not seem to have an effect here. This prop is not mentioned in the docs either.

I can always rotate it in my custom frame processor but it seems like it should be presented in portrait if the device is in portrait.

Reproduceable Code

No response

What happened instead?

Frame should be the correct orientation in portrait.

Relevant log output

No response

Device

galaxy s20

VisionCamera Version

2.12.0 and 2.13.1

Additional information

• I am using Expo
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[jslok]

Here's screenshots of what I mean. In each screenshot, the top is the camera live view and the bottom is the frame from frameprocessor.

[mrousavy]

Hey, thanks for the detailed issue and explanation with screenshots.

You're right, this is actually a big issue, the Frame should be oriented correctly - does the Frame.orientation value change at all or is it portrait in every case?

[mrousavy]

I think this is the native orientation of the frame, since Camera sensors are in landscape orientation by default. If I would actually rotate the frame (moving each pixel to it's rotated value), this would be really really slow because I have to do that every frame.

If the Frame.orientation value is correct, there's not much more to do since this is what is expected by all frame processing libraries like MLKit Vision (they all have a orientation flag that tells them how to interpret the frame)

[mrousavy]

Hey! I've rewritten the entire Android codebase of VisionCamera from CameraX to Camera2 in the efforts of ✨ VisionCamera V3.

I just now completed the Camera2 rewrite and I believe the core structure is running, but there might be some edge cases to iron out. Can you try and test the PR #1674 for me to see if you can still reproduce this issue here?

Here's an instruction on how you can test that: #1674 (comment)

If the issue cannot be reproduced with that version/PR anymore, then hoorayy, I fixed it! 🎉
Otherwise please let me know and I'll keep this issue open to keep track of it.

Thank you!

[mrousavy]

Hey - I'm tracking Orientation in this feature request/issue now: #1891 ✨

Make sure to upvote or sponsor to support this feature, and leave a comment if you have any additional thoughts/ideas.