🐛 Camera formats are not working consistently on Android

[bekatd]

What were you trying to do?

I want to adjust coordinates returned from frame processor plugin to draw points on screen correctly

Reproduceable Code

No response

What happened instead?

Camera formats and everything related to it are completely wrong and broken.

• width and heights are swapped,
• unable to set desired formats,
• also photoWidth and photoHeight of useCameraFormat hook are totally wrong, since the width and height of frame in frameprocessor plugin is for example 640x640 while current cameraformat is 2000x4000 :|
• also camera feed strangely zooms on different formats
• also if you log all available formats and look at the format returned by default, you will discover that it can't be found in the available formats :|
• also on every different device everything works differently and wrongly :|

and #910 should be open too

how all of this is even possible :|

Relevant log output

No response

Device

Android devices

VisionCamera Version

2.13.0

Additional information

• I am using Expo
• I have read the Troubleshooting Guide
• I agree to follow this project's Code of Conduct
• I searched for similar issues in this repository and found none.

[mrousavy]

how all of this is even possible :|

Because no code is perfect and I am providing this software without warranty. I built all of it in my free time.

I know that formats on Android are broken, this is because the native CameraX library just sucks. It provides minimal configuration options, and sometimes doesn't work - yet Android docs mention that you should use it and it will be the future of Camera libs on Android, when in reality Camera2 is much more powerful.

I'll gladly work on this issue to refactor the entire API and make it work consistently on both iOS and Android if someone pays me for it, but I won't work hundreds of non-paid hours just because some people complain on GitHub yet offer no actual help/PRs :)

[bekatd]

You are absolutely right and sorry for my sort-of harsh expression. I would be working on this too, but I doubt my knowledge of native code will be sufficient for anything :(

[garmoshka-mo]

Photo & video sizes not configurable neither with presets nor with format props. (With format you can actually change size - but it will be random on android, not even close to width/height of specified format).

I had to hardcode desired resolution with patch: #868 (comment)

[mrousavy]

Hey @garmoshka-mo! That patch on iOS is not safe and might crash on a few devices.

On Android; yes, I left a few comments in the codebase warning that this is not accurate on Android due to CameraX's design. I will rewrite the android part to Camera2 in the V3 PR, sponsor me on GitHub to support this effort. I need a lot of free time to do that.

[mrousavy]

Hey! I've rewritten the entire Android codebase of VisionCamera from CameraX to Camera2 in the efforts of ✨ VisionCamera V3.

I just now completed the Camera2 rewrite and I believe the core structure is running, but there might be some edge cases to iron out. Can you try and test the PR #1674 for me to see if you can still reproduce this issue here?

Here's an instruction on how you can test that: #1674 (comment)

If the issue cannot be reproduced with that version/PR anymore, then hoorayy, I fixed it! 🎉
Otherwise please let me know and I'll keep this issue open to keep track of it.

Thank you!