[firebase_ml_vision] FirebaseVisionImage.fromBytes image format

[andythehood]

In the description for the FirebaseVisionImage.fromBytes method in the firebase_ml_vision package, there is the comment that "NV21 format is expected on Android and that this can be obtained by concatenating the planes of a YUV_420_888 format image". Which is exactly what the example code does:

https://github.com/FirebaseExtended/flutterfire/blob/master/packages/firebase_ml_vision/example/lib/scanner_utils.dart

However, my understanding is that YUV_420_888 is a non-interlaced format (U and V as separate planes) whereas NV21 has the UV planes interlaced, with the first V sample first.

So, if this is right, just concatenating the YUV_420 planes won't give a true NV21 format image, as although the Y plane will be correct, and hence the image will look OK, the colours will be skewed due to the chroma planes being off.

Do the MLKit models really require NV21 or do they not really care about the UV planes that much?

_FirebaseVisionImage FirebaseVisionImage.fromBytes(Uint8List bytes, FirebaseVisionImageMetadata metadata)
package:firebase_ml_vision/firebase_ml_vision.dart

Construct a [FirebaseVisionImage] from a list of bytes.

On Android, expects android.graphics.ImageFormat.NV21 format. Note: Concatenating the planes of android.graphics.ImageFormat.YUV_420_888 into a single plane, converts it to android.graphics.ImageFormat.NV21.

[Salakar]

Looking at the Android SDK docs it does look like you can explicitly set the image format to either format; https://firebase.google.com/docs/reference/android/com/google/firebase/ml/vision/common/FirebaseVisionImageMetadata w

With both NV21 & YV12 being supported there I don't think the MLKit models would care about UV planes as you stated, at least that is my understanding

[google-oss-bot]

Hey @andythehood. We need more information to resolve this issue but there hasn't been an update in 7 weekdays. I'm marking the issue as stale and if there are no new updates in the next 7 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

[google-oss-bot]

Since there haven't been any recent updates here, I am going to close this issue.

@andythehood if you're still experiencing this problem and want to continue the discussion just leave a comment here and we are happy to re-open this.