[Bug report] `NullPointerException` when reading QR code from specific image

[AlexV525]

Describe the bug
When I was trying to read a specific image file that contains a valid QR code, the scanner throws an NPE or is unable to read the data.

D/VisionProcessorBase: Failed to process. Error: Internal error has occurred when executing ML Kit tasks
W/System.err: com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks
W/System.err:     at com.google.mlkit.common.sdkinternal.ModelResource.zza(com.google.mlkit:common@@18.5.0:9)
W/System.err:     at com.google.mlkit.common.sdkinternal.zzn.run(Unknown Source:10)
W/System.err:     at com.google.mlkit.common.sdkinternal.zzt.run(com.google.mlkit:common@@18.5.0:2)
W/System.err:     at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zze(com.google.mlkit:common@@18.5.0:4)
W/System.err:     at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzc(com.google.mlkit:common@@18.5.0:1)
W/System.err:     at com.google.mlkit.common.sdkinternal.zzk.run(Unknown Source:2)
W/System.err:     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
W/System.err:     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
W/System.err:     at com.google.mlkit.common.sdkinternal.MlKitThreadPool.zzd(com.google.mlkit:common@@18.5.0:2)
W/System.err:     at com.google.mlkit.common.sdkinternal.zzi.run(Unknown Source:2)
W/System.err:     at java.lang.Thread.run(Thread.java:923)
W/System.err: Caused by: java.lang.NullPointerException: Attempt to get length of null array
W/System.err:     at com.google.android.gms.internal.mlkit_vision_barcode_bundled.zzec.zzJ(com.google.mlkit:barcode-scanning@@17.0.2:1)
W/System.err:     at com.google.photos.vision.barhopper.BarhopperProto$BarhopperResponse.zzb(com.google.mlkit:barcode-scanning@@17.0.2:1)
W/System.err:     at com.google.android.libraries.barhopper.BarhopperV3.toProto(com.google.mlkit:barcode-scanning@@17.0.2:1)
W/System.err:     at com.google.android.libraries.barhopper.BarhopperV3.recognize(com.google.mlkit:barcode-scanning@@17.0.2:11)
W/System.err:     at com.google.mlkit.vision.barcode.bundled.internal.zza.zzb(com.google.mlkit:barcode-scanning@@17.0.2:5)
W/System.err:     at com.google.android.gms.internal.mlkit_vision_barcode_bundled.zzbk.zza(com.google.mlkit:barcode-scanning@@17.0.2:3)
W/System.err:     at com.google.android.gms.internal.mlkit_vision_barcode_bundled.zzb.onTransact(com.google.mlkit:barcode-scanning@@17.0.2:3)
W/System.err:     at android.os.Binder.transact(Binder.java:1054)
W/System.err:     at com.google.android.gms.internal.mlkit_vision_barcode.zza.zzb(com.google.android.gms:play-services-mlkit-barcode-scanning@@18.0.0:2)
W/System.err:     at com.google.android.gms.internal.mlkit_vision_barcode.zzox.zzd(com.google.android.gms:play-services-mlkit-barcode-scanning@@18.0.0:4)
W/System.err:     at com.google.mlkit.vision.barcode.internal.zzl.zza(com.google.android.gms:play-services-mlkit-barcode-scanning@@18.0.0:14)
W/System.err:     at com.google.mlkit.vision.barcode.internal.zzi.zzc(com.google.android.gms:play-services-mlkit-barcode-scanning@@18.0.0:3)
W/System.err:     at com.google.mlkit.vision.barcode.internal.zzi.run(com.google.android.gms:play-services-mlkit-barcode-scanning@@18.0.0:1)
W/System.err:     at com.google.mlkit.vision.common.internal.MobileVisionBase.zza(com.google.mlkit:vision-common@@17.2.1:2)
W/System.err:     at com.google.mlkit.vision.common.internal.zza.call(Unknown Source:4)
W/System.err:     at com.google.mlkit.common.sdkinternal.ModelResource.zza(com.google.mlkit:common@@18.5.0:8)
W/System.err: 	... 10 more
E/BarcodeProcessor: Barcode detection failed com.google.mlkit.common.MlKitException: Internal error has occurred when executing ML Kit tasks

To Reproduce

The image contains some personal private info, if it's applicable, please leave an email address then I can send the original image.

Example Steps to reproduce the behavior in the sample app:

1. Run the "ML Kit Vision Quickstart" example.
2. Click on 'Run the ML kit quickstart written in Kotlin'
3. Click on 'StillImageActivity'
4. Change "Object Detection" to "Barcode Scanning"
5. Choose the image from the album.

SDK Info:

• com.google.mlkit:barcode-scanning:17.0.2
• GoogleMLKit/BarcodeScanning ~3.2.0

Smartphone:

• Device/Simulator: All devices (including Android and iOS)

[zhouyiself]

Thanks for the report! Can you share your image with zhouyiself@gmail.com?

[AlexV525]

Thanks for the report! Can you share your image with zhouyiself@gmail.com?

@zhouyiself Sure. The image has been sent.

[zeroDivider]

This happened to me too, will it be considered a d fixed soon?
I am using 18.1.0. It worked well when we were using 16.2.1.

[SilverDestiny]

Contacted @AlexV525 by email. Please share the image with me to reproduce this issue. Thanks!

[zhouyiself]

@AlexV525 Hi, I didn't see your image in the email list.

[AlexV525]

@AlexV525 Hi, I didn't see your image in the email list.

Can you check again?

[SilverDestiny]

The root cause is the input Bitmap config is not ARGB_8888, and our engine only supports config ARGB_8888 for Bitmap underneath: https://developer.android.com/reference/android/graphics/Bitmap.Config

It works well for streaming mode, i.e. LivePreviewActivity and CameraXLivePreviewActivity

We'll consider converting Bitmap config to ARGB_8888 if not internally to fix this issue in later releases. Thanks for reporting this bug!

[SilverDestiny]

BTW, the original input Bitmap config is RGBA_F16

[AlexV525]

We'll consider converting Bitmap config to ARGB_8888 if not internally to fix this issue in later releases. Thanks for reporting this bug!

BitmapFactory.Options.outConfig (link) only supported from API 26, so it'll be good if we can convert the Bitmap if it's not ARGB_8888 in the ML Kit internally.

[SilverDestiny]

https://developer.android.com/reference/android/graphics/Bitmap#getConfig() could be used to get the config from bitmap?

[AlexV525]

https://developer.android.com/reference/android/graphics/Bitmap#getConfig() could be used to get the config from bitmap?

Yes, and it requires a memory allocation to read the actual data in IIRC.

[AlexV525]

Workaround with an actual allocation:

val mlImage: InputImage
val shouldConvertBitmap: Boolean
var decodedBitmap: Bitmap? = null
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
    // Decode the bitmap without memory allocation.
    val options = BitmapFactory.Options().apply { inJustDecodeBounds = true }
    BitmapFactory.decodeFile(path, options)
    shouldConvertBitmap = options.outConfig != Bitmap.Config.ARGB_8888
} else {
    val bitmap = BitmapFactory.decodeFile(path)
    shouldConvertBitmap = bitmap.config != Bitmap.Config.ARGB_8888
    // Cache the decoded bitmap.
    decodedBitmap = bitmap
}
mlImage = if (shouldConvertBitmap) {
    val bitmap = decodedBitmap ?: BitmapFactory.decodeFile(path)
    val copiedBitmap = bitmap.copy(Bitmap.Config.ARGB_8888, false)
    // Recycle the original bitmap.
    bitmap.recycle()
    InputImage.fromBitmap(copiedBitmap, 0)
} else {
    InputImage.fromFilePath(this, Uri.fromFile(File(path)))
}
// Recycle the decoded bitmap if it exists.
decodedBitmap?.recycle()