MLKit Vision is reading code128 and 39 barcodes poorly #32

chrisvanbuskirk · 2019-02-20T02:27:07Z

I'm reading a buffer like such:

 let visionImage = VisionImage(buffer: sampleBuffer)
            barcodeDetector.detect(in: visionImage) { (barcodes, error) in
           ...
           }

I'm seeing errors of random characters being exchanged in code128/39 barcodes. No errors are raised. As you can see I'm checking enterprise labels. The 2D data matrix codes scan fine. However, their 1d counterparts randomly fail.

The text was updated successfully, but these errors were encountered:

chrisvanbuskirk · 2019-02-20T11:19:24Z

MMS485Z5 Laser - ALS.pdf

Here is a sample barcode.

chrisvanbuskirk · 2019-02-20T11:20:39Z

I'd like to also make note of the poor choppy performance of the camera while reading barcodes. I'm using the same camera code with Apple's vision with no issues.

ulukaya · 2019-10-09T20:29:14Z

Please re-try with the latest barcode detection model (FirebaseMLVisionBarcodeModel 0.19.0) in MLKit

chrisvanbuskirk · 2019-10-10T14:29:23Z

@ulukaya. I updated and I'm I'm still seeing character errors.

miworking3 · 2020-08-03T15:59:07Z

Hey Chris,

We upgraded the Firebase MLKit API to the standalone Google MLKit API early this year: https://developers.google.com/ml-kit

I tried the the latest vision app with the given barcodes, looks like it works well with S, Q, H, could you give it a try and let us know if that would help your case as well?

Thanks a lot!
Julie from MLKit team

chrisvanbuskirk · 2020-08-03T16:06:33Z

@miworking3 I will test this.

chrisvanbuskirk · 2020-08-04T16:25:23Z

@miworking3 Impressively done. Thanks! Closing this issue.

chrisvanbuskirk · 2020-08-05T19:28:55Z

@miworking3 I am reopening. I was able to have reproduce similar issue, where this barcode produced N51%%3 after several tries.

miworking3 · 2020-08-06T14:19:24Z

Could you help me to reproduce it with the MLKit quickstart app?

I tried to move the camera off and on several times, the result looks consistent to me.

chrisvanbuskirk · 2020-08-06T15:14:38Z

@miworking3 I'm not sure. I'd have to study the sample more carefully. This issue is very rare. You can see in the following screen shot it plugs in the wrong value. I'm not sure why. Position of camera. Some sort of flaw in our barcode. The 2D barcodes are fine.

I'm going to step through the sample, then my code. I may spend the day with it.

chrisvanbuskirk · 2020-08-06T16:02:32Z

@miworking3

Fascinating...

I was using

barcodeDetector.process(visionImage) { (barcodes, error) in
...and changed to method in the sample and am not seeing the problem.

barcodes = try barcodeScanner.results(in: visionImage)
I'm going to do even more testing. The handler seems to have problems.

miworking3 · 2020-08-07T15:15:43Z

Thanks for checking that!
Yes, for the realtime video frames, we would recommend to use the synchronous API results(in:), and there are also other tips for the videos that might be helpful:
For processing video frames, use the results(in:) synchronous API of the detector. Call this method from the AVCaptureVideoDataOutputSampleBufferDelegate's captureOutput(_, didOutput:from:) function to synchronously get results from the given video frame. Keep AVCaptureVideoDataOutput's alwaysDiscardsLateVideoFrames as true to throttle calls to the detector. If a new video frame becomes available while the detector is running, it will be dropped.

https://developers.google.com/ml-kit/vision/barcode-scanning/ios

chrisvanbuskirk · 2020-08-07T20:24:22Z

@miworking3

self.session.sessionPreset = .hd1280x720

Seemed to really fix the errors. Not closing just yet, cause I have more testing..but it's going great so far.

miworking3 · 2020-08-10T14:55:27Z

Thank you @chrisvanbuskirk for verifying this. We appreciate the data point from your side.
Yes, the preset can affect the results, and 1280x720 is a recommended setting by our doc as well:
The named capture session presets—AVCaptureSessionPresetDefault, AVCaptureSessionPresetLow, AVCaptureSessionPresetMedium, and so on)—are not recommended, however, as they can map to unsuitable resolutions on some devices. Instead, use the specific presets such as AVCaptureSessionPreset1280x720.

https://developers.google.com/ml-kit/vision/barcode-scanning/ios

ruslan-podurets · 2020-08-14T11:39:12Z

Hi, have issue with 128 code barcode, when ml kit returns wrong values. I use com.google.mlkit:barcode-scanning:16.0.2 with default camera resolution settings (tried different). Also analyze in this way:
val mediaImage = imageProxy.image
val image = InputImage.fromMediaImage(mediaImage, imageProxy.imageInfo.rotationDegrees)
scanner.process(image)
This issue happens on many Android and IPhone devices. F.e HTC U11.
Ussually kit returns wrong short text of numbers like "15771449", but sometimes valid long text: "1076839827TanCarol Windell A4030052420".
img url: https://ibb.co/yh3DJFT

miworking3 · 2020-08-14T15:46:11Z

@ruslan-poduret Thanks for reporting this, could you provide more information about the iPhone device that you found this problem? When you mentioned about "This issue happens on many Android and IPhone devices", is it reproducible with the MLKit iOS quickstart app?

BTW: About the android API, I've already forwarded your message to our Android team, he might have further questions about it later.

chrisvanbuskirk · 2020-08-14T16:00:57Z

@miworking I'm afraid I still am seeing bad reads every 40-50 scans. Just seems for the 128's. It's very random and odd, and I can't quite pinpoint why. If it's distance. The barcode quality. Digital v. non digital, etc... There are less errors with some refactoring I have done.

miworking3 · 2020-08-14T16:06:27Z

@chrisvanbuskirk Do you know the expected value of the barcode being read? If so, you can save the samplebuffer to an image right at the time when it fails, and with that information, a further investigation can be continued. Thanks!

chrisvanbuskirk · 2020-08-14T16:23:02Z

@miworking3 I will look into it.

miworking3 · 2020-10-16T18:51:16Z

It's been a while, with previous major concerns clarified, closing this thread. Pls feel free to reopen it if new updates. Thanks!

felipetce · 2021-10-30T02:37:09Z

I have the version 16.0.2 and the same problem happens

miworking3 · 2021-10-30T15:52:59Z

16.0.2 for Android SDK?

felipetce · 2021-11-08T16:16:57Z

I have the version com.google.mlkit:barcode-scanning:16.0.2 and the same problem happens

felipetce · 2021-11-09T22:24:35Z

Hi, as I mentioned, the images were from barcode-scanninig:17.0.0:

felipetce · 2021-11-09T22:32:58Z

After we exchanged Zxing for MLkit, we are having a lot of problems, analyzing the Zxing code, it seems that it had a better validation of the processed barcode

felipetce · 2021-11-09T22:49:31Z

ZXing

ng

SilverDestiny · 2021-11-09T23:08:27Z

I cannot repro your issue for https://user-images.githubusercontent.com/16478800/141014762-323d5709-937c-4a51-ab2b-cca159b29291.jpg
It seems consistently working for me.

For https://user-images.githubusercontent.com/16478800/141014875-66fc3c66-9626-4e16-a7ea-285df1d193ff.jpg, it is reproducible, but this image is cropped so that it doesn't contain the full barcode(left side is missing).

Did you mean ZXing also work for cropped images to detect the full barcode? Could you share an example?

It seems working properly for https://user-images.githubusercontent.com/16478800/141017954-1bdc85fe-d3a0-426d-b94a-37d5d6335196.jpg as well.

We probably can improve our model to consider more for the full barcode and ignore the partial barcode in future releases.

felipetce · 2021-11-09T23:22:21Z

yes, Zxing did not processed barcodes not read completely

felipetce · 2021-11-09T23:24:02Z

By code I verified that the processing of MLKIT barcodes is 1 second

felipetce · 2021-11-09T23:24:48Z

that is, I think the processing is unsafe

felipetce · 2021-11-09T23:25:43Z

we are having a lot of problems having changed the ZXing for ML

felipetce · 2021-11-09T23:26:49Z

As a multinational company and we need the guarantee of the read codes

SilverDestiny · 2021-11-09T23:30:04Z

Could you share an example for the same cropped image between ML Kit and ZXing?

SilverDestiny · 2021-11-09T23:32:58Z

Our benchmark tests showed that for 1600x1200 input image, the inference time is around 100 ms.
The inference time depends on the input image size.

SilverDestiny · 2021-11-09T23:34:29Z

We're working a project which will not request camera permission for barcode scanning, if you think this sounds safer to you.
It will be released soon next year.

SilverDestiny · 2021-11-09T23:35:33Z

Could clarify all the problems except this one? We're happy to hear and will try our best to resolve your problems.

felipetce · 2021-11-09T23:38:00Z

felipetce · 2021-11-09T23:38:05Z

Zxing does not process uncompleted scanned barcodes

felipetce · 2021-11-09T23:39:35Z

I found that we have no way to access the image processing source code

felipetce · 2021-11-09T23:40:28Z

Would it be possible to have access to this image processing code?

felipetce · 2021-11-09T23:42:05Z

It's causing us a lot of production problems with the ZXing switch

felipetce · 2021-11-09T23:46:26Z

I can't reproduce the same problem with ZXing, it seems the image processing is better

SilverDestiny · 2021-11-10T01:34:36Z

We'll work on this to see if we can improve to fix this issue.

wsliaw · 2022-10-29T03:09:18Z

the problem still exists on latest version (android 17.0.2, iOS 2.2.0)
should we reopen it, until not process uncompleted scanned barcodes is implement?
thanks 🙏

miworking3 · 2022-10-31T16:08:08Z

This thread has diverted multiple times, and the latest update is not related to the original title.
Please open a new request here with your request clarified, and we will look further into them ,thanks.

morganchen12 assigned ulukaya Feb 20, 2019

ulukaya transferred this issue from firebase/quickstart-ios Jun 30, 2020

ulukaya transferred this issue from another repository Jun 30, 2020

chrisvanbuskirk closed this as completed Aug 4, 2020

chrisvanbuskirk reopened this Aug 5, 2020

ruslan-podurets mentioned this issue Aug 14, 2020

Barcode scanner detected wrong barcode sometimes #17

Open

miworking3 closed this as completed Oct 16, 2020

miworking3 assigned SilverDestiny and unassigned miworking3 Nov 10, 2021

SilverDestiny removed their assignment Oct 29, 2022

miworking3 assigned miworking3 and unassigned miworking3 Oct 31, 2022

Saloni1232 mentioned this issue Aug 8, 2023

[libbarhopper_v3.so] Java_com_google_android_libraries_barhopper_BarhopperV3_recognizeBufferNative #713

Closed

Saloni123456 mentioned this issue Aug 8, 2023

[ML KIT Crash] [libbarhopper_v3.so] Java_com_google_android_libraries_barhopper_BarhopperV3_recognizeBufferNative[Bug report] Title for the bug #714

Open

MLKit Vision is reading code128 and 39 barcodes poorly #32

MLKit Vision is reading code128 and 39 barcodes poorly #32

Comments

chrisvanbuskirk commented Feb 20, 2019

chrisvanbuskirk commented Feb 20, 2019

chrisvanbuskirk commented Feb 20, 2019 • edited

ulukaya commented Oct 9, 2019

chrisvanbuskirk commented Oct 10, 2019

miworking3 commented Aug 3, 2020

chrisvanbuskirk commented Aug 3, 2020

chrisvanbuskirk commented Aug 4, 2020

chrisvanbuskirk commented Aug 5, 2020

miworking3 commented Aug 6, 2020

chrisvanbuskirk commented Aug 6, 2020 • edited

chrisvanbuskirk commented Aug 6, 2020 • edited

miworking3 commented Aug 7, 2020 • edited

chrisvanbuskirk commented Aug 7, 2020 • edited

miworking3 commented Aug 10, 2020

ruslan-podurets commented Aug 14, 2020

miworking3 commented Aug 14, 2020

chrisvanbuskirk commented Aug 14, 2020 • edited

miworking3 commented Aug 14, 2020

chrisvanbuskirk commented Aug 14, 2020

miworking3 commented Oct 16, 2020

felipetce commented Oct 30, 2021

miworking3 commented Oct 30, 2021

felipetce commented Nov 8, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021 • edited

SilverDestiny commented Nov 9, 2021

felipetce commented Nov 9, 2021 • edited

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

SilverDestiny commented Nov 9, 2021

SilverDestiny commented Nov 9, 2021

SilverDestiny commented Nov 9, 2021

SilverDestiny commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

felipetce commented Nov 9, 2021

SilverDestiny commented Nov 10, 2021

wsliaw commented Oct 29, 2022

miworking3 commented Oct 31, 2022

chrisvanbuskirk commented Feb 20, 2019 •

edited

chrisvanbuskirk commented Aug 6, 2020 •

edited

chrisvanbuskirk commented Aug 6, 2020 •

edited

miworking3 commented Aug 7, 2020 •

edited

chrisvanbuskirk commented Aug 7, 2020 •

edited

chrisvanbuskirk commented Aug 14, 2020 •

edited

felipetce commented Nov 9, 2021 •

edited

felipetce commented Nov 9, 2021 •

edited