Is it possible to decode the QR code image after shooting it with the camera?

[AhnSungHyeon]

We are currently developing an application that uses the Zxing Library to scan QR codes.

The problem here is the time to recognize the QR code according to the camera performance of the smartphone scanning the QR code.

So the way I want to solve is to crop the QR code part of the image shot by the camera and then decode the QR code.

I looked for related materials and found the following material
https://stackoverflow.com/questions/32120988/qr-code-decoding-images-using-zxing-android
I tried but failed.

I can not find the cause of why I failed.

Below is the code I created.
I checked the error log.
In the code below, the source and binaryBitmap data is transparent.

    @Override
    protected void onActivityResult (int requestCode, int resultCode, intent data) {
        if (requestCode == CropImage.CROP_IMAGE_ACTIVITY_REQUEST_CODE) {
            CropImage.ActivityResult result = CropImage.getActivityResult (data);
            if (resultCode == RESULT_OK) {
                Uri resultUri = result.getUri ();

                cropImageView.setImageUriAsync (resultUri);

                try {
                    Bitmap bm = MediaStore.Images.Media.getBitmap (getContentResolver (), resultUri);

                    QRCodeImage_to_Decode (bm);
                    // byte [] currentData = bitmapToByteArray (bm);

                } catch (IOException e) {
                    e.printStackTrace ();
                }

            } else if (resultCode == CropImage.CROP_IMAGE_ACTIVITY_RESULT_ERROR_CODE) {
                Exception error = result.getError ();
            }
        }
    }
-----------------------------------------------------------------------------------------------------
 public void QRCodeImage_to_Decode(Bitmap bitmap){
        Bitmap generatedQRCode = bitmap;
        Log.e("QRCodeImage_to_Decode","generatedQRCode : "+generatedQRCode);
        int width = generatedQRCode.getWidth();
        Log.e("QRCodeImage_to_Decode","width : "+width);
        int height = generatedQRCode.getHeight();
        Log.e("QRCodeImage_to_Decode","height : "+height);
        int[] pixels = new int[width * height];
        Log.e("QRCodeImage_to_Decode","pixels : "+pixels.length);
        generatedQRCode.getPixels(pixels, 0, width, 0, 0, width, height);

        RGBLuminanceSource source = new RGBLuminanceSource(width, height, pixels);
        Log.e("QRCodeImage_to_Decode","source : "+source);

        BinaryBitmap binaryBitmap = new BinaryBitmap(new HybridBinarizer(source));
        Log.e("QRCodeImage_to_Decode","binaryBitmap : "+binaryBitmap);

        QRCodeReader reader = new QRCodeReader();
        Log.e("QRCodeImage_to_Decode","reader : "+reader);
        com.google.zxing.Result result = null;
        Log.e("QRCodeImage_to_Decode","result#1 : "+result);
        try {
            result = reader.decode(binaryBitmap);
            Log.e("QRCodeImage_to_Decode","result#2 : "+result);
        } catch (NotFoundException e) {
            e.printStackTrace();
        } catch (ChecksumException e) {
            e.printStackTrace();
        }  catch (com.google.zxing.FormatException e) {
            e.printStackTrace();
        }
        String text = result.getText();
        Toast.makeText(this,text,Toast.LENGTH_SHORT).show();
        Log.e("QRCodeImage_to_Decode","text : "+text);
    }
------------------------------------------------------------------------------------------------------------
05-16 22:39:15.384 18837-18837/? E/QRCodeImage_to_Decode: generatedQRCode : android.graphics.Bitmap@75d892b
05-16 22:39:15.384 18837-18837/? E/QRCodeImage_to_Decode: width : 2564
05-16 22:39:15.384 18837-18837/? E/QRCodeImage_to_Decode: height : 2424
05-16 22:39:15.384 18837-18837/? E/QRCodeImage_to_Decode: pixels : 6215136
05-16 22:39:16.510 18837-18837/? E/QRCodeImage_to_Decode: source :                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                
05-16 22:39:18.347 18837-18837/? E/QRCodeImage_to_Decode: binaryBitmap :                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      
05-16 22:39:18.370 18837-18837/? E/QRCodeImage_to_Decode: reader : com.google.zxing.qrcode.QRCodeReader@4e0b188
05-16 22:39:18.371 18837-18837/? E/QRCodeImage_to_Decode: result#1 : null
05-16 22:39:18.408 18837-18837/? E/AndroidRuntime: FATAL EXCEPTION: main
                                                   Process: com.example.cameranostest, PID: 18837
                                                   java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=203, result=-1, data=Intent { (has extras) }} to activity {com.example.cameranostest/com.example.cameranostest.Main2Activity}: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String com.google.zxing.Result.getText()' on a null object reference
                                                       at android.app.ActivityThread.deliverResults(ActivityThread.java:3758)
                                                       at android.app.ActivityThread.handleSendResult(ActivityThread.java:3801)
                                                       at android.app.ActivityThread.access$1400(ActivityThread.java:157)
                                                       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1405)
                                                       at android.os.Handler.dispatchMessage(Handler.java:102)
                                                       at android.os.Looper.loop(Looper.java:148)
                                                       at android.app.ActivityThread.main(ActivityThread.java:5551)
                                                       at java.lang.reflect.Method.invoke(Native Method)
                                                       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:730)
                                                       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:620)
                                                    Caused by: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.String com.google.zxing.Result.getText()' on a null object reference
                                                       at com.example.cameranostest.Main2Activity.QRCodeImage_to_Decode(Main2Activity.java:160)
                                                       at com.example.cameranostest.Main2Activity.onActivityResult(Main2Activity.java:54)
                                                       at android.app.Activity.dispatchActivityResult(Activity.java:6463)
                                                       at android.app.ActivityThread.deliverResults(ActivityThread.java:3754)
                                                       at android.app.ActivityThread.handleSendResult(ActivityThread.java:3801) 
                                                       at android.app.ActivityThread.access$1400(ActivityThread.java:157) 
                                                       at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1405) 
                                                       at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                       at android.os.Looper.loop(Looper.java:148) 
                                                       at android.app.ActivityThread.main(ActivityThread.java:5551) 
                                                       at java.lang.reflect.Method.invoke(Native Method) 
                                                       at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:730) 
                                                       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:620) 

[rkistner]

This library (and zxing:core) is designed around continuous scanning. That means it optimizes performance per image over detection results for any specific image. So instead of doing a lot of effort to scan a specific image, it relies on the fact that a lot more (different, probably better quality) images will come in soon, and it will try again on those.

That means that the approach of trying to take a photo and scanning a single image like you tried will not work well, since the library is not designed for that use case.

[chungyau97]

This library (and zxing:core) is designed around continuous scanning. That means it optimizes performance per image over detection results for any specific image. So instead of doing a lot of effort to scan a specific image, it relies on the fact that a lot more (different, probably better quality) images will come in soon, and it will try again on those.

That means that the approach of trying to take a photo and scanning a single image like you tried will not work well, since the library is not designed for that use case.

Hi,
I use 2 android phone for this test case.
1st phone use Camera to scan 2nd phone screen QR code (125x125 resolution) from gallery that is using image view to preview it and is fixed size to (125x125 dp) (dp is equivalent to px).
2nd phone scan the QR code (125x125 resolution) from gallery.

Result
1st phone always success
2nd phone will fail in some QR Code Picture.

So I added picture rotation method (90 clockwise & 90 anti clockwise & 180).
So far it improve some picture that fail to scan.

@rkistner What do you think to improve the success rate of 2nd phone?

[rkistner]

That is currently outside the scope of this project, since it would require a decoder with different trade-offs. This library currently just uses the decoder of zxing:core, so any changes like this will not be supported.

[typelogic]

This library (and zxing:core) is designed around continuous scanning. That means it optimizes performance per image over detection results for any specific image. So instead of doing a lot of effort to scan a specific image, it relies on the fact that a lot more (different, probably better quality) images will come in soon, and it will try again on those.

That means that the approach of trying to take a photo and scanning a single image like you tried will not work well, since the library is not designed for that use case.

This explains, why I am seeing very intermittent QR read error from generated PNG images of my test cases. I loop it thousand of times, and I get intermittent read errors with no correlation at all to CPU and memory usage. I reduced the error scenario when I saved the particular QR code PNG when the QR scan error occurred and I feed this one PNG file for a read and zxing could not read it. I tried the zxing on-line reader, and it could not read also. I then, used BinaryEye which uses zxing under the hood, and it was able to read the QR code in video scanning mode! It was at this point, that I sensed a certain probabilistic component in zxing's algorithm but I don't know how to describe it. Until now. Thanks.