Return box coordinates of scanned codes

[dustin-graham]

Hello,
I'm very pleased with this plugin. One thing that is missing for us is the ability to paint boxes around the scanned codes from the camera. I know MLKit reports the coordinates of the scanned codes, would it be possible to report coordinates along with the barcode values?

Thank you!

[jhoogstraat]

Hi, glad you like the plugin!

That is totally something I will implement into 2.0!
Just please bear with my, as I won't have time to work in the plugin until next year, as I am in my critical phase of my masters thesis right now.

In fact if you are proficient in flutter, Swift or Kotlin, I will gladly accept any pull request on the develop branch (will will become 2.0) :)

[dustin-graham]

I'm happy to contribute. I'll see if I can get this done in the next week or so.

…

On Sun, Dec 12, 2021, 8:54 AM jhoogstraat ***@***.***> wrote: Hi, glad you like the plugin! That is totally something I will implement into 2.0! Just please bear with my, as I won't have time to work in the plugin until next year, as I am in my critical phase of my masters thesis right now. In fact if you are proficient in flutter, Swift or Kotlin, I will gladly accept any pull request on the develop branch (will will become 2.0) :) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAHKLO5YED3UMLKCAUII2C3UQTATVANCNFSM5J3TYS5A> .

[dustin-graham]

@jhoogstraat , I'm working on this feature right now. I have one initial question as I look at this code. Is there a specific reason why the plugin only sends the first code it scans?

@SuppressLint("UnsafeOptInUsageError")
    private fun initialize(configuration: HashMap<String, Any>): Task<PreviewConfiguration> {
        if (this.camera != null)
            throw ScannerException.AlreadyInitialized()

        val pluginBinding = this.pluginBinding ?: throw ScannerException.ActivityNotConnected()
        val activityBinding = this.activityBinding ?: throw ScannerException.ActivityNotConnected()

        val camera = Camera(
            activityBinding.activity,
            pluginBinding.textureRegistry.createSurfaceTexture(),
            configuration
        ) { barcodes ->

            // *** Question: should we return all the codes? *****
            detectionEventSink?.success(encode(barcodes.first()))

        }

        this.camera = camera

        activityBinding.addRequestPermissionsResultListener(camera)

        return camera.requestPermissions()
            .continueWithTask { camera.loadCamera() }
    }

In my use case, my drivers scan a packing label that may have multiple codes on it. The native functionality is to return a list of all visible codes. Would it be ok to just pass them all through?

[jhoogstraat]

Yeah definitely. My use case just only required scanning one code at a time.
It even does on the develop branch:

fast_barcode_scanner/fast_barcode_scanner/android/src/main/kotlin/com/jhoogstraat/fast_barcode_scanner/FastBarcodeScannerPlugin.kt

Line 174 in 6104e14

detectionEventSink?.success(encode(barcodes.first()))

[dustin-graham]

@jhoogstraat , I'm continuing work today on this. I'm looking at the event channel messages we are sending between native and dart. Since I'm adding in coordinates and also passing through all the barcodes instead of just the first in the list, I'd like to move to something like Pigeon to make the platform communication easier. https://pub.dev/packages/pigeon. Any objections to this?

[jhoogstraat]

I never worked with Pigeon. Does using it mean to rewrite the whole communication part between native and flutter?
I tried to keep the implementation as lean as possible for now, but I see that something like Pigeon does have benefits. If you think Pigeon is worth the effort, I support the idea.

[dustin-graham]

@jhoogstraat , thanks for the feedback. Pigeon is new to me, but I've been looking into it. But yes, Pigeon would replace the current communication layer with generated code. It looks to be a fairly large lift, so for now I'm going to first get box coordinates working for the single barcode with the communication layer as-is and then I'll probably open up a separate issue for supporting multiple barcodes at once. Pigeon makes more sense when supporting multiple codes I think.

I'm almost done with drawing the boxes around the barcode. It took a little bit of thinking to make sure the coordinate translation is correct.

I'm going to be adding a CodeBoundaryOverlay to the library which will do the drawing. I'm going to provide an API that will allow the user to specify the Paint. My use case includes a requirement to print the code value along with the box. I'm going to try and find a nice API so that users can add their own additions to the box.

Do you have any additional requests or suggestions for customization?

[dustin-graham]

@jhoogstraat , I was thinking that I need to account for different camera orientations. The orientation system can often be confusing to me, so I apologize if I misunderstand, but it appears that the current code assumes portrait orientation? That seems to be how the native camera code is configured and he sample is locked into portrait. Is landscape orientation supposed to be supported for scanning? I'd actually be completely comfortable if we just support portrait orientation. That is the most conducive to effective scanning UI I think. But please let me know if I'm mistaken.

[jhoogstraat]

Yeah, good idea to put Pigeon into another issue.
Also the CodeBoundaryOverlay is a very cool use of the overlay mechanism!
Allowing the user to provide a Paint should allow the user to completely configure the border, no?

One notion: You only need two coordinates (diagonals) to fully describe the rectangle. I dont know any code type that is not rectangular or square.

I have to ask: Do you base your changes onto the main or the develop branch?
Cause I changed the camera quite a bit.

Orientation and resolution are the hardest thing about cameras and viewfinders. I am okay with portrait only.

[dustin-graham]

@jhoogstraat , I'm based on develop right now.

With regard to coordinates, I'm sending all 4 corners to provide a nice, neat fit. The codes are square, but if your camera is pointed at the code at an angle, the square code is skewed in the 2D image. sending all 4 corners allows us to match the camera skew.

This is especially nice when the camera is rotated slightly. If you just draw the box it creates an upright box that circumscribes all the corners. This looks better:

[jhoogstraat]

You are right, totally missed that!
That looks really really good, I am looking forward to your pr!

[dustin-graham]

@jhoogstraat PR is up.