boundingBox transformation

[saddy001]

Hey, at first thanks for this useful package.
Currently I'm drawing bounding boxes around text in the live preview (examples/main_live.dart) with CustomPainter. However, when using the ocrElement.boundingBox left, top, right and bottom properties, the box appears displaced and not directly around the recognized words.
So I guess the coordinates must be scaled or shifted somehow.

For top and bottom, there seems to be a constant scaling factor that fixes the issue (e.g. ocrElement.boundingBox.top * 0.96). For left and right, this is not the case: If the detected text is on the left hand side of the image, the box appears too far right and if it's on the right hand side, the box is too far left.

I had a look at the new ScannerOverlayShape in example/main.dart, but there is much scaling going on. Could you explain how the X and Y coordinates must be transformed to display the boxes at the right location?

[jaumard]

Hey @saddy001 did you check https://github.com/bparrishMines/mlkit_demo ? it does draw boxes on detected text already, might help you :)

[saddy001]

Yes, they use the following scaling:

final double scaleX = widgetSize.width / imageSize.width;
final double scaleY = widgetSize.height / imageSize.height;

return Rect.fromLTRB(
    rect.left.toDouble() * scaleX,
    rect.top.toDouble() * scaleY,
    rect.right.toDouble() * scaleX,
    rect.bottom.toDouble() * scaleY,
);

I also tried to scale it this way, with the widgetSize being the size parameter of the paint(Canvas canvas, Size size) function in CustomPainter, and imageSize being the CameraMlVision.size.width and height attributes. The bbox is still displaced. Here is the result:

a	b
:-------------------------:	:-------------------------:
:-------------------------:	:-------------------------:

[saddy001]

I got it. I was using CameraMlVision.size, where I should have used _cameraController.value.previewSize.

However, the CameraMlVisionState._cameraController property is private in flutter_camera_ml_vision.dart. Could you please find a way to make it accessible from outside packages or at least define a get-method for previewSize?

[saddy001]

Thanks!