-
Notifications
You must be signed in to change notification settings - Fork 393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add rotated boxes #281
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for the massive PR! A few things to change, let me know if you have questions!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Few things to handle and we're good to go!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Almost there!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All good!
This PR introduces a major feature: rotated bounding boxes.
Boxes were caracterized by a 4-dimension tuple: (xmin, ymin, xmax, ymax), and we introduce a new kind of rotated boxes caracterized by a 5-dimension tuple: (x, y, w, h, alpha), where (x, y) is the center of the box, (w, h) its spatial extension, and alpha the angle of rotation of the box (in degrees) to the horizontal (horizontal box: alpha = 0°).
For the moment, support is only provided for 2 quarters of the circle: only angles of rotation between -90° and +90° are supported (either up-rotated or down-rotated). As shown below, it is impossible to determine more than 1 quarter of 360° without any postprocessing tricks, however, we used the mean aspect ratio of boxes to extend this area to 180°.
However, this allows us to handle most of the rotated documents, which are usually slightly rotated with respect to the horizontal.
The following modules are modified accordingly:
The 2 options for boxes geometry are available in each modules, and the "non-rotated" option is activated by default everywhere.
To instantiate a predictor with rotated boxes, please use:
predictor = models.OCRPredictor(rotated_detection_predictor, recognition_predictor, rotated_bbox=True)
where:
rotated_detection_predictor
is a detection predictor instantiated with an architecture trained on rotated docs ('rotated_bbox' = True
incfg
). However, you can also force any architecture to predict rotated boxes, passing a kwarg"rotated_bbox"=True
in the constructor:rotated_detection_predictor = detection.DetectionPredictor(PreProcessor(output_size=(512, 512), batch_size=batch_size), detection.db_resnet50(rotated_bbox=True, input_shape=(512, 512, 3)))
To plot rotated boxes, use:
page.show(img, rotation=True)
ordocument.show([img1, img2, ...], rotation=False)
If you want to train or evaluate a model with rotated boxes, please pass the argument
rotation = True
in the scriptsevaluate.py
anddetection/train.py
, it will instantiate datasets and metrics properly.Any feedback is welcome!