questions about find the keypoints of rotated objects

[MianMianMeow]

Hi,
Thanks for you great work~ I'm using RTMpose to find the ID cards' 4 corner points. In my case the cards are always rotated by [-180, 180] degrees, and I noticed the trained model does not work with the rotated cards. Below is my train_pipeline, which I modified the rotate_factor=180 for RandomBBoxTransform but it seems there's no improvement. Perhaps I did the data augmentation in a wrong way.

train_pipeline = [
    dict(type='LoadImage', backend_args=backend_args),
    dict(type='GetBBoxCenterScale'),
    # dict(type='RandomFlip', direction='horizontal'),
    # dict(type='RandomHalfBody'),
    dict(
        type='RandomBBoxTransform', scale_factor=[0.5, 1.5], rotate_factor=180),

    dict(type='TopdownAffine', input_size=codec['input_size']),
    dict(type='mmdet.YOLOXHSVRandomAug'),
    dict(
        type='Albumentation',
        transforms=[
            dict(type='Blur', p=0.2),
            dict(type='MedianBlur', p=0.2),
            dict(
                type='CoarseDropout',
                max_holes=1,
                max_height=0.4,
                max_width=0.4,
                min_holes=1,
                min_height=0.2,
                min_width=0.2,
                p=1.0),
        ]),
    dict(type='GenerateTarget', encoder=codec),
    dict(type='PackPoseInputs')
]

Do you have any suggestions for using RTMpose to find the keypoints of strongly rotated objects?

Best regards

[Ben-Louis]

Thank you for using MMPose. Have you checked if the data and annotations have been transformed as expected? We have provided a convenient tool, the dataset browser, to visualize the augmented data.

[MianMianMeow]

Thanks for your reply. By using tools/misc/browse_dataset.py , I checked the annotations of the transformed data and most of them are good. some of the rotated transformed cards having their corner points outside of the image. Not sure if this situation is right. Do you have any suggestions?
original image

transformed image

[Ben-Louis]

Based on this visualization result, I suggest expanding the size of bounding boxes during augmentations. You could try to modify some transforms as follows:

    ...
    dict(type='GetBBoxCenterScale', padding=1.5),
    dict( type='RandomBBoxTransform', scale_factor=[0.75, 1.25], rotate_factor=180),
    ...

This might help get rid of the absence of corners

[MianMianMeow]

Amazing~ I'll have a try. Thanks a lot.