The use of the function RandomRotation90 for image augmentation for TF object detection

[sulebaynes]

Hi Tensorflow community,

I am training an object detection model using the TF object detection API. I believe that the documentation about how to use of the function RandomRotation90 for image augmentation is not clear enough. I would like to use this feature as one of the data augmentation options to train my model. So if it is possible I would like to rotate the input images by 90, 180 and 270 degrees regardless of the direction being clock- or anticlockwise.

The documentation mentions that this function takes the argument keypoint_rot_permutation however this is not clear to me what it does exactly. The first link states that:

Specifies a mapping from the original keypoint indices to 90 degree counter clockwise indices. This is used in the event that keypoints are specified, in which case when the image is rotated the keypoints might need to be permuted.

which is very ambiguous to my understanding.

The entire URL of the documentation with the issue

https://github.com/tensorflow/models/blob/master/research/object_detection/protos/preprocessor.proto
https://github.com/tensorflow/models/blob/master/research/object_detection/builders/preprocessor_builder_test.py

Describe the issue

What is the difference between using no keypoint_rot_permutation or using one of them or all of them?
either:

random_rotation90 {
      probability: 0.5
    }

or

random_rotation90 {
      keypoint_rot_permutation: 3
      probability: 0.5
    }

or

random_rotation90 {
      keypoint_rot_permutation: 3
      keypoint_rot_permutation: 0
      keypoint_rot_permutation: 1
      keypoint_rot_permutation: 2
      probability: 0.5
    }

Is there a way to rotate the images with 90, 180 and 270?

[sulebaynes]

Update: I found a way to rotate the images by 90, 180 and 270 degrees by adding the following to the pipeline.config

data_augmentation_options {
    random_rotation90 {
      probability: 0.20
    }
  }
 data_augmentation_options {
    random_rotation90 {
      probability: 0.20
    }
  }
 data_augmentation_options {
    random_rotation90 {
      probability: 0.20
    }
  }
etc.

however I still do not understand what keypoint_rot_permutation does.

[Petros626]

I think this data augmentation option fits better for keypoints. For example in faces like the eyes, mouth etc. If this solves your question please close this issue