[Question] Random parameterization of WarpAffine

[vfdev-5]

I would like to have a random affine data augmentation specified by random angle, scale, shear angle, 2 translations. Using this input I can produce the matrix M (with 6 values) as suggested in the docs.
The problem is that we cannot do arithmetics with TensorReference.

In details:

class AugmentedPipeline(Pipeline):
    
    def __init__(self, batch_size, num_threads, device_id, seed=12, **dataset_kwargs):

        # ...

        # Random affine warp
        self.affine_proba = ops.CoinFlip(probability=0.5)
        self.affine_rand_angle = ops.Uniform(range=[-90.0, 90.0])
        self.affine_rand_scale = ops.Uniform(range=[-0.75, 1.25])
        self.affine_rand_shear = ops.Uniform(range=[-0.15, 0.15])
        self.affine_rand_translate = ops.Uniform(range=[-0.2, 0.2])
        self.affine = ops.WarpAffine(device="gpu", use_image_center=False)
                
    def define_graph(self):
        
        # ...
        
        img_width = 512 // 2   # here it would be better to have `shape`
        img_height = 512 // 2   # here it would be better to have `shape`
        center = (img_width * 0.5, img_height * 0.5)
        angle = self.affine_rand_angle()
        scale = self.affine_rand_scale()
        print(angle.source, scale.source)
        translate = (img_width * self.affine_rand_translate(), img_height * self.affine_rand_translate())
        mask = self.affine_proba()
        
        # The following method computes M matrix values
        matrix = get_inverse_affine_matrix(center=center, angle=angle, scale=scale, translate=translate)

        images = self.affine(images, matrix=matrix, fill_value=0, interp_type=types.INTERP_CUBIC, mask=mask)
        masks = self.affine(masks, matrix=matrix, fill_value=0, interp_type=types.INTERP_NN, mask=mask)        

        return (images, masks)        

Any suggestions on how to achieve that ?
Thanks !

[JanuszL]

Hi,
For that, you need to create custom op that does the math inside. To make that simpler you can try to use the approach described in #410 - where build-in op is called from the custom one with only some custom arg preprocessing.

[vfdev-5]

@JanuszL thank you for the suggestion ! I close the issue as I got the answer.

Any plans to provide random warp affine ops ?

[JanuszL]

Any plans to provide random warp affine ops ?

Currently, we are focusing on providing operators that are used in the most common networks. Adding ops you are asking for is something that is more than welcome as an external contribution.

[vfdev-5]

@JanuszL Thanks, I see your point. Maybe I misunderstood your docs, but I would like to perform the following data augementation on image and mask:

• rotation (ops.Rotate)
• scale (?)
• XY translations (?)
• shear (?)

Such ops are also rather common. Which ops should I use for that ?

[JanuszL]

Hi,

• rotation - ops.Rotate
• scale - ops.Resize
• XY translations - ops.WarpAffine
• shear - ops.WarpAffine

However, only Rotate will accept argument coming from the random generator. For XY translations and shear it should be easy to create something on your own using flip operator as an example here and here - you need to add Shx and Shy args and populate transformation matrix accordingly in Prepare.
For resize you can try to extend RandomResizeCrop and replace SetupSharedSampleParams that basically tells how to resize and if crop.
Regarding segmentation itself - we haven't got into that area yet sticking to other commitments. Nevertheless, any contribution getting DALI closer to that is more than welcome.