Can I do keypoint occlusion with this library?

[offchan42]

Suppose that I want to randomly put another image on top of the training image, or some synthetic erasing, the library has to tell me which points are occluded.

For example, if I have an image of a face with some keypoints, I want to put a finger image with transparent background on top of some of the keypoints (randomly), is this something achievable with this library?

[aleju]

I don't think what you want is part of the library. There is no occlusion flag associated with keypoints. There is also not yet an augmenter that pastes arbitrary images onto training images.

[offchan42]

OK. Does this library support random erasing on a small image patch?

[offchan42]

Also when I rotate or zoom the image, some of the keypoints will likely be outside the image, how do you deal with this problem for training a coordinate regression model? Do I need to just check the x and y if it's outside the image before calculating heatmap?

[aleju]

The library has CoarseDropout to remove larger sub-areas. Note though that the augmenter currently does not remove keypoints within these dropped areas. Depending on their size that may be desirable (small dropped areas) or undesirable (large dropped areas).

To remove keypoints you currently have to check manually the xy-coordinates. PR #487 will add KeypointsOnImage.clip_out_of_image() and the corresponding augmenter ClipCBAsToImagePlanes, which both remove keypoints outside of the image plane. But the patch is not yet merged.

[offchan42]

OK. This explanation clears my issue. Thank you!