Replies: 1 comment 2 replies
-
sounds reasonable. Are you open to propose a PR ? /cc @shijianjian |
Beta Was this translation helpful? Give feedback.
2 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
In VideoSequential, there is the parameter same_on_frame. If True, each augmentation uses the same _params across frames. However, for gaussian noise, that makes a fixed pattern that repeats accross frames, because it does:
gaussian_noise = _randn_like(input, mean=flags["mean"], std=flags["std"])
self._params["gaussian_noise"] = gaussian_noise
There is such a thing as noise identical between frames in real life, it's called fixed pattern noise, but usually for most sensors it is very small compared to the noise that changes from frame to frame.
So in a video augmentation pipeline like
K.VideoSequential(
K.RandomPlanckianJitter(mode="blackbody"),
K.RandomGamma(gamma=(0.7, 2), gain=(0.7, 1.5), p=0.5),
K.RandomGaussianNoise(mean=0.0, std=0.05, p=0.25),
K.RandomMedianBlur(kernel_size=(3, 3), p=0.25),
K.RandomSaturation(saturation=(0.8, 1.0), p=0.5),
K.RandomSharpness(sharpness=(0.0, 0.5), p=0.5),
torch.nn.Hardtanh(min_val=0, max_val=1.0, inplace=True),
data_format="BTCHW",
same_on_frame=True,
)
The RandomGaussianNoise is not really behaving as expected. I think most users would expect a different noise realization for each frame, but that the video is noisy or not with the given probability. I see that some augmentations like ColorJiggle and ColorJitter are handled specially by VideoSequential. I think GaussianNoise should also have a special behaviour and use frame-varying noise.
What do you think?
Beta Was this translation helpful? Give feedback.
All reactions