-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
About DifferentiableAugment #15
Comments
This implementation utilizes the straight-through gradient estimator, which is also utilized by GDAS. |
@latstars . Thank you for your answer. I understand. I have another question. |
Hi, Raja. Maybe you use a larger dataset, then the iteration is more than the small dataset. However, more iteration will lead to over-optimize, since there is no l2-normalization (weight-decay). I suggest that you can try to search the policy with less epoch or smaller learning rate for augmentation parameters. |
@latstars . Hi. Thank you for your answer. This is impressive. When I reduced the learning rate the policies seem good. In order to validate it, I have conducted several experiments by varying learning rates from 0.5 to 0.0001 where 0.5,0.2,0.1,0.002,0.005 are not working well. But, 0.001, 0.0001,0.0002 are fine (Working fine means that the policies look like your experiment policies --> Just a visual inspection). Now, I'm facing another question. Which policy is good.? How can I determine the policies are good.? Any ideas you can give? The following result is an example of experiments with a learning rate of 0.0001. |
Hi,
I don't understand the DifferentiableAugment class in the implementation. What does it do? Just subtract and add magnitude with images?. Why you have adopted such as this? Is there any specific reason for it?
class DifferentiableAugment(nn.Module):
def init(self, sub_policy):
super(DifferentiableAugment, self).init()
self.sub_policy = sub_policy
The text was updated successfully, but these errors were encountered: