Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scaling dropout layer by keep probability during test time #106

Closed

Conversation

Chris-Nicholls
Copy link

From the dropout paper http://www.cs.toronto.edu/~rsalakhu/papers/srivastava14a.pdf :

If a unit is retained with probability p during training, the outgoing weights of that unit are multiplied by p at test time

The test activations should be scaled by (1-drop_prob), not drop_prob.
For example, if drop prob is 0, this layer should have no effect and we should scale activations by 1.

@ratajczak
Copy link

Hi Chris, it looks like duplicate of #61

@Chris-Nicholls
Copy link
Author

Yup, you're right. Looks like this isn't being maintained anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants