Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Why is LayerNorm default to False? #16

Closed
mathDR opened this issue Nov 2, 2021 · 2 comments
Closed

Why is LayerNorm default to False? #16

mathDR opened this issue Nov 2, 2021 · 2 comments

Comments

@mathDR
Copy link

mathDR commented Nov 2, 2021

Not an issue, but a question: why is the default LayerNorm function set to False? In particular, for the point cloud example, the LayerNorm is not used.

Can you comment on the importance of having the nested LayerNorm activated for the model? That is, in the paper there was not exposition on having LayerNorm activated versus not.

Thanks!

@juho-lee
Copy link
Owner

juho-lee commented Nov 4, 2021

For our experiments, we found LayerNorm not very effective so set False as our default option. But it depends on the problem you want to solve.

@mathDR
Copy link
Author

mathDR commented Nov 4, 2021

Okay Great. So it sounds like False is a good default for your experiments. Thanks for the clarification!

@mathDR mathDR closed this as completed Nov 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants