Skip to content
This repository has been archived by the owner on Feb 26, 2023. It is now read-only.

Add LayerNorm and GroupNorm #58

Merged
merged 2 commits into from Jan 28, 2022
Merged

Add LayerNorm and GroupNorm #58

merged 2 commits into from Jan 28, 2022

Conversation

lkhphuc
Copy link
Contributor

@lkhphuc lkhphuc commented Jan 26, 2022

Add LayerNorm and GroupNorm, also move rename BatchNorm file to Norm.

Unrelated question: I saw in the Treex's module, you defined module's properties as both class variables (dataclass style) and as init's parameters? What is the reason behind this? I though the point of dataclass-like attribute is to reduce the boilerplate in __init__?

@cgarciae
Copy link
Owner

cgarciae commented Jan 26, 2022

Hey @lkhphuc, thanks a lot! I'll give it a review soon.

Meanwhile I'll answer your question:

Unrelated question: I saw in the Treex's module, you defined module's properties as both class variables (dataclass style) and as init's parameters? What is the reason behind this? I though the point of dataclass-like attribute is to reduce the boilerplate in init?

There are 2 reasons I think, the first one is that using class variables is a good way to provide metadata for fields regardless if its a dataclass or not. The second one is that if you do it like this you immediately have support for dataclasses and non-dataclasses which is ideal because you want to have flexibility, Treex (or rather treeo) supports both.

@cgarciae
Copy link
Owner

@lkhphuc reviewed it, I have no comments, great job! 🎉

@cgarciae cgarciae merged commit d83aca4 into cgarciae:master Jan 28, 2022
@lkhphuc lkhphuc deleted the norm branch March 12, 2022 09:09
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants