Skip to content

Unify normalization layers#1545

Merged
copybara-service[bot] merged 2 commits into
google:mainfrom
jheek:unified-normalization
Sep 20, 2021
Merged

Unify normalization layers#1545
copybara-service[bot] merged 2 commits into
google:mainfrom
jheek:unified-normalization

Conversation

@jheek
Copy link
Copy Markdown
Contributor

@jheek jheek commented Sep 16, 2021

  • Refactor normalization helpers to share code and remove subtle differences in implemenation.
  • Use clipping to avoid negative variances due to fusion optimization.

@jheek jheek requested a review from levskaya September 16, 2021 09:18
@google-cla google-cla Bot added the cla: yes label Sep 16, 2021
- Refactor normalization helpers to share code and remove subtle differences in implemenation.
- Use clipping to avoid negative variances due to fusion optimization.
@jheek jheek force-pushed the unified-normalization branch from b3bace1 to 80c91a0 Compare September 16, 2021 12:33
Copy link
Copy Markdown
Collaborator

@levskaya levskaya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks for unifying all this.

Comment thread flax/linen/normalization.py Outdated
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

Merging #1545 (6dadc9e) into main (f458637) will increase coverage by 0.08%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1545      +/-   ##
==========================================
+ Coverage   82.73%   82.81%   +0.08%     
==========================================
  Files          68       68              
  Lines        5572     5576       +4     
==========================================
+ Hits         4610     4618       +8     
+ Misses        962      958       -4     
Impacted Files Coverage Δ
flax/linen/normalization.py 96.61% <100.00%> (+3.56%) ⬆️
flax/core/scope.py 93.78% <0.00%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f458637...6dadc9e. Read the comment docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants