Transformer/fix layer norm#1404

Open

RezaYazdaniAminabadi wants to merge 5 commits intomasterfrom

transformer/fix-layer-norm

Commits on Sep 24, 2021

fix the workspace allocation for the transformer kernel
Reza Yazdani
committed

Commits on Sep 27, 2021

Modify normalize kernels to save variance in floating-point for backward
Reza Yazdani
committed
remove unused grad keys
Reza Yazdani
committed

Commits on Sep 28, 2021

Merge branch 'master' into transformer/fix-layer-norm
jeffra
authored

Commits on Oct 5, 2021

fix some issue with saving the variance for normalize forward kernel
Reza Yazdani
committed