Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for dimension selection for layer normalization #1766

Open
WilliamTambellini opened this issue Dec 6, 2023 · 1 comment
Open
Assignees
Labels
enhancement A feature or an optimization request help wanted

Comments

@WilliamTambellini
Copy link
Contributor

Summary

Add support for dimension selection for layer normalization
Atm the normalization axis is imposed to be the last logical dim:
https://oneapi-src.github.io/oneDNN/dev_guide_layer_normalization.html#difference-between-forward-training-and-forward-inference

Problem statement

Cannot select which axis to normalize on.

Preferred solution

a new runtime param for the norm prim execution.

We are open to prepare a PR if ok.

@WilliamTambellini WilliamTambellini added the enhancement A feature or an optimization request label Dec 6, 2023
@vpirogov
Copy link
Member

vpirogov commented Dec 8, 2023

@WilliamTambellini, thanks for the offer. Considering that this feature involves API changes an RFC describing the problem statement and API changes would be great starting point. An implementation is welcome as well.

@vpirogov vpirogov self-assigned this Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A feature or an optimization request help wanted
Projects
None yet
Development

No branches or pull requests

2 participants