Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix mlp nonlinearity #197

Merged
merged 1 commit into from
Aug 20, 2023
Merged

Fix mlp nonlinearity #197

merged 1 commit into from
Aug 20, 2023

Conversation

ziqi-ma
Copy link

@ziqi-ma ziqi-ma commented Aug 16, 2023

Fix nonlinearity of final MLP - otherwise output is lower-bounded by GELU.

@JeanKossaifi
Copy link
Member

Thanks @ziqi-ma! @m4e7 this fixes the issue you found too right?

@m4e7
Copy link
Contributor

m4e7 commented Aug 18, 2023

Thanks @ziqi-ma! @m4e7 this fixes the issue you found too right?

Yeah, this fixes it! We should also fix MLPLinear.forward(), too: https://github.com/neuraloperator/neuraloperator/blob/a371d0890698eb2617dc805b527449b7acac8581/neuralop/layers/mlp.py#L77C34-L77C34

@JeanKossaifi
Copy link
Member

@ziqi-ma would you be able to make that change too?

@JeanKossaifi
Copy link
Member

I'll merge this @ziqi-ma as this is an important fix and opened #200 to track the other issue if you want to open a PR for it too! Thanks for your contribution!

@JeanKossaifi JeanKossaifi merged commit b470c52 into neuraloperator:main Aug 20, 2023
1 check passed
ziqi-ma pushed a commit to ziqi-ma/neuraloperator that referenced this pull request Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants