Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Add Mish activation #8417

Closed
digantamisra98 opened this issue Nov 18, 2019 · 3 comments
Closed

Feature Request: Add Mish activation #8417

digantamisra98 opened this issue Nov 18, 2019 · 3 comments
Assignees

Comments

@digantamisra98
Copy link

@digantamisra98 digantamisra98 commented Nov 18, 2019

Mish is a new novel activation function proposed in this paper.
It has shown promising results so far and has been adopted in several packages including:

All benchmarks, analysis and links to official package implementations can be found in this repository

It would be nice to have Mish as an option within the activation function group.

This is the comparison of Mish with other conventional activation functions in a SEResNet-50 for CIFAR-10: (Better accuracy and faster than GELU)
se50_1

@raver119 raver119 self-assigned this Nov 18, 2019
@raver119

This comment has been minimized.

Copy link
Contributor

@raver119 raver119 commented Nov 18, 2019

Implemented and merged.

@raver119 raver119 closed this Nov 18, 2019
@AlexDBlack

This comment has been minimized.

Copy link
Contributor

@AlexDBlack AlexDBlack commented Nov 18, 2019

And thanks for the detailed feature request @digantamisra98
It should be up on snapshots within a few days (once we merge back) and of course the next release (which we are working towards at present).

@digantamisra98

This comment has been minimized.

Copy link
Author

@digantamisra98 digantamisra98 commented Nov 18, 2019

@AlexDBlack Thanks for considering the request. Looking forward to the release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.