New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new Activation function: selu #6969

Closed
wants to merge 2 commits into
base: master
from

Conversation

Projects
None yet
3 participants
@cc786537662
Copy link

cc786537662 commented Jun 13, 2017

add new Activation function : selu

#read paper to know more detail:
Self-Normalizing Neural Networks:https : //arxiv.org/abs/1706.02515
#discussion:
https://www.reddit.com/r/MachineLearning/comments/6g5tg1/r_selfnormalizing_neural_networks_improved_elu/

#usage:
just like relu:
model.add(Dense(64, activation='relu'))
---model.add(Dense(64, activation='selu'))

experiment on mnist:(run keras-master\examples\mnist_cnn.py):
with relu:
Test loss: 0.025632923909
Test accuracy: 0.9915
replace relu in network with selu:
Test loss: 0.052635348707
Test accuracy: 0.9837

cc786537662 added some commits Jun 13, 2017

add new Activation function: selu
add new Activation function: selu
#read paper to know more detail:
Self-Normalizing Neural Networks:https : //arxiv.org/abs/1706.02515
#discussion:
https://www.reddit.com/r/MachineLearning/comments/6g5tg1/r_selfnormalizing_neural_networks_improved_elu/
usage:
just like relu:
model.add(Dense(64, activation='relu'))
---model.add(Dense(64, activation='selu'))
add new Activation function: selu
add new Activation function: selu
#read paper to know more detail:
Self-Normalizing Neural Networks:https : //arxiv.org/abs/1706.02515
#discussion:
https://www.reddit.com/r/MachineLearning/comments/6g5tg1/r_selfnormalizing_neural_networks_improved_elu/
usage:
just like relu:
model.add(Dense(64, activation='relu'))
---model.add(Dense(64, activation='selu'))
@cc786537662

This comment has been minimized.

Copy link

cc786537662 commented Jun 13, 2017

forget to add some test in backend_test.py , PR closed.

@ahundt

This comment has been minimized.

Copy link
Contributor

ahundt commented Jun 13, 2017

@cc786537662 You might want to look at #6969 before you add more changes

@Danielhiversen

This comment has been minimized.

Copy link
Contributor

Danielhiversen commented Jun 13, 2017

@ahundt I think you mean #6924

@cc786537662

This comment has been minimized.

Copy link

cc786537662 commented Jun 13, 2017

nice job, that covers my PR,
thank you guys

@ahundt

This comment has been minimized.

Copy link
Contributor

ahundt commented Jun 13, 2017

Yeah that's what I meant thanks thanks

@mdraw mdraw referenced this pull request Jul 15, 2017

Open

SNN (Self-Normalizing Networks) #12

1 of 3 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment