Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

activation function composer #398

Open
silky opened this Issue Jun 5, 2018 · 1 comment

Comments

Projects
None yet
1 participant
@silky
Copy link
Owner

silky commented Jun 5, 2018

or more generally, a function composer

  1. what does the graph of relu look like?
  2. what about the graph of relu . tanh ?

and so on, indefinitely and arbitrarily.

some features:

  • what points should be push through? maybe could add certain kinds of initialisations and ranges
  • add things like drop-out and whatnot.
@silky

This comment has been minimized.

Copy link
Owner Author

silky commented Jun 5, 2018

using hoe and plot

hoe 'map (tanh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map (tanh . tanh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map ((\r -> if r > 0 r else 0) . tanh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map ((\r -> if r > 0 then r else 0) . tanh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map (tanh . (\r -> if r > 0 then r else 0) . tanh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map (tanh . (\r -> if r > 0 then r else 0) . tanh . tanh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map (tanh . (\r -> if r > 0 then r else 0) . tanh . sinh . (\x -> (x - 500)/50)) [0..1000]' | plot
hoe 'map (tanh . (\r -> if r > 0 then r else 0) . tanh . (\s -> exp s / (exp s + 1)) . (\x -> (x - 500)/50)) [0..1000]' | plot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.