-
Notifications
You must be signed in to change notification settings - Fork 93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added basic Keras support; fixes "None" gradient issue #6
Conversation
Fixed whitespace at end of file and between functions
FYI: This is the error/warning I get from tensorflow:
|
In a case of a GAN, where two YFOptimizers are dueling, and the discriminator's loss functions linearly increases from 0 over time, I consistently get an error along these lines in a few epochs:
I am not sure if it is this PR or another, but I can only run YFOptimizer with this PR's code. |
@JDvorak -- quick clarification question; this happens with this PR and the master repo? Or just with this PR? |
@jmhessel Thanks for the PR. @JDvorak, looking from my side, that is mostly like an exploding gradient or zero gradient issue. Could you do the following:
If you could catch the exception and redo the specific iteration, it should also help solve the problem. Not sure what is the proper way to do exception handling in Tensorflow :). |
Hi, @jmhessel |
Hi @jinxin0924, In the readme, we actually recommend YFOptimizer(learning_rate=1.0, momentum=0.0), but I am not sure how you should use it in Keras. Cheers, |
@JianGoForIt and added compute_gradients function in yellowfin.py. |
@JianGoForIt When I am back from vacation, I'll get you those answers. Otherwise, great work! I'm sure I am not alone in wishing to be rid of worrying about the learning rate hyperparameter. |
@jinxin0924 @JianGoForIt I don't know for sure if running with the recommended parameters will solve all of the performance issues, but, it's quite easy to run with the recommended parameters using keras.
should do the trick. However, why not just make the default parameters those? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the commits on python 3 and compute gradient api
This PR...
model.compile(loss='mse', opt=TFOptimizer(YFOptimizer()))
Some concerns I still have...