New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
reproducible results #42
Comments
It's extremely hard to get a deterministic model since you'll have to control the random seeds properly in each Keras layer, e.g. layers like dropout, initializers, etc. have randomness built in. These will have to be controlled well in Keras' Python front-end. If you really want a very robust model, you really should try to take advantage of this randomness nature when train the model. Hope this helps. Closing this for now. |
Can we reopen this? R (and R users) highly prize reproducibility. It is difficult to generate teaching materials, reports, books, and other items if we get different results on every run. I understand the issue about seeds in different places. Perhaps we could set them to something similar to |
Is this issue happening in Python API as well? If so, this should really be an issue there. |
Here is what we have currently in the FAQ: https://keras.rstudio.com/articles/faq.html#how-can-i-obtain-reproducible-results-using-keras-during-development If there is something we can do in the package that facilitates better reproducibility I'm all for it, just let me know what you think we need to do. |
I think setting both R and TF seeds is the best we can probably do here.
…On Sat, Sep 2, 2017 at 6:48 AM JJ Allaire ***@***.***> wrote:
Here is what we have currently in the FAQ:
https://keras.rstudio.com/articles/faq.html#how-can-i-obtain-reproducible-results-using-keras-during-development
If there is something we can do in the package that facilitates better
reproducibility I'm all for it, just let me know what you think we need to
do.
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub
<#42 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AEEnSpr7nBZQdkt5An2c5b6JVttvnTPqks5seTJ0gaJpZM4N7ByA>
.
|
There are some additional Python/NumPy random seeds set in this Keras
Python example:
https://keras.io/getting-started/faq/#how-can-i-obtain-reproducible-results-using-keras-during-development
Do we need to set those as well?
If so perhaps we should add a `session_with_seed` function that sets all of
these values and returns the session (invisibly, since users typically
won't need to compute on it). The `session_with` prefix is important b/c as
a side effect this also sets the TF session.
Actually, perhaps it should be `tf_session_with_seed` and the function
belongs in the tensorflow package?
On Sat, Sep 2, 2017 at 10:06 AM, Yuan (Terry) Tang <notifications@github.com
… wrote:
I think setting both R and TF seeds is the best we can probably do here.
On Sat, Sep 2, 2017 at 6:48 AM JJ Allaire ***@***.***>
wrote:
> Here is what we have currently in the FAQ:
> https://keras.rstudio.com/articles/faq.html#how-can-i-
obtain-reproducible-results-using-keras-during-development
>
> If there is something we can do in the package that facilitates better
> reproducibility I'm all for it, just let me know what you think we need
to
> do.
>
> —
> You are receiving this because you modified the open/close state.
> Reply to this email directly, view it on GitHub
> <#42 (comment)>, or
mute
> the thread
> <https://github.com/notifications/unsubscribe-auth/
AEEnSpr7nBZQdkt5An2c5b6JVttvnTPqks5seTJ0gaJpZM4N7ByA>
> .
>
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#42 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAGXxzmVWWaoOPbBe0rtyj8-15kR4wYZks5seWD-gaJpZM4N7ByA>
.
|
@jjallaire Sounds good to me! I took a stab at rstudio/tensorflow#174 for this. This will be useful for small computations that can run in single thread but unfortunately not too useful for multi-thread though. I do believe people nowadays really try take advantage of this non-reproducibility to create more robust models as well as achieve model parallelism. I would say give it a shot once that PR got merged. I think |
To get it from the |
See docs on how this now works here: https://keras.rstudio.com/articles/faq.html#how-can-i-obtain-reproducible-results-using-keras-during-development |
Hi everyone!
Each time I train a model on the same data set, I receive the different result. I already set the seed set.seed(1) but it does not work. How can I make the results the same for each run?
Thank you very much!
The text was updated successfully, but these errors were encountered: