New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sn.set_random_seed doesn't make initialized weights and training results reproducible #53
Comments
I was able to fix this by adding a parameter init_seed to the sciann.Functional and passing this init_seed to the "seed" parameter of prepare_default_activations_and_initializers (utils/utilities.py) method called inside the sciann.Functional. And now if we use this init_seed then the weights initialized become reproducible (and also the training results). import sciann as sn
from sciann.utils.math import diff
seed = 43634
sn.set_random_seed(seed)
for i in range(4):
x = sn.Variable('x',dtype='float64')
y = sn.Variable('y',dtype='float64')
inputs = [x,y]
p = sn.Functional('p', inputs,2*[10], activation='tanh',init_seed=seed) ## passing the init_seed
## L1-loss
L1 = diff(p, x, order=2) + diff(p, y, order=2)
losses = [L1]
m = sn.SciModel(inputs, losses)
weights = p.get_weights()
## just talking a slice to compare
compare_weights = weights[0][0][0]
print(f'############## Weights Iter: {i}')
print(compare_weights) Output:
@ehsanhaghighat Let me know your though on this, I can raise a pull request with the required code changes. |
you need to move |
With reference to your advice, I tried the implementations below but in both the implementations the initialized weights were not reproducible. 1st Variation - sn.set_random_seed in the begining of the for loopseed = 43634
for i in range(4):
sn.set_random_seed(seed)
x = sn.Variable('x',dtype='float64')
y = sn.Variable('y',dtype='float64')
inputs = [x,y]
p = sn.Functional('p', inputs,2*[10], activation='tanh')
## L1-loss
L1 = diff(p, x, order=2) + diff(p, y, order=2)
losses = [L1]
m = sn.SciModel(inputs, losses)
weights = p.get_weights()
## just talking a slice to compare
compare_weights = weights[0][0][0]
print(f'############## Weights Iter: {i}')
print(compare_weights) Output:
2nd Variation - sn.set_random_seed in the end of the for loopseed = 43634
for i in range(4):
x = sn.Variable('x',dtype='float64')
y = sn.Variable('y',dtype='float64')
inputs = [x,y]
p = sn.Functional('p', inputs,2*[10], activation='tanh')
## L1-loss
L1 = diff(p, x, order=2) + diff(p, y, order=2)
losses = [L1]
m = sn.SciModel(inputs, losses)
weights = p.get_weights()
## just talking a slice to compare
compare_weights = weights[0][0][0]
print(f'############## Weights Iter: {i}')
print(compare_weights)
sn.set_random_seed(seed) Output:
@ehsanhaghighat Am I missing something here ? |
The code below doesn't reproduce the same weights when model is initialized multiple times, even after using sn.set_random_seed function.
Output:
The text was updated successfully, but these errors were encountered: