Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use module-wise random generator to initialize parameter weights #493

Merged
merged 3 commits into from Jul 23, 2019

Conversation

@TE-KazukiYoshiyama
Copy link
Contributor

commented Jul 23, 2019

Here is the old behaviour:

>>> import nnabla as nn
>>> import nnabla.parametric_functions as PF
>>> x = nn.Variable((3, 4))
>>> y1 = PF.affine(x, 2, name='affine1')
>>> y2 = PF.affine(x, 2, name='affine2')
>>> nn.get_parameters()['affine1/affine/W'].d
array([[-0.66892076,  0.10020873],
       [ 0.7217436 ,  0.23586744],
       [ 0.8924928 ,  0.12171594],
       [ 0.73917496, -0.65519905]], dtype=float32)
>>> nn.get_parameters()['affine2/affine/W'].d
array([[-0.66892076,  0.10020873],
       [ 0.7217436 ,  0.23586744],
       [ 0.8924928 ,  0.12171594],
       [ 0.73917496, -0.65519905]], dtype=float32)

since we used the same seed, now different parameters of the same shape has the different initialized values.

@TE-StefanUhlich

This comment has been minimized.

Copy link
Contributor

commented Jul 23, 2019

Thanks for your work.

@TE-StefanUhlich TE-StefanUhlich merged commit d95d431 into master Jul 23, 2019

@TE-StefanUhlich TE-StefanUhlich deleted the fix/20190711-parameter-initializer branch Jul 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.