You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I think that initializing the weights of the linear layer to 0 will produce an output of 0. Probably, it should be glorot_orthogonal(self.lin.weight, scale=2.0)
The text was updated successfully, but these errors were encountered:
Not quite. While it's true they are initialized to 0 and the very first forward pass outputs 0, a single backwards call will update these parameters to nonzero. It's always possible to explore better initialization schemes, but as is, this isn't an issue. The original DimeNet++ implementation offers the flexibility for both schemes, both working better for different targets
In the following line in
reset_parameters
method of OutputPPBlock used in DimeNet++,https://github.com/Open-Catalyst-Project/ocp/blob/b5a197fc3c79a9a5a787aabaa02979be53d296b7/ocpmodels/models/dimenet_plus_plus.py#L192
I think that initializing the weights of the linear layer to
0
will produce an output of0
. Probably, it should beglorot_orthogonal(self.lin.weight, scale=2.0)
The text was updated successfully, but these errors were encountered: