You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Ability to configure the initial parameters for Conv, Linear and Embedding NN ops.
Feature motivation
Currently when Conv2d constructed, the parameters are initialized with random values. For the inference mode, we do not need randomize the weights. In fact, it's wasteful (especially for large models).
Another motivation is that the NN components needs to be flexible and configurable.
(Optional) Suggest a Solution
There should be universal initializer (probably enum) that could be set in NN configurations. The default should be the same init strategy as it's now implemented.
The text was updated successfully, but these errors were encountered:
antimora
changed the title
Strategies for initializing weights and biases should be configurable
Strategies for initializing parameters should be configurable
Mar 9, 2023
antimora
added a commit
to antimora/burn
that referenced
this issue
Mar 9, 2023
Feature description
Ability to configure the initial parameters for Conv, Linear and Embedding NN ops.
Feature motivation
Currently when Conv2d constructed, the parameters are initialized with random values. For the inference mode, we do not need randomize the weights. In fact, it's wasteful (especially for large models).
Another motivation is that the NN components needs to be flexible and configurable.
(Optional) Suggest a Solution
There should be universal initializer (probably enum) that could be set in NN configurations. The default should be the same init strategy as it's now implemented.
The text was updated successfully, but these errors were encountered: