Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strategies for initializing parameters should be configurable #215

Closed
antimora opened this issue Mar 9, 2023 · 0 comments
Closed

Strategies for initializing parameters should be configurable #215

antimora opened this issue Mar 9, 2023 · 0 comments

Comments

@antimora
Copy link
Collaborator

antimora commented Mar 9, 2023

Feature description

Ability to configure the initial parameters for Conv, Linear and Embedding NN ops.

Feature motivation

Currently when Conv2d constructed, the parameters are initialized with random values. For the inference mode, we do not need randomize the weights. In fact, it's wasteful (especially for large models).

Another motivation is that the NN components needs to be flexible and configurable.

(Optional) Suggest a Solution

There should be universal initializer (probably enum) that could be set in NN configurations. The default should be the same init strategy as it's now implemented.

@antimora antimora changed the title Strategies for initializing weights and biases should be configurable Strategies for initializing parameters should be configurable Mar 9, 2023
antimora added a commit to antimora/burn that referenced this issue Mar 9, 2023
@antimora antimora closed this as completed Mar 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant