conv_pointwise_type parameter #6476
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
for more information, see https://pre-commit.ci
Signed-off-by: Dima Rekesh <bmwshop@gmail.com>
Signed-off-by: Dima Rekesh <bmwshop@gmail.com>
for more information, see https://pre-commit.ci
titu1994
left a comment
titu1994
left a comment
There was a problem hiding this comment.
Looks good. But needs to be documented in Nemo ASR docs config section
should we add to this PR or do a separate PR for docs? |
|
Same PR would be good |
Signed-off-by: Dima Rekesh <bmwshop@gmail.com>
|
|
|
||
| The only condition that needs to be met is that **the final layer of the acoustic model must have the hidden dimension defined in ``model_defaults.enc_hidden``**. | ||
|
|
||
| Conformer Encoder |
There was a problem hiding this comment.
Is this for fast conformer ? Some flags are for fast conformer
| self.pointwise_conv2 = nn.Conv1d( | ||
| in_channels=dw_conv_input_dim, out_channels=d_model, kernel_size=1, stride=1, padding=0, bias=True | ||
| ) | ||
| elif self.conv_pointwise_type == 'linear': |
There was a problem hiding this comment.
I wonder. Conv1d with kernel 1 is identical to linear layer, so what if we do something like silently convert the conv kernel shape to linear compatible weights and don't use explicitly the Lineae layer but use functional.linear() instead ?
My assumption is we don't want to break checkpoint compatibility - and we don't need to. We can take a model trained on conv1d and use the weight in a F.linear() after weight conversion - this way no checkpoint incompatibility is there for old models but still there is speedup
Signed-off-by: Dima Rekesh <bmwshop@gmail.com>
for more information, see https://pre-commit.ci
|
@bmwshop |
What does this PR do ?
Adding the conv_pointwise_type parameter that to ConformerEncoder enabling the use of nn.Linear() for a 10% acceleration
Collection: ASR
Changelog
Adding the conv_pointwise_type parameter to
Usage
Before your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information