New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add slaney normalization #589
Conversation
torchaudio/functional.py
Outdated
@@ -430,6 +431,8 @@ def create_fb_matrix( | |||
f_max (float): Maximum frequency (Hz) | |||
n_mels (int): Number of mel filterbanks | |||
sample_rate (int): Sample rate of the audio waveform | |||
norm (Optional[str]): If 'slaney', divide the triangular mel weights by the width of the mel band |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we expected further normalization schemes? Does it make sense to split these normalizations out into its own layer? Are they useful in other contexts (maybe for volume normalization and such)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we expected further normalization schemes?
Yes, we could, see librosa/librosa#1050. Librosa itself is in process of adding other normalization as mentioned in this pull request.
Does it make sense to split these normalizations out into its own layer?
Not according to this comment.
Are they useful in other contexts (maybe for volume normalization and such)?
The normalization is done against f_pts
which is computed within create_fb_matrix
. I'm not aware of other use case.
88913b2
to
9a1994f
Compare
924c37a
to
4150097
Compare
test/test_torchscript_consistency.py
Outdated
@@ -99,7 +99,8 @@ def func(_): | |||
f_max = 20.0 | |||
n_mels = 10 | |||
sample_rate = 16000 | |||
return F.create_fb_matrix(n_stft, f_min, f_max, n_mels, sample_rate) | |||
norm = None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you're exercising torchscript I'd pass the less trivial type which is a string instead of None.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You mean: default to empty string? We don't use that elsewhere, but sounds good to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
Codecov Report
@@ Coverage Diff @@
## master #589 +/- ##
==========================================
+ Coverage 88.99% 89.01% +0.01%
==========================================
Files 21 21
Lines 2254 2257 +3
==========================================
+ Hits 2006 2009 +3
Misses 248 248
Continue to review full report at Codecov.
|
@mthrok -- do you have any feedback? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the late reply. Looks good to me. One nit.
@@ -424,7 +424,8 @@ def create_fb_matrix( | |||
f_min: float, | |||
f_max: float, | |||
n_mels: int, | |||
sample_rate: int | |||
sample_rate: int, | |||
norm: str = "", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a public API signature, I think Optional[str]
looks cleaner.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This was made due to comment. I'll leave it as it is for now. We can always extend the str
to Optional[str]
without BC breaking later :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks to me that, that comment was meant for the type of variable to pass when running Torchscript test, not about the function signature.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we allow None
in the signature, then the code should work with/without jit when passing None
. It wasn't though. Is that what you meant?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we allow None in the signature, then the code should work with/without jit when passing None.
Yes, it works.
from typing import Optional
import torch
from torch import Tensor
def bar(foo: Optional[str]=None) -> Tensor:
if foo is None:
return torch.zeros(1, 2)
if foo == "a":
return torch.ones(1, 1)
return torch.empty(1, 1)
ts_bar = torch.jit.script(bar)
for v in [None, "a", "b"]:
print(v)
print(bar(v))
print(ts_bar(v))
produces
None
tensor([[0., 0.]])
tensor([[0., 0.]])
a
tensor([[1.]])
tensor([[1.]])
b
tensor([[-2.8910e+12]])
tensor([[0.]])
also dcshift
uses Optional[float]
and it works fine. for both None and float input for torchscript.
#558
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's just when type is optional, it firstly needs to compare against None
using if var is None
or if var is not None
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://pytorch.org/docs/master/jit_language_reference.html#optional-type-refinement
TorchScript will refine the type of a variable of type Optional[T] when a comparison to None is made inside the conditional of an if-statement or checked in an assert. The compiler can reason about multiple None checks that are combined with and, or, and not. Refinement will also occur for else blocks of if-statements that are not explicitly written.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alrighty, #641 :)
* add slaney normalization. * add torchscript. * convert to string for torchscript compatibility. * flake8. * use string as default.
Update seq2seq_translation_tutorial.py
Added instructive comments
Fixes #287