Error when using Resampler on GPU with torchaudio 0.9

[mravanelli]

Hi,
it looks like there is a small issue when using the resampler with torchaudio 0.9.
I indeed got the error below, when running the following code:

import torch
import torchaudio
resampler = torchaudio.transforms.Resample(16000, 8000)
x = torch.rand(4, 32000, device='cuda:0')
resampler(x)

  File "<stdin>", line 1, in <module>
  File "/home/mila/r/ravanelm/anaconda3/envs/py19/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/mila/r/ravanelm/anaconda3/envs/py19/lib/python3.9/site-packages/torchaudio/transforms.py", line 716, in forward
    return _apply_sinc_resample_kernel(
  File "/home/mila/r/ravanelm/anaconda3/envs/py19/lib/python3.9/site-packages/torchaudio/functional/functional.py", line 1387, in _apply_sinc_resample_kernel
    resampled = torch.nn.functional.conv1d(waveform[:, None], kernel, stride=orig_freq)
RuntimeError: Input type (torch.cuda.FloatTensor) and weight type (torch.FloatTensor) should be the sam

Note that everything is fine with cpu and with torchaudio 0.8.1

[yoyololicon]

@mravanelli
You didn't move your resampler to the same device.
The Resample module in 0.9 now cache kernel weights within it. Use resampler = torchaudio.transforms.Resample(16000, 8000).to('cuda:0') should fix the problem.

[mravanelli]

Hi @yoyololicon,
thank you for your reply.
The issue is that with torchaudio 0.8 the error doesn't show up and this might cause incompatibilities with pre-existing codes written with torchaudio 0.8. Isn't it reasonable to put the kernel in the same device as the input signal (as probably done before)?

[yoyololicon]

Hi @mravanelli
Yes, it was meant to be a BC-breaking change according to #1514, I'll just paste some of the statements here.

To maintain the original behavior would require moving the cached kernel to the device and dtype of the input waveform every time transforms.Resample is called, and this could result in unintended side effects; running a CPU transforms on CUDA inputs would have overhead from constantly moving a cached CPU kernel to CUDA, and the expectation is that the user will manually move the transforms themself to the correct device and dtype (ex/ resample = transforms.Resample; resample = resample.to(device=torch.device('cuda'), dtype=torch.float16). This PR removes the moving of the cached kernel to be of the correct device and dtype at every call to it, which will now throw an error if a user does not move the transform to CUDA but calls the function on a CUDA waveform.

[mravanelli]

ok, thank you for the clarification. I will change my code to make it work with the latest version. Best, Mirco

…

On Wed, 30 Jun 2021 at 09:58, Chin-Yun Yu ***@***.***> wrote: Hi @mravanelli <https://github.com/mravanelli> Yes, it was meant to be a BC-breaking change according to #1514 <#1514>, I'll just copy some of the statements here. To maintain the original behavior would require moving the cached kernel to the device and dtype of the input waveform every time transforms.Resample is called, and this could result in unintended side effects; running a CPU transforms on CUDA inputs would have overhead from constantly moving a cached CPU kernel to CUDA, and the expectation is that the user will manually move the transforms themself to the correct device and dtype (ex/ resample = transforms.Resample; resample = resample.to(device=torch.device('cuda'), dtype=torch.float16). This PR removes the moving of the cached kernel to be of the correct device and dtype at every call to it, which will now throw an error if a user does not move the transform to CUDA but calls the function on a CUDA waveform. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1619 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEA2ZVRLBCR5TQHCR25GL6DTVMPJHANCNFSM47RDP6SA> .

[mthrok]

We have added kernel cache mechanism to transforms.Resample in 0.9.
The kernel is pre-computed when the transform is instantiated, and it has to be moved to the proper device. This semantic follows the regular usage of torch.nn.Module.

Looks like this was missing from 0.9 release note, I will fix the release note.

cc @carolineechen