length mismatch for istft

[HuangZiliAndy]

🐛 Bug

Hi I am having some trouble with torch.istft. It seems that the length doesn't match even when I specify the length (this issue seems to happen with very small probability).

To Reproduce

Steps to reproduce the behavior:

1. download stft.pt from link
2. run the following code

import torch

fname = 'stft.pt'
stft_feat = torch.load(fname)
print("stft_feat", stft_feat.size())

n_fft = 512
hop_length = 320
win_length = 512
window = torch.hann_window(win_length).cuda()
center = True
length = 150079
reconstruct = torch.istft(stft_feat, n_fft, hop_length=hop_length, win_length=win_length, window=window, center=center, length=length)[0]
print("reconstruct", reconstruct.size())

Expected behavior

In my environment, the program printed

stft_feat torch.Size([1, 257, 469])
reconstruct torch.Size([150016])

The length seems to be incorrect. The stft_feat is the output from neural network, I cannot guarantee the value to be very correct.

Environment

• What commands did you used to install torchaudio (conda/pip/build from source)?
  pip in conda environment
• If you are building from source, which commit is it?
• What does torchaudio.__version__ print? (If applicable)
  0.8.0a0+a751e1d

Collecting environment information...
PyTorch version: 1.8.0
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Debian GNU/Linux 9.13 (stretch) (x86_64)
GCC version: (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
Clang version: 3.8.1-24 (tags/RELEASE_381/final)
CMake version: version 3.20.0-rc2

Python version: 3.6 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: 10.2.89
GPU models and configuration:
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti
GPU 2: GeForce RTX 2080 Ti

Nvidia driver version: 440.33.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.18.4
[pip3] pytorch-lightning==1.1.0
[pip3] pytorch-ranger==0.1.1
[pip3] torch==1.8.0
[pip3] torch-optimizer==0.1.0
[pip3] torch-stoi==0.1.2
[pip3] torchaudio==0.8.0a0+a751e1d
[pip3] torchvision==0.9.0
[conda] blas 1.0 mkl
[conda] cuda90 1.0 h6433d27_0 pytorch
[conda] mkl 2019.4 243
[conda] mkl-service 2.3.0 py37he904b0f_0
[conda] mkl_fft 1.0.14 py37ha843d7b_0
[conda] mkl_random 1.1.0 py37hd6b4f25_0
[conda] numpy 1.17.2 py37haad9e8e_0
[conda] numpy-base 1.17.2 py37hde5b4d6_0
[conda] numpydoc 0.9.1 py_0
[conda] pytorch 1.0.0 py3.7_cuda9.0.176_cudnn7.4.1_1 pytorch
[conda] torchvision 0.2.1 py_2 pytorch

Additional context

[vincentqb]

Since I don't have visibility over stft_feat (EDIT: around how that file was generated), I cannot comment on that part. However, have you tried with another library? In particular torch.istft is meant to match with librosa's. Have you tried with the latter?

[HuangZiliAndy]

Since I don't have visibility over stft_feat, I cannot comment on that part. However, have you tried with another library? In particular torch.istft is meant to match with librosa's. Have you tried with the latter?

Hi I think I already attach a link for the stft_feat (https://drive.google.com/file/d/1TjwwGFEotsjAay0osCqw1IlcuANX675R/view?usp=sharing) Would you mind downloading it and have a try? I will also try the librosa and give you feedback. Thanks!

[vincentqb]

Hi I think I already attach a link for the stft_feat (https://drive.google.com/file/d/1TjwwGFEotsjAay0osCqw1IlcuANX675R/view?usp=sharing)

Sorry, what I meant is that I do not know how stft_feat was generated :) do you have more details about this?

[HuangZiliAndy]

reconstruct_librosa = librosa.istft(stft_feat.data.cpu().numpy().squeeze(), hop_length=hop_length, win_length=win_length, window='hann', center=center, length=length)
print("reconstruct_librosa", reconstruct_librosa.shape)

# reconstruct torch.Size([150016])
# reconstruct_librosa (150079,)

Sorry for my late reply, I tried librosa and the output length is correct. My stft_feat is not extracted, it is predicted. My situation is that the neural model will predict a real positive value STFT mask M and I will multiply that with the STFT feature (imaginary) of a real audio R. So the stft_feat F = M * R may not be correct. But I think this question is not related to the stft_feat is valid or not right? It is simply about the output length I guess?

[HuangZiliAndy]

Any ideas on this? @vincentqb Thanks a lot!

[nateanl]

I have reproduced your result. The reason is that the hop size is 320 that is a little big when n_fft is 512.
Let's see the implementation of torch.istft:

  const auto start = center ? n_fft / 2 : 0;
  const auto end = lengthOpt.has_value()? start + lengthOpt.value() : - n_fft / 2;

The start is set to 256 since center is True in your case.
The end is set to start + lengthOpt.value() that is 256+150079 = 150335.
After slicing, the shape should be 150079, looks perfect.

However, the expected_signal_length is n_fft + hop_length * (n_frames - 1) that is 512+320*468 = 150272. It's smaller than 150335
After slicing operation [256:150272], the shape of your waveform becomes 150016.

I would suggest using hop_length as 256 that is common in speech processing.

In librosa, they added a padding logic if the expected length is shorter than the given length. @mthrok What's your opinion on this?

[nateanl]

Hi @HuangZiliAndy, I have added the padding logic to torch.stft in pytorch/pytorch#63469. Now the issue should be resolved in the nightly version soon. However, to make sure the signal is invertible, I suggest using a different center and hop_length setting to avoid padding. Thanks.