biquad filter similar to SoX #275

engineerchuan · 2019-09-10T15:17:50Z

Use biquad filter similar to SoX.

engineerchuan · 2019-09-11T01:01:28Z

Hi Community,

We are currently exploring ways to make torchaudio's dependence on SoX optional (per #260).

A subset of SoX's functionality is its frequency filtering operations e.g. highpass, lowpass, bandpass, etc.. Sox's implementation are based on the digital biquad filter. (https://en.wikipedia.org/wiki/Digital_biquad_filter). This WIP PR ports SoX's implementation of the biquad effect, lowpass, and highpass filters.

Question: Which parts of the frequency filtering functions would be helpful to include in torchaudio vs keep separate?

One approach would be to provide only the core "execution layer" and leave the filter design outside the library.
For an IIR filter like biquad, this could mean implementing the biquad effect but asking the user to supply the coeffs (b0, b1, b2, a0, a1, a2). Or perhaps implementing a general IIR difference equation execution engine.
For a general FIR filter, the user could use filter design libraries like scipy.signal to design the impulse response. Then we can use torch's GPU accelerated convolution functions to execute.

@vincentqb

Thank you,

Chuan

vincentqb · 2019-09-11T13:47:30Z

Relates to #260

vincentqb

I took a quick look, and good job so far :)

A test is failing, but does not appear related to your changes.

test/test_datasets_vctk.py::TestVCTK::test_make_manifest FAILED

examples/filter_file.py

test/test_functional_filtering.py

torchaudio/functional.py

torchaudio/functional_filtering.py

…tional_filtering to functional_sox_convenience

torchaudio/functional.py

torchaudio/__init__.py

torchaudio/functional.py

vincentqb · 2019-09-16T17:32:16Z

torchaudio/filtering.cpp

+    assert(output_waveform.size(0) == n_channels);
+    assert(output_waveform.size(1) == n_frames);
+
+    auto input_accessor = input_waveform.accessor<float,2>();


We might need to look into using the appropriate C/GPU interface here. Unfortunately, we do not yet have the CI tools for anything else but linux on CPU.

I added GPU typedefs based on my understanding of code but we need to test this.

vincentqb · 2019-09-16T17:33:32Z

torchaudio/filtering.cpp

+    int n_order = a_coeffs.size(0); // n'th order - 1 filter
+    assert(a_coeffs.size(0) == b_coeffs.size(0));
+
+    for (int64_t i_channel = 0; i_channel < n_channels; ++i_channel) {


Can we do the channels in one pass for each frame?

I'm not sure there is necessarily a faster way. Would you use tensors to slice all channels at each frame? Would that be much faster?

Slicing would be great, yes. I'd expect this to work faster on GPU.

@yf225 -- We want to implement a transformation lfilter that could "almost" be implemented by convolutions. We can't because of the way the terms depend on each other. @engineerchuan suggests implementing it in C++ (see here, in torchaudio/filtering.cpp), since the for-loop in time is much faster and comparable to scipy in speed on CPU. We shouldn't need a for-loop over the channels though. Thoughts on things to be careful about here?

Since the computation for each channel doesn't depend on each other, we might be able to use at::parallel_for (https://github.com/pytorch/pytorch/blob/cc61af3c3d8ca8b46f7234383513b5166e10150c/aten/src/ATen/Parallel.h#L48) to speed up on CPU, and thrust::for_each (or a custom CUDA kernel launch) to speed up on GPU.

torchaudio/functional_sox_convenience.py

test/test_functional_filtering.py

… into assets

…quad

engineerchuan · 2019-09-17T15:46:14Z

3 different implementations of lfilter were explored in this commit.

lfilter "Element-wise" computation using tensor accessors (https://pytorch.org/cppdocs/notes/tensor_basics.html#efficient-access-to-tensor-elements)
lfilter_tensor "Slice" computation where the difference equation is evaluated simultaneously across all channels by adding and subtracting slices.
lfilter_tensor_matrix "Matrix" computation where we recast the difference equation computation as a matrix multiply.

All 3 implementations return same result at 1e-5 tolerance for a variety of random inputs. We have confidence they are doing the same math.

Performance (all on CPU):

Input (2 channel x 100K samples):

Looped Implementation took       :  0.0026006698608398438
Tensor Implementation took       :  7.084993362426758
Tensor Matrix Implementation took:  2.870526075363159

Input (10 channel x 10K samples):

Looped Implementation took       :  0.0015499591827392578
Tensor Implementation took       :  0.8290731906890869
Tensor Matrix Implementation took:  0.29657483100891113

Input (100 channel x 10K samples):

Looped Implementation took       :  0.01202535629272461
Tensor Implementation took       :  0.9394674301147461
Tensor Matrix Implementation took:  0.4104940891265869

The basic element wise implementation seems much faster. I need help evaluating my tensor based implementation. Is the method shown of slicing the most efficient way to access? Do I need to contiguous() at all?

Thank you,

Chuan

@yf225 @vincentqb

engineerchuan · 2019-09-17T21:35:33Z

Unwound the cpp lfilter implementations. We will move these to a separate PR.

vincentqb

Thanks for working on this! We're essentially ready to merge this :)

docs/source/functional_sox_compatibility.rst

test/test_datasets_vctk.py

test/test_functional_filtering.py

docs/source/index.rst

test/test_functional_filtering.py

torchaudio/functional_sox_compatibility.py

engineerchuan added 3 commits September 9, 2019 20:09

Add basic low pass filtering

3d7a6e3

Naive first implementation of biquad per SoX implementation

5202837

Add highpass filtering

b98522e

vincentqb reviewed Sep 11, 2019

View reviewed changes

vincentqb reviewed Sep 12, 2019

View reviewed changes

torchaudio/functional_filtering.py Outdated Show resolved Hide resolved

torchaudio/functional_filtering.py Outdated Show resolved Hide resolved

torchaudio/functional_filtering.py Outdated Show resolved Hide resolved

engineerchuan added 10 commits September 14, 2019 20:39

added cpp implementation of filtering

55a904c

More tests of IIR vs FIR

575339b

improved performance by utilizing floats

a791792

remove extraneous Python implementations

b8b9eff

Implement convolve function, add tests

73ae473

Move lfilter and convolve into functional, more tests

ca3afeb

Slight reformatting of functional and test

c19f32d

added additional documentation for convolve and lfilter, renamed func…

54e3834

…tional_filtering to functional_sox_convenience

Fix documentation per issue pytorch#98

c5534c2

Merge https://github.com/pytorch/audio

92e309b

vincentqb reviewed Sep 16, 2019

View reviewed changes

engineerchuan added 9 commits September 16, 2019 14:38

Delete sox_convenience, delete convolve wrapper

ae5de6a

Unwind other issue to separate PR, clean up

ebf3d9e

Follow naming convention for sample rate in functional

361a226

Merge https://github.com/pytorch/audio

5522ce5

use memset instead of implicit initialization

288aedc

fix failing vctk manifest test to account for adding more test audios…

4115a14

… into assets

Trying out a tensor slicing based approach

5469c4d

Adding documentation for lfilter, biquad, highpass_biquad, lowpass_bi…

59cb5ce

…quad

added matrix based implementation of lfilter

388a0c0

engineerchuan added 2 commits September 17, 2019 15:43

adding python lfilter implementation

98b9ec1

Merge https://github.com/pytorch/audio

d09f34f

Unwind cpp implementations for a separate PR

f4c84c8

engineerchuan added 3 commits September 17, 2019 19:37

factor out biquad, lowpass, highpass to sox compatibility

866de47

Adding documentation for functional sox compatibility

3336294

removed some print statements, mainly to kick build tests again

9608edd

vincentqb approved these changes Sep 18, 2019

View reviewed changes

engineerchuan added 4 commits September 18, 2019 08:35

moved previous sox compatibility into functional

445d717

Merge https://github.com/pytorch/audio

ba317d0

Fix module reference in biquad

c54991a

Small wording tweaks to test docstrings

a4cd66d

vincentqb merged commit 8273c3f into pytorch:master Sep 18, 2019

vincentqb changed the title ~~[WIP] Exploring Filtering Capabilities~~ biquad filter similar to SoX Sep 18, 2019

vincentqb mentioned this pull request Dec 20, 2019

Update audio preprocessing tutorial pytorch/tutorials#797

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

biquad filter similar to SoX #275

biquad filter similar to SoX #275

engineerchuan commented Sep 10, 2019

engineerchuan commented Sep 11, 2019 •

edited

vincentqb commented Sep 11, 2019

vincentqb left a comment

vincentqb Sep 16, 2019

engineerchuan Sep 16, 2019

vincentqb Sep 16, 2019

engineerchuan Sep 16, 2019

vincentqb Sep 16, 2019

vincentqb Sep 16, 2019

yf225 Sep 17, 2019

engineerchuan commented Sep 17, 2019 •

edited

engineerchuan commented Sep 17, 2019

vincentqb left a comment

biquad filter similar to SoX #275

biquad filter similar to SoX #275

Conversation

engineerchuan commented Sep 10, 2019

engineerchuan commented Sep 11, 2019 • edited

vincentqb commented Sep 11, 2019

vincentqb left a comment

Choose a reason for hiding this comment

vincentqb Sep 16, 2019

Choose a reason for hiding this comment

engineerchuan Sep 16, 2019

Choose a reason for hiding this comment

vincentqb Sep 16, 2019

Choose a reason for hiding this comment

engineerchuan Sep 16, 2019

Choose a reason for hiding this comment

vincentqb Sep 16, 2019

Choose a reason for hiding this comment

vincentqb Sep 16, 2019

Choose a reason for hiding this comment

yf225 Sep 17, 2019

Choose a reason for hiding this comment

engineerchuan commented Sep 17, 2019 • edited

engineerchuan commented Sep 17, 2019

vincentqb left a comment

Choose a reason for hiding this comment

engineerchuan commented Sep 11, 2019 •

edited

engineerchuan commented Sep 17, 2019 •

edited