Add PowerSpectrum CPU operator #1460

jantonguirao · 2019-11-12T16:45:18Z

Why we need this PR?

Need for a PowerSpectrum CPU operator

What happened in this PR?

Power spectrum CPU implemented in terms of Fft1DCpu kernel

JIRA TASK: [DALI-1148]

Signed-off-by: Joaquin Anton <janton@nvidia.com>

…about data layout (different layouts not yet tested) Signed-off-by: Joaquin Anton <janton@nvidia.com>

…e any dimension in the input data) Signed-off-by: Joaquin Anton <janton@nvidia.com>

Signed-off-by: Joaquin Anton <janton@nvidia.com>

…lex FFT spectrum Signed-off-by: Joaquin Anton <janton@nvidia.com>

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao · 2019-11-22T13:03:44Z

!build

dali-automaton · 2019-11-22T13:48:25Z

CI MESSAGE: [1003248]: BUILD STARTED

dali-automaton · 2019-11-22T14:59:42Z

CI MESSAGE: [1003248]: BUILD FAILED

dali/operators/signal/fft/power_spectrum.h

JanuszL · 2019-11-22T18:40:03Z

dali/operators/signal/fft/power_spectrum.cc

+  kernels::KernelContext ctx;
+  auto in_shape = input.shape();
+  int nsamples = input.size();
+  auto nthreads = ws.HasThreadPool() ? ws.GetThreadPool().size() : 1;


@klecki - is it possible?

Checked with him and it is safe to assume that we have a thread pool. So I'd change that

JanuszL · 2019-11-22T18:43:49Z

dali/test/python/test_operator_power_spectrum.py

+    shape = waveform.shape
+
+    out_shape = list(shape)
+    out_shape[axis] = nfft/2+1


It will differ between python2 and python3

Signed-off-by: Joaquin Anton <janton@nvidia.com>

szalpal

Generally, I'd rebuild the structure a little bit, so that it's more suitable for future GPU operator. My proposal:

template<typename Backend>
class PowerSpectrum : Operator<Backend> {
  bool CanInferOutputs() const override { return true; }
  kernels::KernelManager kmgr_;
  kernels::signal::fft::FftArgs fft_args_;
};

class PowerSpectrumCpu: PowerSpectrum<CPUBackend> {
  bool SetupImpl(std::vector<OutputDesc> &output_desc, const workspace_t<CPUBackend> &ws) override;
  void RunImpl(workspace_t<CPUBackend> &ws) override;
};

But the more important question is: could the nfft argument be per-sample?

dali/operators/signal/CMakeLists.txt

szalpal · 2019-11-26T14:11:15Z

dali/operators/signal/fft/power_spectrum.h

+template <typename Backend>
+class PowerSpectrum : public Operator<Backend> {
+ public:
+  explicit inline PowerSpectrum(const OpSpec &spec)


Suggested change

explicit inline PowerSpectrum(const OpSpec &spec)

explicit PowerSpectrum(const OpSpec &spec) // NOLINT

If method is defined in class body it's implicitly inline. I believe that declaration is redundant. Also linter complaints on explicit

will change

szalpal · 2019-11-26T14:17:43Z

dali/operators/signal/fft/power_spectrum.cc

+template <>
+bool PowerSpectrum<CPUBackend>::SetupImpl(std::vector<OutputDesc> &output_desc,
+                                          const workspace_t<CPUBackend> &ws) {
+  output_desc.resize(1);


Suggested change

output_desc.resize(1);

output_desc.resize(kNumOutputs);

That way it's clearer what the 1 actually is

szalpal · 2019-11-26T14:18:19Z

dali/operators/signal/fft/power_spectrum.cc

+DALI_SCHEMA(PowerSpectrum)
+    .DocStr(R"code(Power spectrum of signal.)code")
+    .NumInput(1)
+    .NumOutput(1)


Suggested change

.NumOutput(1)

.NumOutput(kNumOutputs)

I think this magic number can be described better

szalpal · 2019-11-26T14:20:11Z

dali/operators/signal/fft/power_spectrum.h

+  using Operator<Backend>::RunImpl;
+
+  kernels::KernelManager kmgr_;
+  kernels::signal::fft::FftArgs fft_args_;


Suggested change

kernels::signal::fft::FftArgs fft_args_;

const kernels::signal::fft::FftArgs fft_args_;

This would provide good sanity check, that the args are consistent through all the calls

as you can see in the constructor, I need to set the members individually with a switch. I'd have to create temporaries instead if I marked this const. Doable but it'd decrease readability, I think

szalpal · 2019-11-26T14:22:42Z

dali/operators/signal/fft/power_spectrum.cc

+#include "dali/kernels/signal/fft/fft_cpu.h"
+#include "dali/pipeline/data/views.h"
+
+#define FFT_SUPPORTED_NDIMS (1, 2, 3)


Why only these particular dims are supported? Is it kernel's restrictions?

Yes, it is restricted at the kernels side. Also, this is what would (and a little beyond) make sense for PowerSpectrum. You are calculating the power spectrum on either a 1D signal, or 2D with the second dimension being the number of channels. 3 dims is actually quite unlikely but one might have the windows already extracted (or provided externally) and use the PowerSpectrum operator to generate the Spectrogram

szalpal · 2019-11-26T14:23:59Z

dali/operators/signal/fft/power_spectrum.cc

+      output_desc[0].shape.set_tensor_shape(i, req.output_shapes[0][0].shape);
+    }
+  ), // NOLINT
+  (


This parenthesis is redundant I guess ;)

How so? Can I have this fail case without the parenthesis?

Work on my machine ;)

DALI/dali/operators/color/hsv.cc

Line 71 in fe65df0

), DALI_FAIL(make_string("Unsupported output type: ", output_type_))) // NOLINT

Signed-off-by: Joaquin Anton <janton@nvidia.com>

dali-automaton · 2019-11-26T16:09:56Z

CI MESSAGE: [1008058]: BUILD STARTED

dali-automaton · 2019-11-26T16:25:02Z

CI MESSAGE: [1008058]: BUILD FAILED

dali-automaton · 2019-11-27T10:04:13Z

CI MESSAGE: [1008058]: BUILD PASSED

jantonguirao changed the title ~~Fft op impl~~ [WIP] PowerSpectrum CPU Nov 12, 2019

jantonguirao force-pushed the fft_op_impl branch 5 times, most recently from 84a7d68 to 9505613 Compare November 15, 2019 14:06

jantonguirao added 24 commits November 20, 2019 16:12

[WIP] FFT CPU kernel impl

85b89a3

Signed-off-by: Joaquin Anton <janton@nvidia.com>

WIP

354c463

Signed-off-by: Joaquin Anton <janton@nvidia.com>

FFT CPU kernel impl (ongoing)

dfb9a5d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Complete FFT tests against naive DFT implementation. Lift constraint …

c0632dc

…about data layout (different layouts not yet tested) Signed-off-by: Joaquin Anton <janton@nvidia.com>

Support different data layouts for FFT (transformation axis can now b…

cecc7d4

…e any dimension in the input data) Signed-off-by: Joaquin Anton <janton@nvidia.com>

Move FFT1D to kernels::signal

5260ce3

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Rework Get1DSlices

a04c990

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Add pimpl based implementation for FFT 1D CPU

4074f0e

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Code review fixes

b18eeda

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Using std::complex as OutputType in FFT kernel (when calculating comp…

aa6ef1c

…lex FFT spectrum Signed-off-by: Joaquin Anton <janton@nvidia.com>

Code review fixes

f24ed28

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Removing log power spectrum case

fe50a5f

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix typo

6d59748

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix typo

b40ace7

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Code review fixes

4bbbf82

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Introduce ForAxis

f09f8db

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Moving utils

2bb2839

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Review fixes

7ff9bd5

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Fix lint

b33ebf7

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Avoid leaking ffts symbols

718dd94

Signed-off-by: Joaquin Anton <janton@nvidia.com>

[WIP] FFT operator

6cef17a

Signed-off-by: Joaquin Anton <janton@nvidia.com>

WIP FFT CPU operator implementation

c805211

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Adjust to kernels::signal namespace

9c7852b

Signed-off-by: Joaquin Anton <janton@nvidia.com>

WIP

6e418e7

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao added 2 commits November 20, 2019 16:20

Add PowerSpectrum (with tests)

0ab471b

Signed-off-by: Joaquin Anton <janton@nvidia.com>

lint

bbfa604

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the fft_op_impl branch from c9be203 to bbfa604 Compare November 20, 2019 15:20

Fix previous rebase

45d5bb6

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the fft_op_impl branch from 75017c7 to 45d5bb6 Compare November 20, 2019 15:25

jantonguirao added 3 commits November 20, 2019 16:29

Move audio/fft to signal/fft

d18d3f5

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Add power as integer, add support for 1D input

023f414

Signed-off-by: Joaquin Anton <janton@nvidia.com>

Minor fixes

eea5e0d

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao changed the title ~~[WIP] PowerSpectrum CPU~~ Add PowerSpectrum CPU operator Nov 20, 2019

jantonguirao requested review from klecki, JanuszL and mzient November 20, 2019 15:56

szalpal reviewed Nov 22, 2019

View reviewed changes

dali/operators/signal/fft/power_spectrum.h Show resolved Hide resolved

JanuszL reviewed Nov 22, 2019

View reviewed changes

Code review fixes

2348acf

Signed-off-by: Joaquin Anton <janton@nvidia.com>

jantonguirao force-pushed the fft_op_impl branch from 508d481 to 2348acf Compare November 25, 2019 10:59

JanuszL approved these changes Nov 26, 2019

View reviewed changes

Merge remote-tracking branch 'upstream/master' into fft_op_impl

1994b38

szalpal reviewed Nov 26, 2019

View reviewed changes

Apply fixes from code review

a21a95a

Signed-off-by: Joaquin Anton <janton@nvidia.com>

awolant approved these changes Nov 26, 2019

View reviewed changes

jantonguirao merged commit 319b113 into NVIDIA:master Nov 27, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add PowerSpectrum CPU operator #1460

Add PowerSpectrum CPU operator #1460

jantonguirao commented Nov 12, 2019 •

edited

Loading

jantonguirao commented Nov 22, 2019

dali-automaton commented Nov 22, 2019

dali-automaton commented Nov 22, 2019

JanuszL Nov 22, 2019

jantonguirao Nov 25, 2019

JanuszL Nov 22, 2019 •

edited

Loading

jantonguirao Nov 25, 2019 •

edited

Loading

szalpal left a comment •

edited

Loading

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019 •

edited

Loading

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019

szalpal Nov 26, 2019

jantonguirao Nov 26, 2019

dali-automaton commented Nov 26, 2019

dali-automaton commented Nov 26, 2019

dali-automaton commented Nov 27, 2019

	explicit inline PowerSpectrum(const OpSpec &spec)
	explicit PowerSpectrum(const OpSpec &spec) // NOLINT

	kernels::signal::fft::FftArgs fft_args_;
	const kernels::signal::fft::FftArgs fft_args_;

Add PowerSpectrum CPU operator #1460

Add PowerSpectrum CPU operator #1460

Conversation

jantonguirao commented Nov 12, 2019 • edited Loading

Why we need this PR?

What happened in this PR?

jantonguirao commented Nov 22, 2019

dali-automaton commented Nov 22, 2019

dali-automaton commented Nov 22, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JanuszL Nov 22, 2019 • edited Loading

Choose a reason for hiding this comment

jantonguirao Nov 25, 2019 • edited Loading

Choose a reason for hiding this comment

szalpal left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jantonguirao Nov 26, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dali-automaton commented Nov 26, 2019

dali-automaton commented Nov 26, 2019

dali-automaton commented Nov 27, 2019

jantonguirao commented Nov 12, 2019 •

edited

Loading

JanuszL Nov 22, 2019 •

edited

Loading

jantonguirao Nov 25, 2019 •

edited

Loading

szalpal left a comment •

edited

Loading

jantonguirao Nov 26, 2019 •

edited

Loading