Optimise dedispersion routine

[mickveldhuis]

This MR relates to #58, with the goal of bringing the execution time of the dedispersion kernel down to 1.5-2 seconds. Here we specifically use the testfdd binary as a benchmark.

Some improvements in no particular order:

• Using std::cosf() and std::sinf() instead of std::cos() and std::sin, which seems to speed-up the kernel by around 0.5 seconds.

Other additions:

• A CMake flag to enable/disable OpenMP for debugging / benchmarking purposes.
• A CMake flag to enable/disable granular benchmarks in FDDPlan::execute()

[mickveldhuis]

After a discussion with @csbnw we noticed the following:

• The dedispersion kernel itself is likely not an order of magnitude slower; though still worth investigating.
• The PACE code was compiled using the DEBUG flag, which led to xtensor-fftw being multiple orders of magnitude slower; specifically the (refer to https://git.astron.nl/RD/fft-benchmark/ for specific benchmarks)

Note that the original code has more granular benchmarking in the execute() routine:

Initialization time : 0.142000 sec.
Preprocessing time  : 0.255000 sec.
Dedispersion time   : 2.881000 sec.
Postprocessing time : 0.009000 sec.
Output memcpy time  : 0.002000 sec.
Total time          : 3.291000 sec.

We should add the same here to enable a more direct comparison, but in release mode FDDPlan::execute() takes 4.24283 seconds, compared to the 3.291 seconds of the original CPU implementation.

[mickveldhuis]

After a discussion with @csbnw we noticed the following:

* The dedispersion kernel itself is likely not an order of magnitude slower; though still worth investigating.

* The PACE code was compiled using the DEBUG flag, which led to xtensor-fftw being multiple orders of magnitude slower; specifically the (refer to https://git.astron.nl/RD/fft-benchmark/ for specific benchmarks)

Note that the original code has more granular benchmarking in the execute() routine:

Initialization time : 0.142000 sec.
Preprocessing time  : 0.255000 sec.
Dedispersion time   : 2.881000 sec.
Postprocessing time : 0.009000 sec.
Output memcpy time  : 0.002000 sec.
Total time          : 3.291000 sec.

We should add the same here to enable a more direct comparison, but in release mode FDDPlan::execute() takes 4.24283 seconds, compared to the 3.291 seconds of the original CPU implementation.

Added the timings to FDDPlan::execute(), here's a comparison (on DAS-6 node508) between the the PACE implementation (top) and original one (bottom):

PACE/FDDPlan::execute()
-----------------------------------
Initialization time : 0.211336 sec.
Preprocessing time  : 0.570274 sec.
Dedispersion time   : 3.349770 sec.
Postprocessing time : 0.017137 sec.
Output copy time    : 0.094885 sec.
-----------------------------------
Total time          : 4.246690 sec.

RD/DEDISP/FDDPlan::execute()
---------------------------------- 
Initialization time : 0.141000 sec.
Preprocessing time  : 0.252000 sec.
Dedispersion time   : 2.770000 sec.
Postprocessing time : 0.009000 sec.
Output memcpy time  : 0.001000 sec.
-----------------------------------
Total time          : 3.176000 sec.

Conclusions:

• The FFT routines with xtensor are roughly a factor 2 slower than the original FFTW implementation, though it's not a significant chunk of the runtime.
• Initialization of all xarrays is 2x more expensive than the original impl., however we also init about twice as many arrays to use as scratch space.
• The dedispersion kernel itself is about 20% slower.. not significantly slower, but it's worth investigating.

[mickveldhuis]

After a discussion with @csbnw we noticed the following:

* The dedispersion kernel itself is likely not an order of magnitude slower; though still worth investigating.

* The PACE code was compiled using the DEBUG flag, which led to xtensor-fftw being multiple orders of magnitude slower; specifically the (refer to https://git.astron.nl/RD/fft-benchmark/ for specific benchmarks)

Note that the original code has more granular benchmarking in the execute() routine:

Initialization time : 0.142000 sec.
Preprocessing time  : 0.255000 sec.
Dedispersion time   : 2.881000 sec.
Postprocessing time : 0.009000 sec.
Output memcpy time  : 0.002000 sec.
Total time          : 3.291000 sec.

We should add the same here to enable a more direct comparison, but in release mode FDDPlan::execute() takes 4.24283 seconds, compared to the 3.291 seconds of the original CPU implementation.

Added the timings to FDDPlan::execute(), here's a comparison (on DAS-6 node508) between the the PACE implementation (top) and original one (bottom):

PACE/FDDPlan::execute()
-----------------------------------
Initialization time : 0.211336 sec.
Preprocessing time  : 0.570274 sec.
Dedispersion time   : 3.349770 sec.
Postprocessing time : 0.017137 sec.
Output copy time    : 0.094885 sec.
-----------------------------------
Total time          : 4.246690 sec.

RD/DEDISP/FDDPlan::execute()
---------------------------------- 
Initialization time : 0.141000 sec.
Preprocessing time  : 0.252000 sec.
Dedispersion time   : 2.770000 sec.
Postprocessing time : 0.009000 sec.
Output memcpy time  : 0.001000 sec.
-----------------------------------
Total time          : 3.176000 sec.

Conclusions:

* The FFT routines with xtensor are roughly a factor 2 slower than the original FFTW implementation, though it's not a significant chunk of the runtime.

* Initialization of all xarrays is 2x more expensive than the original impl., however we also init about twice as many arrays to use as scratch space.

* The dedispersion kernel itself is about 20% slower.. not significantly slower, but it's worth investigating.

Whoops.. I was comparing the PACE variant that uses the reference kernel with the optimized FDD kernel of the original implementation. Using both USE_CPU=1 and USE_REFERENCE=1, I obtain the following result:

Initialization time : 0.141000 sec.
Preprocessing time  : 0.250000 sec.
Dedispersion time   : 3.718000 sec.
Postprocessing time : 0.010000 sec.
Output memcpy time  : 0.001000 sec.
--------------------------------------
Total time          : 4.122000 sec.

This matches the PACE implementation! The PACE impl. is 3% slower, but I think that's to be expected. Surprisingly our kernel is slightly faster, but the initialization, FFTs, and copying data around take more time in our impl.

[csbnw]

I have one minor comment, other than that it looks good to go. 🙌