PyFFTW3 vs pyFFTW

[mperrin]

Issue by mperrin
Wednesday Apr 24, 2013 at 21:13 GMT
Originally opened as mperrin/webbpsf#10

---

Argh, there are two packaged for FFTW access in Python. Should evaluate & compare.

See: astropy/astropy#660
for some discussion in astropy.

TBD how best to work with planning to support stored wisdom for large 2D FFTs to minimize replanning.

[mperrin]

Comment by mperrin
Wednesday Jun 12, 2013 at 05:53 GMT

---

Test code benchmarks the two versus each other. Benchmark code available at https://gist.github.com/mperrin/5763120

(The following taken verbatim from an email to collaborators)

There are now two different/independent python packages that wrap the FFTW library:
https://pypi.python.org/pypi/PyFFTW3
https://pypi.python.org/pypi/pyFFTW

Note the similar but non-identical names. "pyFFTW" is more recent and appears to be more actively developed now, while PyFFTW3 was last updated in 2010 and does not support Python 3. See the discussion at astropy/astropy#660 . This is kind of annoying on a practical level due to increased hassle + cognitive overload from competing libraries, but oh well.

I've never been able to get PyFFTW3's wisdom saving to work properly so I was motivated to try out the newer package instead. I've now benchmarked the two and it turns out that the speeds are effectively identical. This is good. But it took a bit of fussing to get the code working properly (different calling conventions, etc). So, for posterity, here's my speed test results on my Mac Pro, plus my test script.

I'll probably switch over to pyFFTW for the next version of WebbPSF.

---

Array size: 1024 x 1024
Direction: forward
dtype: <type 'numpy.complex128'>
ncores: 16

Now planning numpy in the forward direction
no planning required
Time elapsed planning: 0.0000 s
Now planning pyfftw3 in the forward direction
Time elapsed planning: 1.6392 s
Now planning pyfftw in the forward direction
Time elapsed planning: 1.2817 s

FFT type: numpy Time Elapsed: 0.0931 s
FFT type: pyfftw3 Time Elapsed: 0.0205 s
FFT type: pyfftw Time Elapsed: 0.0145 s

---

Array size: 2048 x 2048
Direction: forward
dtype: <type 'numpy.complex128'>
ncores: 16

Now planning numpy in the forward direction
no planning required
Time elapsed planning: 0.0000 s
Now planning pyfftw3 in the forward direction
Time elapsed planning: 5.7729 s
Now planning pyfftw in the forward direction
Time elapsed planning: 4.6653 s

FFT type: numpy Time Elapsed: 0.5265 s
FFT type: pyfftw3 Time Elapsed: 0.0571 s
FFT type: pyfftw Time Elapsed: 0.0621 s

---

Array size: 4096 x 4096
Direction: forward
dtype: <type 'numpy.complex128'>
ncores: 16

Now planning numpy in the forward direction
no planning required
Time elapsed planning: 0.0000 s
Now planning pyfftw3 in the forward direction
Time elapsed planning: 18.9021 s
Now planning pyfftw in the forward direction
Time elapsed planning: 17.1350 s

FFT type: numpy Time Elapsed: 2.1550 s
FFT type: pyfftw3 Time Elapsed: 0.1492 s
FFT type: pyfftw Time Elapsed: 0.1573 s

[mperrin]

Comment by keflavich
Thursday Jun 13, 2013 at 20:43 GMT

---

Finally got pyfftw installed; looks like I get comparable numbers all around:

Array size: 1024 x 1024
Direction: forward
dtype: <type 'numpy.complex128'>
ncores: 16

Now planning numpy in the forward direction
    no planning required
    Time elapsed planning: 0.0000 s
Now planning pyfftw3 in the forward direction
    Time elapsed planning: 1.6490 s
Now planning pyfftw in the forward direction
    Time elapsed planning: 1.2462 s

FFT type: numpy         Time Elapsed: 0.0944 s
FFT type: pyfftw3       Time Elapsed: 0.0296 s
FFT type: pyfftw        Time Elapsed: 0.0217 s
--------------------------------------------------------
Array size: 2048 x 2048
Direction: forward
dtype: <type 'numpy.complex128'>
ncores: 16

Now planning numpy in the forward direction
    no planning required
    Time elapsed planning: 0.0000 s
Now planning pyfftw3 in the forward direction
    Time elapsed planning: 6.2908 s
Now planning pyfftw in the forward direction
    Time elapsed planning: 4.6149 s

FFT type: numpy         Time Elapsed: 0.4798 s
FFT type: pyfftw3       Time Elapsed: 0.0732 s
FFT type: pyfftw        Time Elapsed: 0.1685 s
--------------------------------------------------------
Array size: 4096 x 4096
Direction: forward
dtype: <type 'numpy.complex128'>
ncores: 16

Now planning numpy in the forward direction
    no planning required
    Time elapsed planning: 0.0000 s
Now planning pyfftw3 in the forward direction
    Time elapsed planning: 18.8597 s
Now planning pyfftw in the forward direction
    Time elapsed planning: 16.8630 s

FFT type: numpy         Time Elapsed: 2.2864 s
FFT type: pyfftw3       Time Elapsed: 0.1584 s
FFT type: pyfftw        Time Elapsed: 0.3184 s
--------------------------------------------------------

(mine's a Mac Pro with 2xQuad-Core Intel Xeon 2.93 GHz with hyperthreading, so ncores=16 is OK)

[mperrin]

Comment by keflavich
Monday Jun 17, 2013 at 16:12 GMT

---

Added some more results with an added test: https://gist.github.com/keflavich/5797994 (a fork of your gist, see specifically https://gist.github.com/keflavich/5797994#file-results-txt). Without planning first, pyfftw still outperforms numpy, but barely. Also, I got some wildly inconsistent results so I suspect my machine must have found something else to do during some of the tests...

[mperrin]

Comment by mperrin
Thursday Sep 04, 2014 at 03:38 GMT

---

Poppy and webbpsf now both updated to use pyfftw.