Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add custom easyblock for FFTW (REVIEW) #1083

Merged
merged 10 commits into from Jan 10, 2017
Merged

Conversation

boegel
Copy link
Member

@boegel boegel commented Jan 6, 2017

This easyblock takes care of building FFTW with use of AVX* instructions enabled when suitable, i.e. when the host on which it is being built supports these instructions.

(note: requires easybuilders/easybuild-framework#2074)

Using this easyblock for building FFTW results in the same multi-cycle build as is currently done in the FFTW easyconfigs (by specifying configopts as a list):

$ eb FFTW.eb -x | grep configure_step -A 1
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-single --enable-mpi --with-pic --enable-openmp --enable-threads --enable-avx --enable-avx2 --enable-sse2"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-mpi --with-pic --enable-openmp --enable-threads --enable-avx --enable-avx2 --enable-sse2"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-long-double --enable-mpi --with-pic --enable-openmp --enable-threads"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-quad-precision --with-pic --enable-openmp --enable-threads"

The use of advanced SIMD instructions (SSE2, AVX*), building of the libraries for different precisions and for MPI/OpenMP/threads can be controlled via custom easyconfig parameters supported by this easyblock, see below.

$ eb -e EB_FFTW -a

...

EASYBLOCK-SPECIFIC
------------------
...
use_avx*                 Configure with --enable-avx (if None, auto-detect support for AVX) [default: None]
use_avx2*                Configure with --enable-avx2 (if None, auto-detect support for AVX2) [default: None]
use_avx512*              Configure with --enable-avx512 (if None, auto-detect support for AVX512) [default: None]
use_sse2*                Configure with --enable-sse2 (if None, auto-detect support for SSE2) [default: None]
with_double_prec*        Enable building of double precision library [default: True]
with_long_double_prec*   Enable building of long-double precision library [default: True]
with_mpi*                Enable building of FFTW MPI library [default: True]
with_openmp*             Enable building of FFTW OpenMP library [default: True]
with_quad_prec*          Enable building of quad precision library [default: True]
with_single_prec*        Enable building of single precision library [default: True]
with_threads*            Enable building of FFTW threads library [default: True]

...

Example easyconfig to use along with this easyblock (eb FFTW.eb --include-easyblocks fftw.py):

name = 'FFTW'
version = '3.3.5'

homepage = 'http://www.fftw.org'
description = """FFTW is a C subroutine library for computing the discrete Fourier transform (DFT)
 in one or more dimensions, of arbitrary input size, and of both real and complex data."""

toolchain = {'name': 'gompi', 'version': '2016b'}
toolchainopts = {'pic': True}

sources = [SOURCELOWER_TAR_GZ]
source_urls = [homepage]

moduleclass = 'numlib'

TODO:

  • take into account --optarch (in particular --optarch=GENERIC)
  • extend to also auto-detecting for ARM (neon, asimd) and POWER (altivec) (thoughts on this @geimer, @JackPerdue?)

cc: @akesandgren, @jhein32, @wpoely86, @rjeschmi, @verdurin

@JackPerdue
Copy link
Contributor

JackPerdue commented Jan 6, 2017 via email

@boegel
Copy link
Member Author

boegel commented Jan 7, 2017

@JackPerdue done!

We should also do this for AARCH64, but I'm tempted to postpone that to a future PR... Thoughts @geimer?

@boegel
Copy link
Member Author

boegel commented Jan 7, 2017

Last commit (cd2781d) also takes into account --optarch=GENERIC, resulting in the --enable-avx* configure options not being used when it is used, see below.

Thoughts on this @pescobar?

$ eb FFTW.eb -x | grep configure_step -A 1
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-single --enable-mpi --with-pic --enable-openmp --enable-threads --enable-avx --enable-avx2 --enable-sse2"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-mpi --with-pic --enable-openmp --enable-threads --enable-avx --enable-avx2 --enable-sse2"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-long-double --enable-mpi --with-pic --enable-openmp --enable-threads"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-quad-precision --with-pic --enable-openmp --enable-threads"

$ eb FFTW.eb -x --optarch=GENERIC | grep configure_step -A 1
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-single --enable-mpi --with-pic --enable-openmp --enable-threads"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-mpi --with-pic --enable-openmp --enable-threads"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-long-double --enable-mpi --with-pic --enable-openmp --enable-threads"
--
[configure_step method]
  running command " ./configure --prefix=/Users/kehoste/.local/easybuild/software/FFTW/3.3.5-gompi-2017a --enable-quad-precision --with-pic --enable-openmp --enable-threads"

@geimer
Copy link
Contributor

geimer commented Jan 7, 2017

@boegel I believe enabling NEON on AArch64 is now fairly trivial. The question is more whether the resulting library works. I'm up for giving this new easyblock a try on my ARM box, though I have no clue how to test FFTW...

@boegel
Copy link
Member Author

boegel commented Jan 7, 2017

@geimer it seems like FFTW has a test suite included, you should be able to run it by including runtest = 'check' in your FFTW easyconfig file...

Are you up for looking into updating this easyblock to also cover AARCH64, or shall I?

I think it's just checking for the neon (only for single precision unless it's AARCH64 then also for double precision) and asimd (single/double only)?

@geimer
Copy link
Contributor

geimer commented Jan 7, 2017

@boegel neon should only show up on AArch32. As far as I know, the AArch64 architecture always includes a double-precision-capable NEON unit which is flagged as asimd (Advanced SIMD). So, yes, two additional if's should suffice.

prec_configopts.append('--enable-%s' % flag)

# NEON (ARM) only for single precision and double precision (on AARCH64)
if (prec == 'single' and self.neon) or (prec == 'double' and self.asimd):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this if condition is correct. I believe one can pass --enable-neon for both single- and double-precision if asimd is flagged.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't asimd imply neon?

I can update this to use prec == 'single' and (self.asimd or self.neon), but it wouldn't make a difference in practice?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is: the neon flag is only available on AArch32 if NEON support is available. And it is single-precision only. On AArch64, there is no neon flag, but only the asimd flag indicating single- and double-precision NEON support. So yes, I believe it would make a difference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks for the clarification, fixed in c7899c2

@geimer
Copy link
Contributor

geimer commented Jan 7, 2017

@JackPerdue You may actually also be interested in VSX support, which seems to be available in POWER7 and above. However, as far as I can see there is no flag in /proc/cpuinfo that indicates availability. But maybe there is some other pseudo-file (e.g., under /sys)?

@boegel boegel changed the title add custom easyblock for FFTW (WIP) add custom easyblock for FFTW (REVIEW) Jan 7, 2017
@JackPerdue
Copy link
Contributor

processor : 0
cpu : POWER7 (architected), altivec supported
clock : 4228.000000MHz
revision : 2.1 (pvr 004a 0201)

Given: https://en.wikipedia.org/wiki/IBM_POWER_microprocessors I'd say grepping /proc/cpuinfo for POWER7 would do.

Thanks for the suggestion. I'll have to look at that here at some point (we are about to reprovision our Power7 cluster)

# along with EasyBuild. If not, see <http://www.gnu.org/licenses/>.
##
"""
EasyBuild support for building and installing FFTW, implemented as an easyblock
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:author?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed, thanks

@boegel
Copy link
Member Author

boegel commented Jan 7, 2017

@JackPerdue VSC support detection implemented in easybuilders/easybuild-framework#2078, I'll update this easyblock accordingly once that is merged

@boegel
Copy link
Member Author

boegel commented Jan 7, 2017

My tests with this easyblock on latest develop of EasyBuild framework (on Intel Haswell, Sandy Bridge and AMD Opteron Magny Cours) all works out fine: FFTW built with this easyblock and running the provided test suite via make check works as expected.


# auto-detect CPU features that can be used and are not enabled/disabled explicitly,
# but only if --optarch=GENERIC is not being used
if self.cfg['auto_detect_cpu_features'] and build_option('optarch') != OPTARCH_GENERIC:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All x86_64 CPUs support at least SSE2, so this could be enabled even with OPTARCH_GENERIC.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, ok, that complicates things a little bit, but I'll look into it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

implemented in cb14353, please rereview?

@boegel
Copy link
Member Author

boegel commented Jan 10, 2017

Thanks for the review/feedback @wpoely86, @geimer and @JackPerdue!

prec_configopts.append('--enable-%s' % flag)

# Altivec (POWER) and SSE only for single precision
for flag in ['altivec', 'sse']:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

altivec does not work for GCC 5 for single precision with Power, see FFTW/fftw3#59 (I also tested GCC6 and the tests fail there as well)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants