Building on Edison (intel-nersc)

[xylar]

I'm building the current ocean/develop on Edison, details below. When I run make intel-nersc, I'm running into an error:

(cd inc; /global/homes/x/xylar/mpas_work/model/edison/temp/src/tools/registry/parse < ../Registry_processed.xml )

Please verify that both the operating system and the processor support Intel(R) F16C instructions.

Makefile:32: recipe for target 'gen_includes' failed

The issue is that the compiler is configured for ivybridge, whereas the login nodes are sandybridge (see http://www.nersc.gov/users/computational-systems/edison/updates-and-status/open-issues/compiling-serial-codes-to-run-on-login-nodes-in-the-intel-programming-environment/)

I am able to build successfully if I make the following edits to the Makefile:

-       "FC_SERIAL = ftn" \
-       "CC_SERIAL = cc" \
-       "CXX_SERIAL = CC" \
+       "FC_SERIAL = ifort" \
+       "CC_SERIAL = icc" \
+       "CXX_SERIAL = icpc" \

However, editing the Makefile isn't a very practical solution, since I would need to do this every time I create a new branch on Edison.

How are other folks handling MPAS builds on Edison?

Details

Steps to set up the environment:

module unload PrgEnv-intel
module unload PrgEnv-cray
module unload PrgEnv-gnu
module unload intel
module unload cce
module unload gcc
module unload cray-parallel-netcdf
module unload cray-parallel-hdf5
module unload cray-libsci
module unload cray-netcdf
module unload cray-hdf5
module unload cray-netcdf-hdf5parallel
module unload papi
module unload cray-petsc
module unload esmf

module load PrgEnv-intel/6.0.4
module unload intel
module load intel/18.0.1.163
module unload cray-libsci

module unload cray-netcdf-hdf5parallel
module load cray-netcdf-hdf5parallel/4.4.1.1.3
module load cray-hdf5-parallel/1.10.1.1
module load cray-parallel-netcdf/1.8.1.3

export CORE=ocean
export NETCDF=${NETCDF_DIR}
export PNETCDF=${PARALLEL_NETCDF_DIR}
export PIO=/global/homes/x/xylar/mpas_work/model/${NERSC_HOST}/pio/pio2.3.1
export USE_PIO2=true

The result:

> module list
Currently Loaded Modulefiles:
  1) modules/3.2.10.6
  2) nsg/1.2.0
  3) cray-mpich/7.6.2
  4) git/2.15.1
  5) altd/2.0
  6) darshan/3.1.4
  7) craype-ivybridge
  8) craype-network-aries
  9) craype/2.5.12
 10) udreg/2.3.2-6.0.5.0_13.12__ga14955a.ari
 11) ugni/6.0.14-6.0.5.0_16.9__g19583bb.ari
 12) pmi/5.0.12
 13) dmapp/7.1.1-6.0.5.0_49.8__g1125556.ari
 14) gni-headers/5.0.12-6.0.5.0_2.15__g2ef1ebc.ari
 15) xpmem/2.2.4-6.0.5.1_8.18__g35d5e73.ari
 16) job/2.2.2-6.0.5.0_8.47__g3c644b5.ari
 17) dvs/2.7_2.2.64-6.0.5.2_15.3__g97b34c4
 18) alps/6.5.28-6.0.5.0_18.6__g13a91b6.ari
 19) rca/2.2.16-6.0.5.0_15.34__g5e09e6d.ari
 20) atp/2.1.1
 21) PrgEnv-intel/6.0.4
 22) intel/18.0.1.163
 23) cray-netcdf-hdf5parallel/4.4.1.1.3
 24) cray-hdf5-parallel/1.10.1.1
 25) cray-parallel-netcdf/1.8.1.3

[xylar]

E3SM appears to solve this problem by adding the flag:

-target-cpu=sandybridge

when building tools that are to run on login nodes.

[philipwjones]

@xylar there is already a flag for this: when you type make, use the flag:
TOOL_TARGET_ARCH=“-target-cpu=sandybridge”

[xylar]

Yep, I just figured that out. Thanks @philipwjones.