Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Added support for ARCHER (Cray XC30). #864

Open
wants to merge 47 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
0784717
lightning.afrl changes
Mar 25, 2015
b226d41
added some host packages
cekees Mar 25, 2015
ac0e31f
added lightning profile
cekees May 21, 2015
9df2e03
added host-openblas.yaml for lightning
cekees May 21, 2015
30150cb
Merge remote-tracking branch 'ssh/master'
cekees May 21, 2015
d2bfbfa
Merge remote-tracking branch 'ssh/master' into cekees/lightning_afrl_…
cekees May 21, 2015
1efcb30
support for lightning.afrl
cekees May 21, 2015
5240c6e
added insert_location step for non-relocatable packages
cekees Jun 12, 2015
6d458ee
updating proteus stacks
cekees Jul 7, 2015
8408a3d
Merge remote-tracking branch 'origin/master' into update_proteus
cekees Jul 7, 2015
c3b54ea
adding h5py, netcdf, ode to default stack
cekees Jul 7, 2015
d52c497
Merge branch 'update_proteus' into add_location
cekees Jul 7, 2015
5e3b73a
added python-netscdf4
cekees Jul 9, 2015
ae31159
Merge remote-tracking branch 'ssh/master' into stable/lightning
cekees Jul 20, 2015
1bfd4cd
added h5py and ode deps to proteus stacks
cekees Jul 20, 2015
c743af0
Merge remote-tracking branch 'ssh/proteus_update' into stable/lightning
cekees Jul 20, 2015
27e17c3
Merge branch 'proteus_update' into add_location
cekees Jul 21, 2015
805c2fa
updating linux2 stack
cekees Jul 21, 2015
d14708f
Merge remote-tracking branch 'origin/add_pcs_api' into proteus_update
cekees Jul 21, 2015
d51a47b
add pcs_api
cekees Jul 21, 2015
d4830df
Merge branch 'proteus_update' into add_location
cekees Jul 21, 2015
296966f
Merge branch 'add_jupyter-pip' into proteus_update
cekees Jul 22, 2015
1331d75
updated jupyter-pip and pythreejs
cekees Jul 22, 2015
2eaa563
Merge remote-tracking branch 'origin/master' into proteus_update
cekees Sep 10, 2015
0355f7b
removed host cmake dependency
cekees Sep 10, 2015
d1e3d75
Merge remote-tracking branch 'ssh/proteus_update' into stable/lightning
cekees Sep 11, 2015
7808dd3
added ode and h5py
cekees Sep 11, 2015
07dbd7f
Merge remote-tracking branch 'ssh/master'
cekees Sep 11, 2015
1a34d81
Merge remote-tracking branch 'ssh/master' into stable/lightning
cekees Sep 12, 2015
b3da785
Merge remote-tracking branch 'ssh/master' into stable/lightning
cekees Oct 1, 2015
acf9d9d
corrected machine type to XC30
cekees Oct 3, 2015
b95922b
Merge remote-tracking branch 'ssh/master' into stable/lightning
cekees Oct 3, 2015
6a6deeb
Added support for ARCHER (Cray XC30).
Oct 30, 2015
02cfd0f
Added specific version numbers for petsc and petsc4py.
Nov 3, 2015
b2276b1
Restored backward compatibility for CrayXE6.
Nov 3, 2015
5552c2d
Merge branch 'master' of https://github.com/hashdist/hashstack into s…
Nov 5, 2015
bb922b8
host-libsci-blas used by blas and scipy modules, and host-libsci-lapa…
Nov 6, 2015
8cf79e0
placed patch for sed command in separate loop as per the fix on fix_h…
Nov 6, 2015
4ed653a
modified make command such that MAKE_NP is set
Nov 6, 2015
fe76e07
Added pip package to allow make develop.
Nov 13, 2015
f780c7f
Added sections to allow customised build paths.
Nov 13, 2015
531cfbd
Removed pip module.
Nov 13, 2015
d390e9f
Added CrayXC30 section.
Nov 13, 2015
d8ad2c4
Added CrayXC30 section.
Nov 13, 2015
30e9f95
Merge remote-tracking branch 'origin/stable/lightning' into stable/ar…
Nov 16, 2015
9a02342
Removed redundant preConfigureCrayXC30 def.
Nov 16, 2015
b3e1a2e
Reinstated https link to github repository.
Nov 16, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions archer.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
extends:
- file: linux.yaml
78 changes: 78 additions & 0 deletions examples/proteus.archer.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# This profile file controls your <#> (HashDist) build environment.

# In the future, we'll provide better incorporation of
# automatic environment detection. For now, have a look
# at the YAML files in the top-level directory and choose
# the most *specific* file that matches your environment.

extends:
- file: archer.yaml

# The packages list specifies all the packages that you
# require installed. <#> will ensure that all packages
# and their dependencies are installed when you build this
# profile.

parameters:
machine: CrayXC30
HOST_MPICC: cc
HOST_MPICXX: CC
HOST_MPIF77: ftn
HOST_MPIF90: ftn
HOST_MPIEXEC: aprun
PATH: |
/usr/bin:/bin:/usr/sbin:/sbin
PROLOGUE: |
source /opt/modules/default/init/bash; export MODULEPATH=/opt/cray/craype/default/modulefiles:/opt/cray/ari/modulefiles:/opt/cray/modulefiles:/opt/modulefiles:/cm/local/modulefiles:/cm/shared/modulefiles:/opt/modules/packages-archer; export MODULESHOME=/opt/modules/default; export MODULE_VERSION=3.2.10; export MODULE_VERSION_STACK=3.2.10; export CRAY_CPU_TARGET=ivybridge; echo "unloading modules"; module unload PrgEnv-cray; echo "loading modules"; module load PrgEnv-gnu; module load cray-mpich; module load cray-libsci; module load cray-tpsl; echo "setting default compilers"; export CC=gcc; export CXX=g++; export FC=gfortran; export F77=gfortran; export F90=gfortran; export CPP=cpp;
DYNAMIC_EXE_LINKER_FLAGS: -dynamic

packages:
launcher:
cmake:
python:
host: false
build_with: |
openssl,bzip2,sqlite
blas:
use: openblas
lapack:
# use: host-acml-lapack
daetk:
mpi:
use: host-mpi
mpi4py:
nose:
hdf5:
ipython:
matplotlib:
parmetis:
without_check: true
petsc:
version: 3.6.1
without_check: true
build_with: |
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should update to the latest petsc version by adding version: 3.6.1 above without_check: true

parmetis
download: |
ptscotch, superlu, superlu_dist, spooles, hypre, blacs, scalapack, mumps
coptflags: -O2
link: shared
debug: false
petsc4py:
version: 3.6.0
with_conf: true
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

petsc4py currently needs version: 3.6.0 to work with petsc version 3.6.1. (The petsc4py 3.6.1 was not out yet last time I checked).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've made the changes you recommended and daetk now compiles.

pytables:
sphinx:
superlu:
sympy:
tetgen:
triangle:
memory_profiler:
scipy:
# netcdf4f:
# python-netcdf4:
h5py:
ode:
# pyode:
# vtk:
# pyvtk:

7 changes: 4 additions & 3 deletions pkgs/hdf5/hdf5.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,16 @@ build_stages:
bash: |
export CC=${MPICC}

- when: machine == 'CrayXE6'
- when: machine == 'CrayXE6' or machine == 'CrayXC30'
name: make_exe_builds_static
handler: bash
after: configure
before: make
bash: |
for f in {test,testpar,tools/*,perform,hl/*,hl/*/*}/Makefile
for f in {src,test,testpar,tools/*,perform,hl/*,hl/*/*}/Makefile
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mbareford this causes a runtime problem with the hdf5 shared libraries when I test on our XC30. The issue is that the '-static' option added in the original patch was only intended for the directories in the original list so it breaks the shared libs at runtime when it is also applied to src. I made a branch fix_hdf5_sed that applies your sed fix in a second loop and verified that it works. Could you merge that into your branch and test on Archer?
/cc @tridelat @adimako

do
sed -i 's/CCLD = $(CC)/CCLD = $(CC) -static/g' $f
sed -i "s/sed -e 's\/-L\/:\/g' -e 's\/ \/\/g'/sed -e 's\/-L\/:\/g' -e 's\/-rpath=\/:\/g' -e 's\/-Wl,\/\/g' -e 's\/ \/\/g'/g" $f
done

- name: configure
Expand All @@ -40,7 +41,7 @@ build_stages:
'--enable-largefile=yes',
'--with-default-api-version=v18']

- when: machine == 'CrayXE6'
- when: machine == 'CrayXE6' or machine == 'CrayXC30'
name: configure
extra: ['--enable-static-exec',
'--with-zlib=${ZLIB_DIR}',
Expand Down