Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Update MAPL and other GEOS-ESM packages #174

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
1434223
Update GFE and MAPL packages
mathomp4 Sep 19, 2022
dd12538
Add FLAP
mathomp4 Sep 19, 2022
5e969bc
Bring back old yafyaml
mathomp4 Sep 19, 2022
5fdccf7
Try and fix up CI issues
mathomp4 Sep 19, 2022
7a88dd0
Fix more formatting issues
mathomp4 Sep 19, 2022
a1a5dbc
Fix more formatting issues. Again.
mathomp4 Sep 19, 2022
bff5c07
Hopefully final style issue
mathomp4 Sep 19, 2022
7c3c89c
Split msg in hopes of passing CI
mathomp4 Sep 19, 2022
d157dc1
Make msg small
mathomp4 Sep 19, 2022
45f3e4a
Fix up depends_on
mathomp4 Sep 19, 2022
2b0073c
Add mepo
mathomp4 Sep 21, 2022
4374475
Add 32bit option to FMS
mathomp4 Sep 21, 2022
75f5978
Add python and perl as prereqs of MAPL
mathomp4 Sep 28, 2022
024e0e6
Add geosgcm package
mathomp4 Sep 28, 2022
1802f57
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Nov 22, 2022
37ad170
Update for beta
mathomp4 Nov 22, 2022
e76cffb
Update fargparse
mathomp4 Nov 22, 2022
1d2b1e5
Update pfunit
mathomp4 Nov 22, 2022
bb74cdb
Add esmf 8.4.0
mathomp4 Nov 22, 2022
eda087a
Update versions in MAPL
mathomp4 Nov 22, 2022
64f9a62
Add GEOSgcm v10.23.1
mathomp4 Nov 22, 2022
d6159e0
Update mepo
mathomp4 Nov 22, 2022
77bb746
Move GFE packages from jcsda-emc to builtin
mathomp4 Nov 30, 2022
639cf7b
Add tclune, remove kgerheiser
mathomp4 Nov 30, 2022
ef144c0
Make tclune prime maintainer
mathomp4 Nov 30, 2022
c66c45f
update geosgcm package
mathomp4 Nov 30, 2022
a617cb9
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Dec 9, 2022
11cd1a9
Fix v3.3.0
mathomp4 Dec 12, 2022
8d0c71d
Update MAPL versions
mathomp4 Dec 13, 2022
fb3c55e
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Dec 14, 2022
bbb982a
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Jan 5, 2023
fe7ef86
cleanup esmf
mathomp4 Jan 5, 2023
871a4f5
Add MAPL 2.34
mathomp4 Jan 5, 2023
00a9498
Add mepo 1.48
mathomp4 Jan 5, 2023
c5da6c6
Update geosgcm package
mathomp4 Jan 5, 2023
274a664
Pull in @bd4 pfunit changes from mainline spack
mathomp4 Jan 5, 2023
afe0a5f
Update fms comment
mathomp4 Jan 18, 2023
0249644
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Feb 2, 2023
fff78c5
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Feb 27, 2023
c1b264c
Update mapl versions
mathomp4 Feb 27, 2023
1b1cce2
Move geosgcm to use complete tarball
mathomp4 Feb 27, 2023
866633f
Move to jcsda esmf package
mathomp4 Feb 27, 2023
83bd44d
Move to spack mainline pfunit
mathomp4 Feb 27, 2023
909df35
Revert back to older pfunit package
mathomp4 Feb 27, 2023
e37cfee
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Mar 7, 2023
d61ce82
Fixes to allow pfunit variant to build
mathomp4 Mar 9, 2023
9cea6e3
Add latest geosgcm version
mathomp4 Mar 9, 2023
3febd00
Add list_url for mapl
mathomp4 Mar 9, 2023
8662c6b
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Mar 14, 2023
84e6e16
Merge branch 'jcsda_emc_spack_stack' into feature/mathomp4/update-add…
mathomp4 Apr 26, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions var/spack/repos/builtin/packages/fms/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,11 @@ class Fms(CMakePackage):
git = "https://github.com/JCSDA/fms.git"
# *DH 20220602

variant(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this we should pull in the updates from https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/fms/package.py (can be a separate PR so you don't have to deal with it)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can do this if you like

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@climbfuji If you can, please. I have a PR in to spack mainline to update some other GFE packages.

It's also possible we might want to wait for the ESMF PR from @theurich before pulling in spack? Not sure.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I will say I tried using the newer spack maintainers() function in this PR but spack wasn't happy. It's possible my spack was old?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We haven't yet pulled that functionality from main spack, so with NOAA-EMC spack it will yell at you about the function being undefined :)

"32bit",
default=False,
description="Build a version of the library with default 32 bit reals",
)
variant(
"64bit",
default=True,
Expand Down Expand Up @@ -98,6 +103,7 @@ class Fms(CMakePackage):

def cmake_args(self):
args = [
self.define_from_variant("32BIT"),
self.define_from_variant("64BIT"),
self.define_from_variant("GFS_PHYS"),
self.define_from_variant("OPENMP"),
Expand Down
80 changes: 69 additions & 11 deletions var/spack/repos/builtin/packages/mapl/package.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2013-2021 Lawrence Livermore National Security, LLC and other
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
Expand All @@ -20,11 +20,37 @@ class Mapl(CMakePackage):
"""

homepage = "https://github.com/GEOS-ESM/MAPL"
url = "https://github.com/GEOS-ESM/MAPL/archive/refs/tags/v2.8.1.tar.gz"
url = "https://github.com/GEOS-ESM/MAPL/archive/refs/tags/v2.33.0.tar.gz"
list_url = "https://github.com/GEOS-ESM/MAPL/tags"
git = "https://github.com/GEOS-ESM/MAPL.git"

maintainers = ["mathomp4", "kgerheiser", "climbfuji", "edwardhartnett", "Hang-Lei-NOAA"]
maintainers = ["mathomp4", "tclune", "climbfuji", "edwardhartnett", "Hang-Lei-NOAA"]

version("develop", branch="develop")
version("main", branch="main")

# Retrieved by spack checksum mapl@x.y.z
version("2.35.2", sha256="12d2c3fa264b702253e4792d858f67002fa04ce1c60db341803bc000abb3b7a2")
version("2.34.3", sha256="8b750754cf5823771f2149d50f9aef585bcf194ca4635e1807c302d4020077e9")
version("2.34.2", sha256="e46a763084027fe0f326d515e0648b814a82720948062405e03046531f7bb948")
version("2.34.1", sha256="d2a504f08a4b416c3993d59630f226925bdaeb71488a1706decc49893dc8bcd0")
version("2.34.0", sha256="4188df84654beed5c7fc3a96a3fb19289ebfc4020c4e14d52088d2ead2745f73")
version("2.33.0", sha256="a36680d3186cd0399240e9739f5497310bac3f9563f55f77775acf53fa5491bb")
version("2.32.0", sha256="f0eaec4b6d2514516a77cf426b656655d66f2e0801e639175dddfbd0648997f3")
version("2.31.0", sha256="dcee4f0d02cc56c29437ab0dfc2f1ae78acce3af84630a55136f8e3626ce36c9")
version("2.30.3", sha256="52bf65fcd5530079dab32ad1127fbaed940e218a99a89ac5bc8f8330b2094962")
version("2.30.2", sha256="508b6d0e42593dbedbbb2524bafe7340be7306f80479fde2e4d55868a29753e7")
version("2.30.1", sha256="df691ac12422184f05f37cb926541d18577ce129e0d6f10e91c90c3922bff572")
version("2.30.0", sha256="71c469d4618ae97813f784208a102a38c4a30d5ff6301c08cb4fdbd1b1931202")
version("2.29.0", sha256="aeca5258bc88526895715e3fd3604d43916b5143e948866fea4c1a608120598d")
version("2.28.0", sha256="3a1f0c9b8b5a1932b2f036deb5463ddbef58a472ee4759c6cc3a4a871b8fe613")
version("2.27.1", sha256="1aeca20b49729d0212bca764510cb069839d1f2b702c46c8b29a6b2535b2052c")
version("2.27.0", sha256="a322257522f7fb2768668c02272ae8246ba4be81aa41d8015ce947ba871ce3fb")
version("2.26.0", sha256="4ef9a1eeffc0521bd6fb4ea5ab261e07d3b5a46f0a6c43673ee169bf0d624bb8")
version("2.25.0", sha256="f3ce71004f4274cedee28852cc105e0bf51a86cafc4d5f5af9de6492c4be9402")
version("2.24.0", sha256="cd15ffd6897c18e64267e5fa86523402eb48cbf638ad5f3b4b5b0ff8939d1936")
version("2.23.1", sha256="563f3e9f33adae298835e7de7a4a29452a2a584d191248c59494c49d3ee80d24")
version("2.23.0", sha256="ae25ec63d0f288599c668f35fdbccc76abadbfc6d48f95b6eb4e7a2c0c69f241")
version("2.22.0", sha256="3356b8d29813431d272c5464e265f3fe3ce1ac7f49ae6d41da34fe4b82aa691a")
version("2.12.3", sha256="e849eff291939509e74830f393cb2670c2cc96f6160d8060dbeb1742639c7d41")
version("2.11.0", sha256="76351e026c17e2044b89085db639e05ba0e7439a174d14181e01874f0f93db44")
Expand All @@ -42,6 +68,12 @@ class Mapl(CMakePackage):
tag="v3.24.0",
when="@2.34.0:",
)
resource(
name="esma_cmake",
git="https://github.com/GEOS-ESM/ESMA_cmake.git",
tag="v3.24.0",
when="@2.34.0:",
)
resource(
name="esma_cmake",
git="https://github.com/GEOS-ESM/ESMA_cmake.git",
Expand All @@ -65,14 +97,22 @@ class Mapl(CMakePackage):
# Patch to add missing MPI Fortran target to top-level CMakeLists.txt
patch("mapl-2.12.3-mpi-fortran.patch", when="@:2.12.3")

variant("flap", default=False)
variant("pflogger", default=False)
variant("flap", default=False, description="Build with FLAP support")
variant("pflogger", default=False, description="Build with pFlogger support")
variant("fargparse", default=False, description="Build with fArgParse support")
variant("esma_gfe_namespace", default=True)
variant("shared", default=True)
variant("shared", default=True, description="Build as shared library")
variant("debug", default=False, description="Make a debuggable version of the library")
variant("extdata2g", default=False, description="Use ExtData2G")
variant("pnetcdf", default=True, description="Use parallel netCDF")
variant("pfunit", default=False, description="Build with pFUnit support")
variant("f2py", default=False, description="Build with f2py support")

variant(
"build_type",
default="Release",
description="The build type to build",
values=("Debug", "Release", "Aggressive"),
)

depends_on("cmake@3.17:")
depends_on("mpi")
Expand All @@ -85,22 +125,40 @@ class Mapl(CMakePackage):
depends_on("esmf", when="@:2.12.99")
depends_on("esmf~debug", when="~debug")
depends_on("esmf+debug", when="+debug")
depends_on("yafyaml@:0.5.1")

depends_on("gftl@1.5.5:")
depends_on("gftl-shared@1.3.1:")

# fArgParse not yet available
conflicts("+fargparse")
# There was an interface change in yaFyaml, so we need to control versions
# MAPL 2.22 and older uses older version, MAPL 2.23+ and higher uses newer
depends_on("yafyaml@1.0-beta5", when="@:2.22+extdata2g")
depends_on("yafyaml@1.0.4:", when="@2.23:+extdata2g")

# pFlogger depends on yaFyaml in the same way. MAPL 2.22 and below uses old
# yaFyaml so we need to use old pFlogger, but MAPL 2.23+ uses new yaFyaml
depends_on("pflogger@:1.6", when="@:2.22+pflogger")
depends_on("pflogger@1.9.1:", when="@2.23:+pflogger")

# fArgParse v1.4.1 is the first usable version with MAPL
depends_on("fargparse@1.4.1:", when="+fargparse")

depends_on("pfunit@4.2: +mpi +fhamcrest", when="+pfunit")
depends_on("flap", when="+flap")

depends_on("ecbuild")

depends_on("python@3:")
depends_on("py-numpy", when="+f2py")
depends_on("perl")

def cmake_args(self):
args = [
self.define_from_variant("BUILD_WITH_FLAP", "flap"),
self.define_from_variant("BUILD_WITH_PFLOGGER", "pflogger"),
self.define_from_variant("ESMA_USE_GFE_NAMESPACE", "esma_gfe_namespace"),
self.define_from_variant("BUILD_WITH_FARGPARSE", "fargparse"),
self.define_from_variant("BUILD_SHARED_MAPL", "shared"),
self.define_from_variant("USE_EXTDATA2G", "extdata2g"),
self.define_from_variant("USE_F2PY", "f2py"),
"-DCMAKE_C_COMPILER=%s" % self.spec["mpi"].mpicc,
"-DCMAKE_CXX_COMPILER=%s" % self.spec["mpi"].mpicxx,
"-DCMAKE_Fortran_COMPILER=%s" % self.spec["mpi"].mpifc,
Expand Down
14 changes: 10 additions & 4 deletions var/spack/repos/builtin/packages/pfunit/package.py
Original file line number Diff line number Diff line change
Expand Up @@ -113,20 +113,26 @@ def cmake_args(self):
self.define_from_variant("BUILD_SHARED", "shared"),
"-DCMAKE_Fortran_MODULE_DIRECTORY=%s" % spec.prefix.include,
self.define_from_variant("BUILD_DOCS", "docs"),
"-DMAX_ASSERT_RANK=%s" % spec.variants["max_array_rank"].value,
]

if self.spec.satisfies("%gcc@10:"):
args.append("-DCMAKE_Fortran_FLAGS_DEBUG=-g -O2 -fallow-argument-mismatch")

if spec.satisfies("@4.0.0:"):
args.append("-DSKIP_MPI=%s" % ("YES" if "~mpi" in spec else "NO"))
args.append("-DSKIP_OPENMP=%s" % ("YES" if "~openmp" in spec else "NO"))
args.append("-DSKIP_FHAMCREST=%s" % ("YES" if "~fhamcrest" in spec else "NO"))
args.append("-DSKIP_ESMF=%s" % ("YES" if "~esmf" in spec else "NO"))
args.append("-DMAX_ASSERT_RANK=%s" % spec.variants["max_array_rank"].value)
else:
if spec.satisfies("%gcc@10:"):
args.append("-DCMAKE_Fortran_FLAGS_DEBUG=-g -O2 -fallow-argument-mismatch")

args.append(self.define_from_variant("MPI", "mpi"))
args.append(self.define_from_variant("OPENMP", "openmp"))
args.append("-DMAX_RANK=%s" % spec.variants["max_array_rank"].value)

if spec.satisfies("@:4.2.1") and spec.satisfies("%gcc@5:"):
# prevents breakage when max_array_rank is larger than default. Note
# that 4.0.0-4.2.1 still had a 512 limit
args.append("-DCMAKE_Fortran_FLAGS=-ffree-line-length-none")

if spec.satisfies("+mpi"):
args.extend(
Expand Down
19 changes: 19 additions & 0 deletions var/spack/repos/jcsda-emc/packages/flap/package.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)


from spack.package import *


class Flap(CMakePackage):
"""Fortran command Line Arguments Parser for poor people"""

homepage = "https://github.com/mathomp4/FLAP"
url = "https://github.com/mathomp4/FLAP/archive/refs/tags/geos/v1.10.0.tar.gz"
git = "https://github.com/mathomp4/FLAP.git"

maintainers = ["mathomp4"]

version("geos", branch="geos", submodules=True)
84 changes: 84 additions & 0 deletions var/spack/repos/jcsda-emc/packages/geosgcm/package.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack.package import *


class Geosgcm(CMakePackage):
"""
GEOS Earth System Model GEOSgcm Fixture
"""

homepage = "https://github.com/GEOS-ESM/GEOSgcm"
url = "https://github.com/GEOS-ESM/GEOSgcm/releases/download/v10.25.0/GEOSgcm-v10.25.0.COMPLETE.tar.xz"
git = "https://github.com/GEOS-ESM/GEOSgcm.git"

maintainers = ["mathomp4", "tclune"]

def url_for_version(self, version):
url_base = "https://github.com/GEOS-ESM/GEOSgcm/releases/download/"
url = url_base + "v{0}/GEOSgcm-v{0}.COMPLETE.tar.xz"

return url.format(version)

version("main", branch="main")

version("10.25.1", sha256="f541dc38163c554177c21746999e8cd9fb2daa127de1a4171ef82ac1a536fa2f")
version("10.25.0", sha256="08887ca652387d51d92e62fd16c0944e6c2a53513abb081cc1150a28d128c4c3")
version("10.24.0", sha256="59e35b446f258a36ba41f753c5724eb08c7cd0ccbafca1cfb329419cac045e24")
version("10.23.3", sha256="131585394b2ece57af7585247e73d7bff6e27c42daa40cd05b47f90a471d06cf")
version("10.23.2", sha256="0f3adb6f65e57cab372bbf1e00b4fcd5d6e7009d4851d4968d2f4c87345a43ec")
version("10.23.1", sha256="6469331858ef005bf0d8eb90fe1c4d7b37648bebc0529a32284a0ceaf8d9274f")
version("10.23.0", sha256="99579429f6116e4f4e82587f423132dc896f7cfd336499487094ddeb227b450a")

variant("f2py", default=False, description="Build with f2py support")
variant("extdata2g", default=True, description="Use ExtData2G")

variant(
"build_type",
default="Release",
description="The build type to build",
values=("Debug", "Release", "Aggressive"),
)

depends_on("cmake@3.17:")
depends_on("mpi")
depends_on("ecbuild")

# These are for MAPL AGC and stubber
depends_on("python@3:")
depends_on("py-pyyaml")
depends_on("perl")

# These are similarly the dependencies of MAPL. Not sure if we'll ever use MAPL as library
depends_on("hdf5")
depends_on("netcdf-c@4.9.0:")
depends_on("netcdf-fortran@4.6.0:")
depends_on("esmf@8.4.0:")
depends_on("gftl@1.5.5:")
depends_on("gftl-shared@1.3.1:")
depends_on("yafyaml@1.0.4:", when="+extdata2g")
depends_on("pflogger@1.9.1:")
depends_on("fargparse@1.4.1:")
depends_on("flap@geos")

# MAPL as library would be like:
# depends_on("mapl@2.34:+flap+pflogger+extdata2g+fargparse")
# but we don't want to do this in general due to the speed of MAPL development

# When we move to FMS as library, we'll need to add this:
#depends_on("fms@2022.04:~gfs_phys+fpic~quad_precision+32bit+64bit+yaml constants=GEOS")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the time comes, we'll need a way to describe the different variants in the module names (not now!), e.g. fms/2022.04-geos based on the constants value).


def cmake_args(self):
args = [
self.define_from_variant("USE_F2PY", "f2py"),
self.define_from_variant("USE_EXTDATA2G", "extdata2g"),
self.define("CMAKE_MODULE_PATH", self.spec["esmf"].prefix.cmake),
"-DCMAKE_C_COMPILER=%s" % self.spec["mpi"].mpicc,
"-DCMAKE_CXX_COMPILER=%s" % self.spec["mpi"].mpicxx,
"-DCMAKE_Fortran_COMPILER=%s" % self.spec["mpi"].mpifc,
]

return args
32 changes: 32 additions & 0 deletions var/spack/repos/jcsda-emc/packages/mepo/package.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Copyright 2013-2022 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack.package import *


class Mepo(Package):
"""
Tool to manage (m)ultiple git r(epo)sitories
"""

homepage = "https://github.com/GEOS-ESM/mepo/"
url = "https://github.com/GEOS-ESM/mepo/archive/refs/tags/v1.47.0.tar.gz"

maintainers = ['mathomp4']

# retrieved with spack checksum mepo@x.y.z
version("1.48.0", sha256="df873e2b9c6f63bd7c3e31a6656980660af22cbe48db39c80c1b54feb0c98043")
version("1.47.0", sha256="5e9484f4cd5d2c7c6a84954e949ce78cf94b0a709779f586ecbd17d50920f2ff")
version("1.46.0", sha256="bf4dc62807b82f0ddc110a9e188141a70eb3e5207121b26519dcef39938d6c52")
version('1.45.0', sha256='276ca8eb12f7bd9e5117a7f7a2596147456b7db05dbe93f5cd778da0b2ed80de')

depends_on('python@3.9:', type=('run'))
depends_on('py-pyyaml', type=('run'))

def install(self, spec, prefix):
install_tree(self.stage.source_path, prefix)

def setup_run_environment(self, env):
env.prepend_path('PATH', self.prefix)