Skip to content

Commit

Permalink
Merge BRIDGE into main in prep for release v18.0.0 (#744)
Browse files Browse the repository at this point in the history
  • Loading branch information
gmao-rreichle committed Mar 22, 2024
2 parents f8e314e + a699160 commit 2deea0b
Show file tree
Hide file tree
Showing 70 changed files with 7,640 additions and 3,819 deletions.
10 changes: 6 additions & 4 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
@@ -1,22 +1,24 @@
version: 2.1

# Anchor to prevent forgetting to update a version
baselibs_version: &baselibs_version v7.7.0
# Anchors in case we need to override the defaults from the orb
#baselibs_version: &baselibs_version v7.17.0
#bcs_version: &bcs_version v11.4.0

orbs:
ci: geos-esm/circleci-tools@1
ci: geos-esm/circleci-tools@2

workflows:
build-test:
jobs:
# Build GEOSldas
- ci/build:
name: build-GEOSldas-on-<< matrix.compiler >>
context:
- docker-hub-creds
matrix:
parameters:
compiler: [ifort, gfortran]
baselibs_version: *baselibs_version
#baselibs_version: *baselibs_version
repo: GEOSldas
mepodevelop: false
persist_workspace: false # Needs to be true to run fv3/gcm experiment, costs extra
7 changes: 3 additions & 4 deletions .github/workflows/enforce-labels.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@ jobs:
require-label:
runs-on: ubuntu-latest
steps:
- uses: mheap/github-action-required-labels@v3
- uses: mheap/github-action-required-labels@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
mode: minimum
count: 1
labels: "0 diff,0 diff trivial,Non 0-diff,0 diff structural,0-diff trivial,Not 0-diff,0-diff,automatic,0-diff uncoupled"
labels: "0 diff,0 diff trivial,Non 0-diff,0 diff structural,0-diff trivial,Not 0-diff,0-diff,automatic,0-diff uncoupled,github_actions"
add_comment: true
message: "This PR is being prevented from merging because you have not added one of our required labels: {{ provided }}. Please add one so that the PR can be merged."

blocking-label:
runs-on: ubuntu-latest
steps:
- uses: mheap/github-action-required-labels@v3
- uses: mheap/github-action-required-labels@v5
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
Expand All @@ -30,4 +30,3 @@ jobs:
labels: "Contingent - DNA,Needs Lead Approval,Contingent -- Do Not Approve"
add_comment: true
message: "This PR is being prevented from merging because you have added one of our blocking labels: {{ provided }}. You'll need to remove it before this PR can be merged."

4 changes: 2 additions & 2 deletions .github/workflows/push-to-develop.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Run the action
uses: devops-infra/action-pull-request@v0.4
uses: devops-infra/action-pull-request@v0.5.5
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
source_branch: develop
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release-tarball.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
path: ${{ github.event.repository.name }}-${{ github.event.release.tag_name }}

- name: Checkout mepo
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
repository: GEOS-ESM/mepo
path: mepo
Expand Down
59 changes: 35 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ This document explains how to build, set up, and run the GEOS land modeling and

## How to Build GEOSldas

### Step 1: Load the Build Modules
### Step 1: Load the Build Modules

Load the `GEOSenv` module provided by the GMAO Software Infrastructure team. It contains the latest `git`, `CMake`, and `mepo` modules and must be loaded in any interactive window that is used to check out and build the model.

Expand All @@ -13,51 +13,60 @@ module use -a (path)
module load GEOSenv
```

where `(path)` depends on the computer and operating system:
where `(path)` depends on the computing system; at NCCS, `(path)` also depends on the operating system (SLES12 on Skylake and Cascade Lake nodes; SLES15 on Milan nodes, as of Jan. 2024):

| System | Path |
| ------------- |---------------------------------------------------|
| NCCS | `/discover/swdev/gmao_SIteam/modulefiles-SLES12` |
| NCCS Discover | `/discover/swdev/gmao_SIteam/modulefiles-SLES12` |
| | `/discover/swdev/gmao_SIteam/modulefiles-SLES15` |
| NAS | `/nobackup/gmao_SIteam/modulefiles` |
| GMAO desktops | `/ford1/share/gmao_SIteam/modulefiles` |

Step 1 can be coded into the user's shell configuration file (e.g., `.bashrc` or `.cshrc`). See the [GEOSgcm Wiki](https://github.com/GEOS-ESM/GEOSgcm/wiki/) for sample shell configuration files.

### Step 2: Obtain the Model

For development work, clone the _entire_ repository and use the `develop` branch as your starting point (equivalent to the `UNSTABLE` tag in the old CVS repository):
For development work, clone the _entire_ repository and use the `develop` branch as your starting point:
```
git clone -b develop git@github.com:GEOS-ESM/GEOSldas.git
```
For science runs, you can also obtain a specific tag or branch _only_ (as opposed to the _entire_ repository), e.g.:
For science runs, you can also obtain a specific tag or branch _only_ (as opposed to the _entire_ repository), e.g.:
```
git clone -b v17.9.1 --single-branch git@github.com:GEOS-ESM/GEOSldas.git
```


### Step 3: Build the Model

To build the model in a single step, do the following:
To build the model in a single step, do the following from a head node:
```
cd ./GEOSldas
parallel_build.csh
```
from a head node. Doing so will check out all the external repositories of the model (albeit only on the first run, [see subsection on mepo below](#mepo)!) and build the model. When done, the resulting model build will be found in `build/` and the installation will be found in `install/`, with setup scripts like `ldas_setup` in `install/bin`.
```
This checks out all the external repositories of the model (albeit only on the first run, [see subsection on mepo below](#mepo)!) and then builds and installs the model.

At **NCCS**, the default is to build GEOSldas on SLES12 (Skylake or Cascade Lake nodes); to build GEOSldas on SLES15 (Milan nodes), use `parallel_build.csh -mil`.

The resulting model build is found in `build[-SLESxx]/`, and the installation is found in `install[-SLESxx]/`, with setup scripts like `ldas_setup` in `install[-SLESxx]/bin`.

To obtain a build that is suitable for debugging, use `parallel_build.csh -debug`, which will build in `build-Debug/` and install in `install-Debug/`. There is also an option for aggressive optimization. For details, see [GEOSldas Wiki](https://github.com/GEOS-ESM/GEOSldas/wiki).
To obtain a build that is suitable for debugging, use `parallel_build.csh -debug`, which builds in `build-Debug[-SLESxx]/` and installs in `install-Debug[-SLESxx]/`. There is also an option for aggressive optimization. For details, see the [GEOSldas Wiki](https://github.com/GEOS-ESM/GEOSldas/wiki).

See below for how to build the model in multiple steps.
Instructions for building the model in multiple steps are provided below.

---

## How to Set Up (Configure) and Run GEOSldas

a) Set up the job as follows:

a) At **NCCS**, GEOSldas must be built, configured, and run on the same operating system. To run GEOSldas on Milan nodes (SLES15), start with `ssh discover-mil`.

b) Set up the job as follows:

```
cd (build_path)/GEOSldas/install/bin
cd (build_path)/GEOSldas/install[-SLESxx]/bin
source g5_modules [for bash or zsh: source g5_modules.[z]sh]
./ldas_setup setup [-v] (exp_path) ("exe"_input_filename) ("bat"_input_filename)
```
```

where

Expand All @@ -70,19 +79,19 @@ where

The three arguments for `ldas_setup` are positional and must be ordered as indicated above.

The latter two files contain essential information about the experiment setup.
The latter two files contain essential information about the experiment setup.
Sample files can be generated as follows:
```
```
ldas_setup sample --exeinp > YOUR_exeinp.txt
ldas_setup sample --batinp > YOUR_batinp.txt
```

Edit these sample files following the examples and comments within the sample files.
Edit these sample files following the examples and comments within the sample files.

The ldas_setup script creates a run directory and other directories at:
`[exp_path]/[exp_name]`

Configuration input files will be created at:
Configuration input files are created at:
`[exp_path]/[exp_name]/run`

For more options and documentation, use any of the following:
Expand All @@ -92,16 +101,19 @@ ldas_setup sample -h
ldas_setup setup -h
```

b) Configure the experiment output by editing the ```./run/HISTORY.rc``` file as needed.
c) Configure the experiment output by editing the ```./run/HISTORY.rc``` file as needed.

c) Run the job:
d) Run the job:
```
cd [exp_path]/[exp_name]/run/
sbatch lenkf.j
```

For more information, see the files in `./doc/`.
Moreover, descriptions of the configuration (resource) parameters are included in the sample "exeinp" and "batinp" files that can be generated using `ldas_setup`.
At **NCCS**, the appropriate SLURM directive `#SBATCH --constraint=[xxx]` is automatically added into `lenkf.j` depending on the operating system.

For more information, see the files in `./doc/`. Moreover, descriptions of the configuration (resource) parameters are included in the sample "exeinp" and "batinp" files that can be generated using `ldas_setup`.



-----------------------------------------------------------------------------------

Expand Down Expand Up @@ -138,15 +150,14 @@ We currently do not allow in-source builds of GEOSldas. So we must make a direct
```
mkdir build
```
The advantages of this is that you can build both a Debug and Release version with the same clone if desired.

#### Run CMake
CMake generates the Makefiles needed to build the model.
```
cd build
cmake .. -DBASEDIR=$BASEDIR/Linux -DCMAKE_Fortran_COMPILER=ifort -DCMAKE_INSTALL_PREFIX=../install
```
This will install to a directory parallel to your `build` directory. If you prefer to install elsewhere change the path in:
This installs into a directory parallel to your `build` directory. If you prefer to install elsewhere change the path in:
```
-DCMAKE_INSTALL_PREFIX=<path>
```
Expand All @@ -156,7 +167,7 @@ and CMake will install there.
```
make -j6 install
```
If you are at NCCS, you **should** run `make -j6 install` on an interactive _compute_ node.
If you are at NCCS, you **should** run `make -j6 install` on an interactive _compute_ node.


## Contributing
Expand Down
19 changes: 13 additions & 6 deletions components.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,44 @@ GEOSldas:
env:
local: ./@env
remote: ../ESMA_env.git
tag: v4.9.1
tag: v4.23.0

cmake:
local: ./@cmake
remote: ../ESMA_cmake.git
tag: v3.28.0
tag: v3.41.0

ecbuild:
local: ./@cmake/@ecbuild
remote: ../ecbuild.git
tag: geos/v1.3.0

NCEP_Shared:
local: ./src/Shared/@NCEP_Shared
remote: ../NCEP_Shared.git
tag: v1.3.0
sparse: ./config/NCEP_Shared.sparse

GMAO_Shared:
local: ./src/Shared/@GMAO_Shared
remote: ../GMAO_Shared.git
tag: v1.9.7
sparse: ./config/GMAO_Shared.sparse
tag: v1.9.0

GEOS_Util:
local: ./src/Shared/@GMAO_Shared/@GEOS_Util
remote: ../GEOS_Util.git
tag: v2.0.1
tag: v2.0.7
sparse: ./config/GEOS_Util.sparse

MAPL:
local: ./src/Shared/@MAPL
remote: ../MAPL.git
tag: v2.39.1
tag: v2.44.1

GEOSgcm_GridComp:
local: ./src/Components/GEOSldas_GridComp/@GEOSgcm_GridComp
remote: ../GEOSgcm_GridComp.git
tag: v2.5.2
sparse: ./config/GEOSgcm_GridComp_ldas.sparse
tag: v2.1.1

3 changes: 3 additions & 0 deletions config/GEOS_Util.sparse
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
/CMakeLists.txt
/pre/remap_restart
/pre/CMakeLists.txt
8 changes: 8 additions & 0 deletions config/NCEP_Shared.sparse
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
!/*
/NCEP_sp
/NCEP_w3
/NCEP_bufr
/NCEP_bacio
/NCEP_sfcio
/NCEP_sigio
/CMakeLists.txt
75 changes: 74 additions & 1 deletion doc/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,85 @@ this period, LDASsa and GEOSldas development continued in parallel.

In 2019, GEOS LDAS version control transferred from CVS to Git.

This README file contains the history of stable GEOSldas versions ("tags") in Git, followed by older, CVS LDASsa and GEOSldas versions and change logs.
This README file contains the history of stable GEOSldas Releases in Git, followed by older, CVS LDASsa tags and GEOSldas versions and change logs.


Overview of Git Releases:
============================

[v18.0.0](https://github.com/GEOS-ESM/GEOSldas/releases/tag/v18.0.0) - 2024-03-22
------------------------------

- 0-diff vs. v17.13.1 except for MAPL bug fix ([PR #734](https://github.com/GEOS-ESM/GEOSldas/pull/734)).

- Notes:
- Release uses original GEOSldas repository structure. Next release is expected to use a revised repository structure.

- Science changes:
- Added MODIS snow cover fraction assimilation ([PR #512](https://github.com/GEOS-ESM/GEOSldas/pull/512)).
- Added ASCAT soil moisture assimilation ([PR #656](https://github.com/GEOS-ESM/GEOSldas/pull/656), [PR #703](https://github.com/GEOS-ESM/GEOSldas/pull/703), [PR #723](https://github.com/GEOS-ESM/GEOSldas/pull/723), [PR #729](https://github.com/GEOS-ESM/GEOSldas/pull/729)).
- New update_type=13 for ASCAT soil moisture and SMAP brightness temperature assimilation ([PR #703](https://github.com/GEOS-ESM/GEOSldas/pull/703)).
- New update_type=13 replaces update_type=[1,2], which has been disabled.
- Requires ASCAT mask file ([PR #723](https://github.com/GEOS-ESM/GEOSldas/pull/723), [PR #729](https://github.com/GEOS-ESM/GEOSldas/pull/729)).
- Disabled CatchmentCNCLM45 model option (LSM_CHOICE=3) ([PR #707](https://github.com/GEOS-ESM/GEOSldas/pull/707), [GEOSgcm_GridComp PR #900](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/900)).
- Added support for GEOS-IT surface met forcing ([PR #688](https://github.com/GEOS-ESM/GEOSldas/pull/688)).
- Added CATCHMENT_SPINUP mode ([PR #647](https://github.com/GEOS-ESM/GEOSldas/pull/647), [GEOSgcm_GridComp PR #751](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/751)).

- GEOSgcm_GridComp v2.5.2:
- Improved MODIS-based snow albedo (v2) in make_bcs package ([GEOSgcm_GridComp PR #687](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/687)).
- Major source code cleanup:
- Stieglitz snow model ([GEOSgcm_GridComp PR #834](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/834)).
- make_bcs package ([GEOSgcm_GridComp PR #763](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/763), [GEOSgcm_GridComp PR #786](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/786), [GEOSgcm_GridComp PR #846](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/846)).
- coeffsib ([GEOSgcm_GridComp PR #845](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/845)).
- Fixed CDCR2 long_name ([GEOSgcm_GridComp PR #818](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/818)).
- Optional checks of snow states after application of LDAS increments ([GEOSgcm_GridComp PR #834](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/834)).
- Refined check for allowable bcs versions for CatchCN ([GEOSgcm_GridComp PR #882](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/882)).
- Fixed treatment of atmospheric CO2 input file for CatchCN ([PR #663](https://github.com/GEOS-ESM/GEOSldas/pull/663), [GEOSgcm_GridComp PR #771](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/771)).
- Fixed bug when reading vegetation type ity from restart ([GEOSgcm_GridComp PR #757](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/757)).
- Fixed bug to correct vegetation fraction assessment in GetIds_carbon (getids.F90) for CatchCN ([GEOSgcm_GridComp PR #770](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/770)).

- Interface:
- Use boundary conditions inputs in revised directory layout and naming convention ([PR #680](https://github.com/GEOS-ESM/GEOSldas/pull/680)).

- Utilities:
- GMAO_Shared v1.9.7 ([PR #725](https://github.com/GEOS-ESM/GEOSldas/pull/725)).
- GEOS_Util v2.0.7 ([PR #725](https://github.com/GEOS-ESM/GEOSldas/pull/725)).
- Sparse checkout of GEOS_Util ([PR #711](https://github.com/GEOS-ESM/GEOSldas/pull/711)).
- Improved remap_restarts package ([GEOS_Util PR #43](https://github.com/GEOS-ESM/GEOS_Util/pull/19), [GEOS_Util PR #43](https://github.com/GEOS-ESM/GEOS_Util/pull/43), [GEOS_Util PR #53](https://github.com/GEOS-ESM/GEOS_Util/pull/53))
- Added NCEP_Shared v1.3.0 ([PR #656](https://github.com/GEOS-ESM/GEOSldas/pull/656)).

- Infrastructure:
- Updated for NCCS/Discover hardware:
- Added support for SLES15/Milan nodes ([PR #693](https://github.com/GEOS-ESM/GEOSldas/pull/693)).
- Removed support for Haswell nodes ([PR #681](https://github.com/GEOS-ESM/GEOSldas/pull/681)).
- Support for running GEOSldas at the NASA Advanced Supercomputing (NAS) facility ([PR #706](https://github.com/GEOS-ESM/GEOSldas/pull/706)).
- Allow experiment setup from another user's build ([PR #733](https://github.com/GEOS-ESM/GEOSldas/pull/733), [PR #740](https://github.com/GEOS-ESM/GEOSldas/pull/740)).
- ESMA_env v4.23.0, Baselibs v7.16.0 ([PR #681](https://github.com/GEOS-ESM/GEOSldas/pull/681)).
- ESMA_cmake v3.41.0 ([PR #725](https://github.com/GEOS-ESM/GEOSldas/pull/725)).
- MAPL v2.44.1 ([PR #734](https://github.com/GEOS-ESM/GEOSldas/pull/734)).
- CircleCI Orb v2 ([PR #694](https://github.com/GEOS-ESM/GEOSldas/pull/694)).
- Replaced FLAP with fArgParse ([PR #669](https://github.com/GEOS-ESM/GEOSldas/pull/669)).
- Updated versions of GitHub Actions ([PR #739](https://github.com/GEOS-ESM/GEOSldas/pull/739)).


- Documentation:
- Updates and corrections ([PR #728](https://github.com/GEOS-ESM/GEOSldas/pull/728)).

- Bug fixes and other minor changes:
- Fixed bug that degraded simulation when writing (MAPL-binary) instantaneous output with bit shaving ([PR #734](https://github.com/GEOS-ESM/GEOSldas/pull/734)).
- Removed requirement for mwRTM parameter input files ([PR #685](https://github.com/GEOS-ESM/GEOSldas/pull/685)).
- Support for reading corrected precipitation from aggregated daily netcdf files ([PR #718](https://github.com/GEOS-ESM/GEOSldas/pull/718)).
- Updated generate_catchincr_hist.py and sample documents for coupled land-atmosphere data assimilation ([PR #698](https://github.com/GEOS-ESM/GEOSldas/pull/698)).
- Bug fix to avoid NaN for ens std-dev in debug mode ([PR #679](https://github.com/GEOS-ESM/GEOSldas/pull/679)).
- Implementation changes for FFT used in perturbations ([PR #679](https://github.com/GEOS-ESM/GEOSldas/pull/679)).
- Some cleanup of unused variables ([PR #679](https://github.com/GEOS-ESM/GEOSldas/pull/679)).
- Updated met forcing path when coupled with ADAS ([PR #682](https://github.com/GEOS-ESM/GEOSldas/pull/682)).
- Added more export variable definitions to tile_bin2nc4.F90 ([PR #676](https://github.com/GEOS-ESM/GEOSldas/pull/676)).
- Fixed LONG_NAME for longwave radiation variables ([PR #674](https://github.com/GEOS-ESM/GEOSldas/pull/674), [GEOSgcm_GridComp PR #764](https://github.com/GEOS-ESM/GEOSgcm_GridComp/pull/764)).
- Fixed bug in matlab reader MAPL_ReadForcing_fullfile.m ([PR #665](https://github.com/GEOS-ESM/GEOSldas/pull/665)).
- Renamed GEOSldas_GridComp/Shared to GEOSldas_GridComp/LDAS_Shared ([PR #714](https://github.com/GEOS-ESM/GEOSldas/pull/714)).

------------------------------
[v17.13.1](https://github.com/GEOS-ESM/GEOSldas/releases/tag/v17.13.1) - 2023-06-26
------------------------------

Expand Down
Loading

0 comments on commit 2deea0b

Please sign in to comment.