Configuring and Compiling
Before downloading any of the ROMS repositories, ensure that your ~/.gitconfig has the appropriate git-lfs configuration for correctly downloading some ECCOFS and roms_test input and observation NetCDF files. Otherwise, the required application input NetCDF files will fail. The Git LFS is a command-line extension and specification for managing large files with Git. A sample of the configuration file looks like this:
more ~/.gitconf
[User]
name = GivenName MiddleName FamilyName
email = your@email
[credential]
helper = cache --timeout=7200
helper = store --file ~/.my-credentials
[filter "lfs"]
clean = git-lfs clean -- %f
smudge = git-lfs smudge -- %f
process = git-lfs filter-process
required = trueAlternatively, you may execute git lfs pull at your location of this repository to download a viable version of the Git LFS files from the remote repository in GitHub. Also, to automatically add the LFS filter to your existing ~/.gitconfig, you could use git lfs install from anywhere in your computer directories.
We highly recommend that users define the ROMS_ROOT_DIR variable in their computer shell logging environment, specifying the location where the User cloned or downloaded the ROMS source code. That is, use the value of pwd from the computer directory where you executed git clone https://github.com/myroms/roms.git:
setenv ROMS_ROOT_DIR MyDownlodLocationDirectory
The GNU make and CMake build scripts will use this environmental variable when compiling roms or any of the Test Cases without the need to customize the location of the ROMS source code.
Users must study the build scripts to learn how to customize them. Notice that there are a few options for their execution:
Usage:
./build_roms.sh [options]
or
./cbuild_roms.sh [options]
Options:
-b Compile a specific ROMS GitHub branch
build_roms.sh -j 10 -b feature/kernel
-g Compile with debug flag (slower code)
build_roms.sh -g -j 10
-j [N] Compile in parallel using N CPUs,
omit argument for all available CPUs
-pio Compile with the PIO (Parallel I/O) NetCDF library
Otherwise, it used the standard NetCDF library (slower)
build_roms.sh -pio -j 10
-p macro Prints any Makefile macro value. For example,
build_roms.sh -p FFLAGS
-noclean Do not clean already compiled objectsCheck ROMS repository for available branches other than main and develop. Note that the feature branches are under development and intended for advanced users, superusers, and beta testers. Regular and novice users must use the default develop branch. Alternatively, after downloading the code, type:
git branch -aIf using the -b option, the build script will clone the ROMS source code from GitHub into your project sub-directory src, for example:
git clone https://www.github.com/myroms/roms.git src
cd src
git checkout feature/kernelThe CMake build scripts (cbuild_roms.csh and cbuild_roms.sh) are more complicated and are intended for advanced users.
The CMake build scripts (cbuild_roms.csh and cbuild_roms.sh) are more complicated and are intended for advanced users.
The ECCOFS application uses a ROMS split RBL4D-Var mixed-resolution algorithm for computer efficiency. It is configured with ECCOFS 3 km and 6 km grids. In the mixed-resolution scheme, the 4D-Var outer loops (Background and Analysis phases) are run at 3km grid (1667x1443x50) resolution. In contrast, the inner loops (Increment phase) are run at a coarser 6km grid (834x722x50) resolution to accelerate the computations. The coarse grid 4D-Var increments are interpolated to the finer grid in the Analysis phase using the roms_interp and roms2roms CLASS objects.
In the Background phase, the coarse grid trajectory needed to linearize the tangent linear (TLM) and adjoint (ADM) model kernels is extracted by a decimation of the 3km grid solution using the CPP option GRID_EXTRACT. Grid decimation is only possible if the parent grid (ECCOFS3km, Lm=1665 and Mm=1441) size satisfies MOD(Lm+1, 2) = 0 and MOD(Mm+1, 2) = 0.
The mixed-resolution split 4d-Var scheme requires a different build script (build_split.csh, build_split.sh, cbuild_split.csh, or cbuild_split.sh) with slightly different options than shown above:
Usage:
./build_split.sh [options]
or
./cbuild_roms.sh [options]
Options:
-da Compile data assimilation executable (inner loops)
build_split.sh -da -j 10
-nl Compile nonlinear trajectory executable (outer loops)
build_split.sh -nl -j 10
-b Compile a specific ROMS GitHub branch
build_split.sh -nl -j 10 -b feature/kernel
-g Compile with debug flag (slower code)
build_split.sh -g -nl -j 10
-j [N] Compile in parallel using N CPUs,
omit argument for all available CPUs
-p macro Prints any Makefile macro value. For example,
build_split.sh -p FFLAGS
-pio Compile with the PIO (Parallel I/O) NetCDF library
Otherwise, it used the standard NetCDF library (slower)
build_split.sh -nl -pio -j 10
-noclean Do not clean already compiled objectsTo compile ROMS data assimilation executables for 4D-Var outer and inner loops, use:
build_split.sh -nl -pio -j 10 creates executable romsM_nl
build_split.sh -da -pio -j 10 creates executable romsM_daNotice it asks to compile with the PIO-NetCDF library (-pio option) to speed up the computations. Also, the 4D-Var split scheme uses two different executables for ROMS:
romsM_nl Nonlinear driver for RBL4D-Var Background and Analysis phases, outer loops
romsM_da Data assimilation driver for RBL4D-Var Increment phase, inner loopsPlease review the build script because it contains CPP options for each executable. Also, it sets standard CPP options for both executables. This strategy is preferable to having a ROMS header file for the 3km and 6km grids.
To submit the job on 12 CPUs via SLURM or not, use:
sbatch submit_mixres_rbl4dvar.sh or
submit_mixres_rbl4dvar.sh > & log &Users may modify the number of processors to use in the submit_mixres_rbl4dvar.sh script.