Configuring and Compiling

Configuring GIT

Before downloading any of the ROMS repositories, ensure that your ~/.gitconfig has the appropriate git-lfs configuration for correctly downloading some ECCOFS and roms_test input and observation NetCDF files. Otherwise, the required application input NetCDF files will fail. The Git LFS is a command-line extension and specification for managing large files with Git. A sample of the configuration file looks like this:

   more ~/.gitconf

   [User]
        name = GivenName MiddleName FamilyName
        email = your@email
   [credential]
        helper = cache --timeout=7200
        helper = store --file ~/.my-credentials
   [filter "lfs"]
        clean = git-lfs clean -- %f
        smudge = git-lfs smudge -- %f
        process = git-lfs filter-process
        required = true

Alternatively, you may execute git lfs pull at your location of this repository to download a viable version of the Git LFS files from the remote repository in GitHub. Also, to automatically add the LFS filter to your existing ~/.gitconfig, you could use git lfs install from anywhere in your computer directories.

Compiling ROMS

We highly recommend that users define the ROMS_ROOT_DIR variable in their computer shell logging environment, specifying the location where the User cloned or downloaded the ROMS source code. That is, use the value of pwd from the computer directory where you executed git clone https://github.com/myroms/roms.git:

setenv ROMS_ROOT_DIR  MyDownlodLocationDirectory

The GNU make and CMake build scripts will use this environmental variable when compiling roms or any of the Test Cases without the need to customize the location of the ROMS source code.

Users must study the build scripts to learn how to customize them. Notice that there are a few options for their execution. To compile a regular nonlinear kernel ROMS configuration, as the one Forward subdirectory, we have:

Usage:

   ./build_roms.sh [options]
or
   ./cbuild_roms.sh [options]

Options:

    -b          Compile a specific ROMS GitHub branch

                  build_roms.sh -j 10 -b feature/kernel

    -g          Compile with debug flag (slower code)

                  build_roms.sh -g -j 10

    -j [N]      Compile in parallel using N CPUs,
                  omit argument for all available CPUs

    -pio        Compile with the PIO (Parallel I/O) NetCDF library
                  Otherwise, it used the standard NetCDF library (slower)

                  build_roms.sh -pio -j 10

    -p macro    Prints any Makefile macro value. For example,

                  build_roms.sh -p FFLAGS

    -noclean    Do not clean already compiled objects

Check ROMS repository for available branches other than main and develop. Note that the feature branches are under development and intended for advanced users, superusers, and beta testers. Regular and novice users must use the default develop branch. Alternatively, after downloading the code, type:

git branch -a

If using the -b option, the build script will clone the ROMS source code from GitHub into your project sub-directory src, for example:

git clone https://www.github.com/myroms/roms.git src
cd src
git checkout feature/kernel

The CMake build scripts (cbuild_roms.csh and cbuild_roms.sh) are more complicated and are intended for advanced users.

Compiling ROMS Split 4D-Var

The ECCOFS application uses a ROMS split RBL4D-Var mixed-resolution algorithm for computer efficiency. It is configured with ECCOFS 3 km and 6 km grids. In the mixed-resolution scheme, the 4D-Var outer loops (Background and Analysis phases) are run at 3km grid (1667x1443x50) resolution. In contrast, the inner loops (Increment phase) are run at a coarser 6km grid (834x722x50) resolution to accelerate the computations. The coarse grid 4D-Var increments are interpolated to the finer grid in the Analysis phase using the roms_interp and roms2roms CLASS objects.

In the Background phase, the coarse grid trajectory needed to linearize the tangent linear (TLM) and adjoint (ADM) model kernels is extracted by a decimation of the 3km grid solution using the CPP option GRID_EXTRACT. Grid decimation is only possible if the parent grid (ECCOFS3km, Lm=1665 and Mm=1441) size satisfies MOD(Lm+1, 2) = 0 and MOD(Mm+1, 2) = 0.

The mixed-resolution split 4d-Var scheme requires a different build script (build_split.csh, build_split.sh, cbuild_split.csh, or cbuild_split.sh) with slightly different options than shown above:

Usage:

    ./build_split.sh [options]
or
   ./cbuild_roms.sh [options]

Options:

    -da         Compile data assimilation executable (inner loops)

                  build_split.sh -da -j 10

    -nl         Compile nonlinear trajectory executable (outer loops)

                  build_split.sh -nl -j 10

    -b          Compile a specific ROMS GitHub branch

                  build_split.sh -nl -j 10 -b feature/kernel

    -g          Compile with debug flag (slower code)

                  build_split.sh -g -nl -j 10

    -j [N]      Compile in parallel using N CPUs,
                  omit argument for all available CPUs

    -p macro    Prints any Makefile macro value. For example,

                  build_split.sh -p FFLAGS

    -pio        Compile with the PIO (Parallel I/O) NetCDF library
                  Otherwise, it used the standard NetCDF library (slower)

                  build_split.sh -nl -pio -j 10

    -noclean    Do not clean already compiled objects

How to Compile ROMS Split 4D-Var

To compile ROMS mised-resolution, splitdata assimilation executables for 4D-Var outer and inner loops, use:

  build_split.sh -nl -pio -j 10                 creates executable romsM_nl
  build_split.sh -da -pio -j 10                 creates executable romsM_da

Notice it asks to compile with the PIO-NetCDF library (-pio option) to speed up the computations. Also, the 4D-Var split scheme uses two different executables for ROMS:

    romsM_nl       Nonlinear driver for RBL4D-Var Background and Analysis phases, outer loops

    romsM_da       Data assimilation driver for RBL4D-Var Increment phase, inner loops

Please review the build script because it contains CPP options for each executable. Also, it sets standard CPP options for both executables. This strategy is preferable to having a ROMS header file for the 3km and 6km grids.

To submit the job on N CPUs via SLURM or not, use:

  sbatch submit_mixres_rbl4dvar.sh        or
  submit_mixres_rbl4dvar.sh > & log &

Users may modify the number of processors to use in the submit_mixres_rbl4dvar.sh script.