Skip to content

7. Running the models

jrulent edited this page Jun 12, 2023 · 1 revision

Running SRIL34 physics

To run the model, first copy you NEMO and XIOS executables in the directory you want to run the model from, for eg:

cp /work/n01/n01/<user>/<path_to_nemo_compile>/NEMO_4.0.4/cfgs/<configuration_name>/BLD/bin/nemo.exe . 
cp /work/n01/n01/<user>/<path_to_xios_compile>/xios-2.5/bin/xios_server.exe .

Now rename the NEMO executable, as archer2 has issues reading the *.exe appendix. IMPORTANT: only rename the executable that you copied in your running directory!

cd /<run_directory> 
mv nemo.exe nemo

To run the model you must have your executables nemo and xios.exe, the domain_cfg.nc, the coordinates.nc , in your running directory, together with the *.xml file (that should come up when you compile nemo), your namelist_ref and your namelist_cfg, for eg :

-rwxr-xr-x 1 jrule n01     109310 Nov  4  2021 namelist_ref
-rwxr-xr-x 1 jrule n01       1702 Nov  4  2021 context_nemo.xml
-rwxr-xr-x 1 jrule n01      12948 Nov  4  2021 domain_def_nemo.xml
-rwxr-xr-x 1 jrule n01     110461 Nov  4  2021 field_def_nemo-oce.xml
-rwxr-xr-x 1 jrule n01       4420 Nov  4  2021 file_def_nemo-oce.xml
-rwxr-xr-x 1 jrule n01       6922 Nov  4  2021 grid_def_nemo.xml
-rwxr-xr-x 1 jrule n01       1156 Nov  4  2021 iodef.xml
-rwxr-xr-x 1 jrule n01   22089760 Nov  3  2021 nemo
-rwxr-xr-x 1 jrule n01   12331336 Nov  3  2021 xios_server.exe
-rwxr-xr-x 1 jrule n01   20006632 Nov  2  2021 coordinates.nc
-rwxr-xr-x 1 jrule n01  642521292 Nov  2  2021 domain_cfg.nc
-rwxr-xr-x 1 jrule n01      62306 Dec 22  2021 namelist_cfg

Amend the namelist_cfg with the paths to your boundary conditions, and restart files (or initial conditions). Also, set the parameters of the run.

Now all is set-up, you can run the model using a slurm script. To create the slurm script, see archer's instruction. An example of what you might run on your command line (changing the n01_SANH to your account name) is:

/work/n01/shared/nemo/mkslurm_hetjob -S 2 -s 8 -m 2 -C 75 -g 4 -N 128 -t 00:10:00 -a n01-SANH -j TEST  > TEST.slurm

to run it use:

sbatch TEST.slurm 

Running SRIL34_FABM

Generate tracer scenarios

Daily

To generate the passive tracer release scenarios use the script create_scenario.python. This will generate a NetCDF file with daily information on the tracer release (in this example urea) which will be read in by the model.

To generate your scenario, activate your conda environment first (eg. nrct_env_py3). On livljobs, use:

module load anaconda
conda activate nrct_env_py3

Amend the create_scenario.python to set up your scenario. The script is set to generate a file starting from 2021-01-01 00:00:00 UTC. From this date, you choose the day in which the tracers start being release day0s, the duration of the leak in number of days leak_durations, the amount of tracer being released total_tonnes and the conversion factor to from tonnes to mmol (eg. urea). Also choose the coordinates of the tracer release location (based on the SRIL34 domain), and chose the output file name for you release scenario.

NOTE: the conversion factor will change depending on the molar mass of the chemical you are simulating. For urea, this is 60.06g/mol)

    ntracers=1                          # number of tracers to generate
    day0s=array([144],dtype=int)        # first day of leak for each tracer (144=25 May)
    leak_durations=array([1],dtype=int)  # duration of the leak for each tracer in days
    total_tonnes=array([1100.])          # total mass in tonnes for each of the tracer
    conversion_factors=array([16650016.65]) # tonnes to mmol (=1e9/60.06) - Urea

    Yindex=array([153],dtype=int)       # Lat index per each tracer leak
    Xindex=array([102],dtype=int)       # Lon index per each tracer leak

    name='Scenario_urea1100_leak1day.nc'

Once your scenario is ready, run the script with:

python create_scenario.python

You should now have a 365 days scenarios file. Upload this in your running directory (on Archer2). The file is read in by the model through the fabm_input.nml; make sure you have the right Scenario name here before running the model. Also, as your file is daily, make sure the frequency is of 24 hours.


!              !  file name   ! frequency (hours) ! variable  ! time interp. !  clim  ! 'yearly'/ ! weights  ! rotation ! land/sea mask !
!              !              !  (if <0  months)  !   name    !   (logical)  !  (T/F) ! 'monthly' ! filename ! pairing  ! filename      !
&variable
   name='T1_flux/flux'
   sn=         'Scenario_urea1100_leak1day.nc',24,'T1_flux', .false.,  .true.,   'yearly',     '',         '',        ''

Hourly

to simulate tracer releases that occur in less than a daily, you will need to generate and hourly tracer scenario release. To do this use the script create_scenario_hourly.python. This is essentially the same as create_scenario.python, but with some changes to express everything in hours, rather than days. This means that when you change the script to set up your scenario, you will need to set the start time of you tracer release, and the duration of the leak in hours rather than days. eg, to set a scenario where the passive tracers are released in 6 hours:

    ntracers=1                          # number of tracers to generate
    day0s=array([144*24],dtype=int)        # first day of leak for each tracer (144=25 May)
    leak_durations=array([6],dtype=int)  # duration of the leak for each tracer in hours
    total_tonnes=array([1100.])          # total mass in tonnes for each of the tracer
    conversion_factors=array([16650016.65]) # tonnes to mmol (=1e9/60.06) - Urea

    Yindex=array([153],dtype=int)       # Lat index per each tracer leak
    Xindex=array([102],dtype=int)       # Lon index per each tracer leak

    name='Scenario_urea1100_leak6hour.nc'

Also, before running the model, make sure that you amend frequency in the fabm_input.nml to read in the scenario in hours rather than days.


!              !  file name   ! frequency (hours) ! variable  ! time interp. !  clim  ! 'yearly'/ ! weights  ! rotation ! land/sea mask !
!              !              !  (if <0  months)  !   name    !   (logical)  !  (T/F) ! 'monthly' ! filename ! pairing  ! filename      !
&variable
   name='T1_flux/flux'
   sn=         'Scenario_urea1100_leak6hour.nc',1,'T1_flux', .false.,  .true.,   'yearly',     '',         '',        ''

Physics and tracer namelists

To run the SRIL34-FABM coupled configuration, the first step are the same as for the physics only (described above). You need to copy the nemo and xios executable in your running directory, and change the physics namelist_cfg in the same way you would change the physics only model.

You will need to make sure that your SANH_FABM configuration is using the right keys. To check, open the /cfgs/SANH_FABM/cpp_SANH_FABM.fcm file (saved where you compiled nemo) with vi cpp_SANH_FABM.fcm. They keys should be:

 bld::tool::fppkeys   key_mpp_mpi key_vectopt_loop key_nosignedzero key_iomput key_top key_fabm

if this is not the case, add them to the file.

You will need extra *.xml files your directory compared to the phisics only. This is a list of what your .xml should look like:

-rwxr-xr-x 1 jrule n01   1912 Apr 28 15:49 context_nemo.xml
-rwxr-xr-x 1 jrule n01  12948 Apr 28 15:49 domain_def_nemo.xml
-rwxr-xr-x 1 jrule n01    309 May  4 13:52 field_def_custom.xml
-rwxr-xr-x 1 jrule n01   3787 Sep  1 12:46 field_def_fabm.xml
-rwxr-xr-x 1 jrule n01 110461 Apr 28 15:49 field_def_nemo-oce.xml
-rwxr-xr-x 1 jrule n01  29695 Apr 28 15:49 field_def_nemo-pisces.xml
-rwxr-xr-x 1 jrule n01   5319 Jul 22 10:46 file_def_nemo_1d_setting.xml
-rwxr-xr-x 1 jrule n01   5319 Jul 22 10:50 file_def_nemo_1h_setting.xml
-rwxr-xr-x 1 jrule n01   5319 Aug  2 15:18 file_def_nemo.xml
-rwxr-xr-x 1 jrule n01   6922 Apr 28 15:49 grid_def_nemo.xml
-rwxr-xr-x 1 jrule n01   1156 May  4 13:52 iodef.xml

you will also need a namelist_top_ref and namelist_top_cfg to control the tracers. Before running, set the run information in the namelist_top_cfg. The tracer scenarios are not defined in the namelist file. When running the model will automatically look for a NetCDF file defining the scenario, so make sure you also have this file (eg: Scenario_urea1100_leak1day.nc) in the running directory.

With all files in place you can run the model using a slurm script, eg:

sbatch TEST.slurm