Redesigning JULES tile input

Background

JULES uses the tile approach for expressing subgrid heterogeneity. In short, this means that a small number of surface types are described (eg broadleaf tree, shrub, bare soil etc) and each gridbox has a fractional occupation of these, adding up to unity. This approach is computationally efficient and offers various advantages over the alternative 'mosaic' approach.

The data required to set JULES up in this regard comes from two sources, a fraction ancillary that describes the fractional coverage for each tile type, and a parameters namelist that describes the values to be used in the various processes represented by the model.

The problem

The way in which these two inputs are used, and how tiling is implemented throughout JULES is currently unsafe, and difficult to work with. Problems include:

• Dump and diagnostic files are not self-describing. It is only working knowledge of the model configurations (in standalone) that allow the user to identify what the surface tiles are. There is nothing to identify what surface types are present in the file metadata.
• The namelits are very long (jules_pftparm is 117 lines) comma separated lists, making them very tricky to work with.
  • To add a new tile one typically resorts to copy/pasting into a spreadsheet to allow addition of interstitial columns or writing processing scripts to generate them.
• There is no connection to ensure that the configuration ancillaries, dumps and the parameter namelists are compatible.
• There are historical instances of hard-coding tile indices (notably UKCA) which compromised the separation of code and configuration.
• Upgrade macros to add new parameters are brittle, only allowing upgrade of a limited range of suites based on known configurations identified by different values of npft.
  • This assumed incorrectly that all configurations with a common value of npft were the same e.g. two configurations with npft = 9 exist (ncpft = 0 and ncpft = 4) leading to many incomplete macros not taking account of the configurations with crops.
• The traceability of parameters namelists is challenging.
  • For example, an accidental change to some parameter could easily be progagated around the entire community through sharing of suites.
• Managing suites to contain different optional surface configurations requires the whole namelist to be repeated rather than adding an additional tile in the opt file, making it challenging to upgrade one parameter value and ensurating it's propagated through the optional configurations.

The initial proposal

JULES could read these values from a new file, the tile parameter file, with a new input along the lines of:

ntile = 9
fraction_file = /path/to/frac/file
parameter_file = /path/to/parameter/file
tile_short_names = 'BLT', 'NLT', ..., 'soil',...

Both the fraction and parameter files, which could be version controlled, would have to have entries that match the short names. This would be checked at runtime. The parameter file would contain a number of entries along the line of

short_name = 'BLT'
long_name = 'Broadleaf Tree GL7'
type = vegetation
special_id = 101
param1 = 0.123
...
paramn = 1.365

Special ID is intended to provide a means of doing very specific switching in the code. A tile type could have multiple special IDs.

Essentially, we are making a new ancillary file.

Known challenges

• Tooling would need to be provided to extract namelists into external files
• Hard coding of tile index or tile ID, notably UKCA and MORUSES (these are historical and to best knowledge have been eradicated)
• A workflow for publishing and storing the tile parameter files
• Choosing a format that works for all parent models (UM, JULES, LIS etc)
• Works for CABLE (should be fine)

Working towards tile parameter files

Some of the problems previously listed can be addressed by the existing duplicate namelist metadata functionality. The LFRic core documentation gives a good description of this functionality, however, as JULES does not use the configurator tool, the way the duplicate namelist is handled in the JULES code is different. By splitting the tile input namelists i.e. jules_pftparm, jules_nvegparm, jules_triffid, jules_red into one namelist per tile, identified by the instance_key_member, the namelist management becomes much more straightforward. For example:

1. Additional tiles can be added to a configuration by copying the exising namelist with a different value of the instance key, with the appropriate parameter values either in the main configuration file or in an opt file. Existing namelists do not need to be changed.
2. An existing tile can be easily identified by its instance key to be deleted or replaced by another.
3. Upgrade macros can add new items or replace existing values by specifying the instance key rather than resorting to proxies like npft or nnvg meaning a single upgrade macro will correctly upgrade any configuration.

jules_pftparm as a duplicate namelist

MetOffice/jules#115 defines jules_pftparm as a duplicate namelist, allowing multiple instances, using !instance_key_member=pft_name_io:

[namelist:jules_pftparm]
compulsory=true
description=This section is organised into one panel per PFT using the instance key "pft_name_io".
duplicate=true
!instance_key_member=pft_name_io

[namelist:jules_pftparm=pft_name_io]
compulsory=true
description=Unique descriptor for plant functional type.
    	   =Allows multiple instances of a namelist; one per PFT.
!kind=default
sort-key=Panel-H01
!type=character
values='brd_leaf', 'brd_leaf_dec', 'brd_leaf_eg_temp', 'brd_leaf_eg_trop',
	  ='c3_crop', 'c3_grass', 'c3_irrig', 'c3_pasture', 'c4_crop', 'c4_grass',
	  ='c4_irrig', 'c4_pasture', 'ndl_leaf', 'ndl_leaf_dec', 'ndl_leaf_eg',
	  ='shrub', 'shrub_dec', 'shrub_eg', 'usr_type#1'

Where the allowed values for pft_name_io are based on defined varieties in jules_surface_types (example vn8.2).

After the upgrade macro is run to convert jules_pftparm to a duplicate namelist, each namelist appears as a separate panel in the Rose GUI.

Subsequent upgrade macros to add new items, or change the values of existing items, become far simpler and use the instance key pft_name_io to specify what value an item should be added with or changed to for each surface type. The following is an example of an upgrade macro that changes the values of the existing item a_wl_io as an illustration:

class vn82_t115_example(MacroUpgrade):

	"""Upgrade macro from JULES by Maggie Hendry"""

	BEFORE_TAG = "vn8.2_t115"
	AFTER_TAG = "vn8.2_t115_example"

	def upgrade(self, config, meta_config=None):
    	"""Upgrade a JULES runtime app configuration."""

    	RMDI = str(-(2**30))
    	for obj in config.get_value():
        	if re.search(r'namelist:jules_pftparm', obj):
         		pft_name_io = self.get_setting_value(config,[obj,"pft_name_io"])
            	if pft_name_io in ["'brd_leaf'", "'ndl_leaf'", "'ndl_leaf_eg'"]:
            		a_wl_io = "0.65"
    	    	elif pft_name_io in ["'brd_leaf_dec'", "'brd_leaf_eg_temp'"]:
            		a_wl_io = "0.78"
            	elif pft_name_io in ["'brd_leaf_eg_trop'"]:
           			a_wl_io = "0.845"
            	elif "c3" in pft_name_io or "c4" in pft_name_io:
         			a_wl_io = "0.005"
            	elif pft_name_io in ["'ndl_leaf_dec'"]:
					a_wl_io = "0.8"
            	elif pft_name_io in ["'shrub'"]:
            		a_wl_io = "0.10"
            	elif pft_name_io in ["'shrub_dec'", "'shrub_eg'"]:
        			a_wl_io = "0.13"
            	else:
            	    a_wl_io = RMDI
            	    msg = f"{pft_name_io} not found, a_wl_io set to RMDI."
            	    self.add_report(info=msg, is_warning=True)
            	self.change_setting_value(config,[obj,"a_wl_io"],a_wl_io)

    return config, self.reports

In reality this upgrade macro should have resulted in a null change as the values were changed to their existing values, however when it was tested it highlighted that existing configurations in the Rose stem tests had been historically incorrectly identified and previous upgrade macros had been incomplete with two configurations existing with npft=9, those with ncpft=0 and ncpft=4. This distinction has not been made in 17/21 existing upgrade macros. Before this work can progress, this error needs to be rectified by MetOffice/jules#136.