Skip to content

Metadata Input

Jump to bottom
Joe-Wagstaff edited this page Jun 6, 2025 · 17 revisions

Metadata

To run crspy-lite (and process CRNS data more broadly) a collection of site descriptors and variables are required. In crspy-lite these are requested to be inputted by the user when the tool is first run. After this they are coverted to and stored in a config.json file, which can be found in "\your-folder\inputs\config.json". Note that not all variables are required. For example some can be calculated within crspy-lite or are included to define limits for different variables: such as rainfall.

Please use the table below to view what these variables are, what they do and if they are required; and/or click here for a full example of the config.json file.

NAME UNITS DESCRIPTION REQUIRED
COUNTRY - Country code for location of site Yes
SITENUM - Assigned number for site Yes
SITENAME - Name of site Yes
INSTALL_DATE - Date of site installation No
LATITUDE degrees Latitude of site. Values in range -90 to 90 Yes
LONGITUDE degrees Longitude of site. Values in range from 0 to 360 Yes
ELEV m Elevation of site above sea level Yes
TIMEZONE - Timezone of the site No
RC gv Cutoff Rigidity of the site No - crspy-lite can calculate this
LW % (0:1) Percent Lattice Water (as a number between 0 and 1) Yes
SOC % (0:1) Percent Soil Organic Carbon (as a number between 0 and 1) Yes
BD g/cm3 Bulk Density Yes
BETA_COEFF - Store of the calculated beta coefficient for each individual site No - crspy-lite can calculate this
REFERENCE_PRESS mb Reference pressure Yes
N0 - Theoetical neutron count for absolute dry soil conditions No - but must otherwise supply a calibration dataset so crspy-lite can calculate this
PRECIP_MAX mm Maximum value of hourly precipitation. Values above this are removed No - optional limit
SM_MAX cm3/cm3 Maximum value of hourly soil moisture (i.e. porosity). Values above this are removed No - optional limit
FIELD_CAPACITY cm3/cm3 The amount of soil moisture or water content held in the soil after excess water has drained away No - optional visualisation parameter
WILTING_POINT cm3/cm3 The amount of water that is held so tightly by the soil matrix that roots cannot absorb it and plants will wilt No - optional visualisation parameter

Additional Info

Input Time Series Data

During the first time you run crspy-lite you will also be prompted to input time series data, known as "raw data". Crspy-lite can accomodate .txt and .csv files by reading a filepath, as well as data from online sources by reading a URL. Therefore you must either provide a filepath or a URL of your input data. You can also input a filepath of calibration data, if you wish to use crspy-lite to calibrate your site. The table below summarises the three inputs:

NAME DESCRIPTION REQUIRED
RAW_DATA_FILEPATH File path of input timeseries data from the CRNS Must provide either a RAW_DATA_FILEPATH OR RAW_DATA_URL
RAW_DATA_URL URL of input timeseries data from the CRNS Must provide either a RAW_DATA_FILEPATH OR RAW_DATA_URL
CALIB_DATA_FILEPATH File path of Calibration data Required if no N0 number is given (see table above)

Click on the links for further details on the required format of the Input Timeseries Data of the Calibration Data.

Modifying the config file

The config.json file is effectively the base of operations in crspy-lite. It stores all the site metadata, general processing parameters and filepaths. There is no user expectation to modify it, however for more experienced users or users looking to understand more about the processing, the config.json file provides a quick and useful way to achieve these aims. Below is a brief guide on how you can modify it:

Filepaths

If you want to change your input data or calibration data, simply open the config.json file, and locate the relevent filepath (it will be located under the "filepaths" section). Then edit and save the file to apply the changes. Next time you run crspy-lite you'll have soil moisture data and figures for another site! See below for an example:

"raw_data_filepath": "C:\\Users\\YourName\\Documents\\your-folder\\raw_data.txt",
"raw_data_url": null,
"calib_data_filepath": "C:\\Users\\YourName\\Documents\\your-folder\\calibration_data.csv",

Note that any empty parameter or variable in the config.json file is listed as "null"!

Processing Methods

With CRNS being an evolving discipline, new data processing methodologies are frequently researched and developed. Crspy-lite has been developed to easily accomodate new developments that can be toggled by the user, so that the tool can be kept up-to-date and relevent to the community. Two new methods are currently included in crspy-lite. These can be changed by opening the config.json file, accessing the "general" section, and modifying the parameters as shown below:

1. Soil moisture processing equation: : defaulted to the Desilets et al., (2010) approach:

"sm_calc_method": "desilets",

but can be changed to the Köhli et al., (2021) approach:

"sm_calc_method": "kohli",

2. Incoming Intensity Correction Factor: : defaulted to the McJannet and Desilets, (2023) approach:

"intensity_correction_method": "mcjannet",

but can be changed to the Hawdon et al., (2014) approach:

"intensity_correction_method": "hawdon",

Visualisation

The config.json file can also be used to improve the quality of the figures produced by crspy-lite. This is by:

  1. Imposing limits to remove extreme values in the data (that have not already been removed by crspy-lite's internal quality control checks).

  2. Enabling a comparison to relevant variables. E.g. The field capacity and wilting point of a site so you can observe when soil moisture is too great or too low for optimum plant growth and crop yield.

Open, edit and save the config.json file in the same manner as before; modifying the variables as exemplified below:

"precip_max": null,
"sm_max": 0.5472,
"field_capacity": 0.32,
"wilting_point": 0.095,

Clone this wiki locally