Skip to content

Input Timeseries Data

Joe-Wagstaff edited this page May 15, 2025 · 9 revisions

To run crspy-lite, you need timeseries data of your sensors "counting rate". You also need timeseries data of several meteorological variables for the processing, in order to form correction factors, calibrate the sensor and perform quality control. See below for guidance on the formating of this input data:

Required Columns

The table below summarises the required variables that comprise the input data, denoting the expected column name and unit. Please note that all columns must be the same length. Any gaps in the data should be filled with 'nan' values.

NAME UNITS DESCRIPTION
DATETIME Datetime Datetime of the observation. Format: "yyyy-mm-dd hh:mm:ss"
MOD Count Moderated neutron count for time interval.
PRESS mb Pressure sensor measurement.
TEMP Celsius External Temperature at the site. This would be an external reading.
E_RH (%) External Relative Humidity at the site.
I_RH (%) Relative humidity inside the sensor box.
BATT Volts Voltage of the CRNS battery.
RAIN mm Rainfall at the site.

Automatic formatting

Crspy-lite will automatically format your input data in several ways. This includes:

  • Detecting the delimiter of your input data and reading it correspondingly.
  • Resampling your input data to an hourly temporal resolution.
  • Fixing the dataframe to values "on the hour" (e.g. 10:00 instead of 10:13).
  • Flip the dataframe if the dates are the wrong way around.
  • Remove whitespace in columns and drop any empty ones (e.g. columns with only NaN values in them).

Structuring the input data

Crspy-lite is dependent on the user to structure their input data correctly and upload it via either a filepath or a url to the config.json file. Check the table below for a formatting checklist:

Filepath URL
Included all required columns Included all required columns
Correct column names Correct column names
File is .txt or .csv with a consistent delimiter Consistent delimiter (tab or comma recommended)
Ensure the filepath sting is correctly formatted Ensure the link is publically accessible (no authentication required)
Ensure the link points directly to the file of the data

For clarification of these formatting requirements click on the links for examples of input timeseries data via a filepath and a URL respectively. You can also see the example of the config file to check how the structure of the filepath string.

Clone this wiki locally