-
Notifications
You must be signed in to change notification settings - Fork 0
Input Timeseries Data
To run crspy-lite, you need timeseries data of your sensors "counting rate". You also need timeseries data of several meteorological variables for the processing, in order to form correction factors, calibrate the sensor and perform quality control. See below for guidance on the formating of this input data:
The table below summarises the required variables that comprise the input data, denoting the expected column name and unit. Please note that all columns must be the same length. Any gaps in the data should be filled with 'nan' values.
| NAME | UNITS | DESCRIPTION |
|---|---|---|
| DATETIME | Datetime | Datetime of the observation. Format: "yyyy-mm-dd hh:mm:ss" |
| MOD | Count | Moderated neutron count for time interval. |
| PRESS | mb | Pressure sensor measurement. |
| TEMP | Celsius | External Temperature at the site. This would be an external reading. |
| E_RH | (%) | External Relative Humidity at the site. |
| I_RH | (%) | Relative humidity inside the sensor box. |
| BATT | Volts | Voltage of the CRNS battery. |
| RAIN | mm | Rainfall at the site. |
Crspy-lite will automatically format your input data in several ways. This includes:
- Detecting the delimiter of your input data and reading it correspondingly.
- Resampling your input data to an hourly temporal resolution.
- Fixing the dataframe to values "on the hour" (e.g. 10:00 instead of 10:13).
- Flipping the dataframe if the dates are the wrong way around.
- Removing whitespace in columns and drop any empty ones (e.g. columns with only NaN values in them).
Crspy-lite is dependent on the user to structure their input data correctly and upload it via either a filepath or a url to the config.json file. Check the table below for a formatting checklist:
| Filepath | URL |
|---|---|
| Included all required columns | Included all required columns |
| Correct column names | Correct column names |
File is .txt or .csv with a consistent delimiter
|
Consistent delimiter (tab or comma recommended) |
| Ensure the filepath sting is correctly formatted | Ensure the link is publically accessible (no authentication required) |
| Ensure the link points directly to the file of the data |
For clarification of these formatting requirements click on the links for examples of input timeseries data via a filepath and a URL respectively. You can also see the example of the config file to check how the structure of the filepath string.