- Import ocean vertical profiles (SeaBird CTD files)
- Apply quality control (flags)
- Reformat to MedAtlas format or netCDF-CF-SDN Complaint
- Create metadata for each profile
- Plot stations
- Show warnings and helpful info on the screen
The CTD is a cluster of sensors which measure conductivity (to derive salinity), temperature, and pressure. Other sensors may be added to the cluster, including some that measure chemical or biological parameters, such as dissolved oxygen and chlorophyll fluorescence. The CTD cast collects high-resolution data that upon retrieval is downloaded to a computer and specialized software is used to graph and interpret data. However, as stated by the SeaDataNet consortium, many times the collected data are neither easily accessible nor standardized. They are not always validated and their security and availability have to be insured in the future.
Hundreds of CTD profiles are recorded by Instituto Español de Oceanografía each year and they are routinely incorporated to the SeaDataNet infrastructure network, which in a unique virtual data management system provide integrated data sets of standardized quality on-line.
Thus, each individual record coming from a CTD instrument must be properly flagged to point how confidence is (Quality Flag Scale), data stored in files according to common data transport formats and standard vocabularies and associated metadata created to allow the dataset to be findable and interoperable (Common Data Index).
To achieve this goal, SeaDataNet provides some tools to prepare XML metadata files (MIKADO), convert from any type of ASCII format to the SeaDataNet ODV and Medatlas ASCII formats as well as the SeaDataNet NetCDF format (NEMO) and check the compliance of a file (OCTOPUS).
All these tasks are time-consuming and require human interaction that can lead to errors or lack of uniformity in data. Taking into account that CTD data usually follows the same format and involves similar processing, software has been developed to perform all these tasks straightforward. Technically, the software is written in Python and there is also a Flask version to serve the application through web. To learn about the deployment of this application in a web server, please, read this page.
You need to download and install all components in a virtual environment. Once the environment is active, run in your console:
python app.pyopen your browser and verify that the application is up and running on port 5000
127.0.0.1:5000Download and install all components in a virtual environment. The main file is checking.py. You need to make some changes before execute it:
- Turn flash_messages variable in config.py.
- Set your path with your files in config.py. This is the route of your xml file and your cnv files. No other kind of files can coexist in this folder. In this case, the xml is the Cruise Summary Report (CSR) file.
- Uncomment the last block of lines at the end of this file.
For production environments have a look at the deployment documentation.
To add more variables or modify the output parameters, you must modify some files inside the ctd folder:
- SBEtoP09.json: in this file you must map your SeaBird variable to the P09 vocabulary
- cfg_IEO.json: this is a configuration file with specific requirements of IEO, as for example, decimal digits. These are the output variables.
Most of the tests done by the application simply reproduce the procedure recommended by GTSPP. Its real-time quality control manual lays out in detail the automatic tests to be carried out on temperature and salinity data (se also SeaDataNet Data Quality Control Procedures manual).
In order not to reinvent the wheel, some parts of the CoTeDe Python package created by Guilherme P. Castelao and the python-ctd library have been adopted.