-
Notifications
You must be signed in to change notification settings - Fork 1
Real Time File Description
dpsnowden edited this page Jul 22, 2013
·
65 revisions
#Table of Contents
Introduction
Global Attributes
File Naming Convention
Axis Definitions
Variable Definitions
Other Stuff
The Global Attributes that are included in the real time glider files are listed and defined below. These attributes derive from several sources as indicated in the square brackets next to the terms. More information on these sources can be found at the following locations.
- CF1.6 Section 2.6 of the current (v1.6) Climate and Forecast conventions
- ACDD Attribute Conventions for Dataset Discovery Home page and Current Standard
- NODC Guidance from NOAA's National Oceanographic Data Center on netCDF templates to promote good stewardship and archiving. NODC Templates and global attribute suggestions. In addition to listing useful attributes NODC provides a style guide to help in populating the global attributes with useful and clear information.
- IMOS/ANFOG IMOS Data Management manual version 3.1
- IOOS Internal discussion within the IOOS Glider Data Team.
- acknowledgement [ACDD]
- A place to acknowledge various type of support for the project that produced this data.
- cdm_data_type [ACDD]
- This attribute is used by THREDDS to identify the feature type, what THREDDS calls a "dataType". The current THREDDS choices are: Grid, Image, Station, Swath, and Trajectory. The mandatory value to be used with the glider file format is Trajectory.
- comment [CF/ACDD]
- Miscellaneous information about the data.
- contributor_name [ACDD]
- A comma separated list with the names of any individuals or institutions that contributed to the creation of this data.
- contributor_role [ACDD]
- A comma separated list with the roles assumed by the individuals or institutions that are referenced in contributor_name.
- creator_email [ACDD]
- Email address for the person principally responsible for creating the digital data set.
- creator_name [ACDD]
- Name of the person principally responsible for creating the digital data set. The data creator's name, URL, and email. The "institution" attribute will be used if the "creator_name" attribute does not exist.
- creator_url [ACDD]
- URL to a page with reference information describing creation of the data set.
- date_created [ACDD]
- The date on which the data was created. Following [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601)
- date_issued [ACDD]
- The date on which this data was formally issued. Following [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601)
- date_modified [ACDD]
- The date on which this data was last modified. Following [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601)
- featureType [CF]
- The CF 1.6 DSG featureType used to encode the data in the data set (e.g trajectory) [Section 9.1](http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#idp6280704)
- file_version [IOOS]
- The version of the IOOS RT Glider template. (e.g. IOOS_Glider_NetCDF_Trajectory_Template_v0.1)
- geospatial_bounds [ACDD]
- Describes geospatial extent using any of the geometric objects (2D or 3D) supported by the Well-Known Text (WKT) format.
- geospatial_lat_max [ACDD]
- The values of geospatial_lon_min and geospatial_lon_max reflect the actual longitude data values. Cases where geospatial_lon_min is greater than geospatial_lon_max indicate the bounding box extends from geospatial_lon_max, through the longitude range discontinuity meridian (either the antimeridian or Prime Meridian), to geospatial_lon_min.
- geospatial_lat_min [ACDD]
- Describes a simple latitude/longitude bounding box. geospatial_lat_min specifies the southernmost latitude; geospatial_lat_max specifies the northernmost latitude; geospatial_lon_min specifies the westernmost longitude; geospatial_lon_max specifies the easternmost longitude of the bounding box.
- geospatial_lat_resolution [ACDD]
- geospatial_lat_units [ACDD]
- Further refinement of the geospatial bounding box can be provided by using these units and resolution attributes.
- geospatial_lon_max [ACDD]
- geospatial_lon_min [ACDD]
- For a more detailed geospatial coverage, see the suggested geospatial attributes.
- geospatial_lon_resolution
- geospatial_lon_units [ACDD]
- geospatial_vertical_max [ACDD]
- geospatial_vertical_min [ACDD]
- Describes a simple vertical bounding box. For a more detailed geospatial coverage, see the suggested geospatial attributes.
- geospatial_vertical_positive [ACDD]
- This attribute indicates which direction is positive (a value of "up" means that z increases up, like units of height, while a value of "down" means that z increases downward, like units of pressure or depth). Required Usage: Select from up or down.
- geospatial_vertical_resolution [ACDD]
- geospatial_vertical_units [ACDD]
- This defines the units applied to the geospatial_vertical_min and geospatial_vertical_max attributes. Conform to [udunits](http://www.unidata.ucar.edu/software/udunits/).
- history [CF/ACDD]
- Provides an audit trail for modifications to the original data. Well-behaved generic netCDF filters will automatically append their name and the parameters with which they were invoked to the global history attribute of an input netCDF file. We recommend that each line begin with a timestamp indicating the date and time of day that the program was executed.
- id [ACDD]
- An identifier for the data set, provided by and unique within its naming authority. The combination of the "naming authority" and the "id" should be globally unique, but the id can be globally unique by itself also. IDs can be URLs, URNs, DOIs, meaningful text strings, a local key, or any other unique string of characters. The id should not include blanks.
- institution [CF/ACDD]
- Specifies where the original data was produced.
- keywords [ACDD]
- A comma separated list of key words and phrases.
- keywords_vocabulary [ACDD]
- If you are following a guideline for the words/phrases in your "keywords" attribute, put the name of that guideline here.
- license [ACDD]
- Describe the restrictions to data access and distribution.
- Metadata_Conventions [ACDD]
- This attribute should be set to "Unidata Dataset Discovery v1.0" or the most current version for netCDF files that follow this convention. Note: be careful with the case here. It is indeed "Metadata_Conventions", with the capitalization. Most attributes are normally lower case. *Also note that this is a proposed attribute, and not yet officially part of ACDD. TODO: Propose change to ACDD including a change to the capitalization and to the reference to Unidata. Suggest refering to the ACDD wiki at ESIP.
- metadata_link [ACDD]
- This attribute provides a link to a complete metadata record for this dataset or the collection that contains this dataset. This attribute is not included in Version 1 of the Unidata Attribute Convention for Data Discovery and is also a proposed attribute. It is recommended here because a complete metadata collection for a dataset will likely contain more information than can be included in granule formats. This attribute contains a link to that information. Note: This is a proposed attribute in ACDD. The latest version of the TDS "UDDC" service accepts either "Metadata_Link" or "metadata_link". NODC recommends the use of the lower case form to be more consistent with other attributes, which with only one or two exceptions are also lower case.
- naming_authority [ACDD]
- See id above. The combination of the "naming authority" and the "id" should be a globally unique identifier for the dataset. Recommended Usage: Recommend using the reverse URL of the institution. (ex., gov.noaa.nodc)
- processing_level [ACDD]
- A textual description of the processing (or quality control) level of the data.
- project [ACDD]
- The scientific project that produced the data.
- publisher_email [ACDD]
- The data publisher's name, URL, and email. The publisher may be an individual or an institution.
- publisher_name [ACDD]
- The data publisher's name, URL, and email. The publisher may be an individual or an institution.
- publisher_url [ACDD]
- The data publisher's name, URL, and email. The publisher may be an individual or an institution.
- references [CF]
- Published or web-based references that describe the data or methods used to produce it.
- sea_name [NODC]
- Contains information on the large scale oceanographic name in which the glider is deployed.
- source [CF]
- The definition from [CF](http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/cf-conventions.html#description-of-file-contents): The method of production of the original data. If it was model-generated, source should name the model and its version, as specifically as could be useful. If it is observational, source should characterize it (e.g., "surface observation" or "radiosonde"). Recommended usage: "Observational data form a profiling glider".
- standard_name_vocabulary [ACDD]
- The name of the controlled vocabulary from which variable standard names are taken. As of 05 July 2013 the most current version of the [CF controlled vocabulary](http://cf-pcmdi.llnl.gov/documents/cf-standard-names) list is v25. Recommended usage: CF v25
- summary [ACDD]
- A paragraph describing the dataset.
- time_coverage_duration[ACDD]
- Describes the temporal coverage of the data as a time range. The temporal coverage of the data can be described with any of the following pairs of values: start/end, start/duration, or end/duration. Recommended Usage: Use ISO 8601 for date and time.
- time_coverage_end [ACDD]
- Describes the temporal coverage of the data as a time range. The temporal coverage of the data can be described with any of the following pairs of values: start/end, start/duration, or end/duration. Recommended Usage: Use ISO 8601 for date and time.
- time_coverage_resolution [ACDD]
- Describes the resolution of the time variable in the data. Recommended Usage: Use ISO 8601 for time resolution (examples: P1Y ,P3M, P10D).
- time_coverage_start [ACDD]
- Describes the temporal coverage of the data as a time range. The temporal coverage of the data can be described with any of the following pairs of values: start/end, start/duration, or end/duration. Recommended Usage: Use ISO 8601 for date and time.
- title [CF]
- A succinct description of what is in the dataset.
- glider-SN_yyyymmddTHHMMSS_rt0.nc: Real-time data with no QC. This is the minimum processing level accepted by the DAC and contains the raw data values with no operator provided quality control.
- glider-SN_yyyymmddTHHMMSS_rt1.nc: Real-time data with operator provided QC
- glider-SN_yyyymmddTHHMMSS_delayed0.nc: Delayed-mode data with no QC
- glider-SN_yyyymmddTHHMMSS_delayed1.nc: Delayed-mode data with operator provided QC
where
- glider
- Identifying name or type abbreviation for the glider
- SN
- vehicle serial number as provided by the manufacturer
- yyyymmddTHHMMSS
- http://en.wikipedia.org/wiki/ISO_8601 [ISO 8601](http://en.wikipedia.org/wiki/ISO_8601) formatted date representing the start time of the data acquisition
- 'rt' or 'delayed'
- string specifying real-time (during deployment) or delayed mode (post-recovery) data acquisition
- 0 or 1
- quality control level where 0 corresponds to none and 1 corresponds to some level of operator applied quality control
Ideally, the glider-SN_yyyymmddTHHMMSS_rt1.nc files will be provided by the individual operators during the deployment and the glider-SN_yyyymmddTHHMMSS_delayed1.nc files will be provided after the glider has been recovered and the full data set processed. It is expected that all files containing operator QC'd data will provide the appropriate VARIABLE_qc variables with corresponding attributes from the CF specification.
# Axis/Dimension Definitions ## Time There are two axes along which geophysical variables are dimensioned:time(length=unlimited)
time_uv(length=1)
TODO: Address the various questions in this section.
The CF feature type (aka [http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.6/aphs04.html](discrete sampling geometry)) used for these files is trajectory. Accordingly, each real time file contains a variable containing information identifying the trajectory.
The ncml depiction of this information is
<variable name="trajectory" shape="trajectory" type="short">
<attribute name="cf_role" value="trajectory_id" />
<attribute name="comment" value="A trajectory can span multiple data files each containing a single segment." />
<attribute name="long_name" value="Unique identifier for each trajectory feature contained in the file" />
</variable>Definition of deployment from the [https://github.com/IOOSProfilingGliders/Real-Time-File-Format/wiki/Glider-Types-and-Sampling#sampling-patterns](wiki page) is: A series of one or more segments completed by a glider between the time of deployment and the time of recovery.
For real time individual files the length of the trajectory variable is 1. Value of the trajectory variable is 1. When many segment files are aggregated into a deployment the trajectory variable, which is identical across all of the segment files, indicates that all variables come from the same deployment.
TODO: IS THIS NECESSARY? Do we need to provide strict guidance for the usage of this variable and its shape?
# Variable Description ## Mandatory VariablesThe NODC netCDF templates have introduced a convention that is very useful for encoding platform and instrument metadata in a modular way. Dimensionless container variables act as place-holders for platform and instrument metadata and are easily referenced from geophysical variables with variable attributes. For example, the following variable, platform, contains information describing the glider platform used to collect the measurements.
int platform
platform:type = "spray"
platform:wmo_id = "NNYYXX" ;
platform:comment = "Spray Glider sp111" ;
platform:id = "sp111" ;
platform:long_name = "Spray Glider sp111" ;
platform:instrument = "instrument_ctd" ;Similarly, the following instrument container variable contains metadata for the CTD sensor that is mounted on the above "spray" glider platform.
int instrument_ctd
instrument_ctd:comment = "Unpumped CTD with a nominal sampling rate of 1Hz." ;
instrument_ctd:serial_number = -1 ;
instrument_ctd:long_name = "Seabird SBD 41CP Conductivity, Temperature, Depth Sensor." ;
instrument_ctd:platform = "platform"QC Codes
Platform Types: The platfrom:type attribute is selected from the following controlled vocabulary.
- slocum
- spray
- seaglider