Using the IMOS User Code Library with MATLAB

Table of Contents

1. Introduction

This document intends to present how to load IMOS NetCDF data into a MATLAB environment, and offers some suggestions about how to use the data once loaded. The starting point is running the NetCDF parser from the IMOS user code library.

The examples provided in this document only represent a tiny bit of the content of most of the NetCDF files. There are usually many more variables available in a NetCDF file, and therefore many other ways to display data.

1.1 Installation of the IMOS User Code Library (MATLAB)

The IMOS User Code Library for MATLAB can be downloaded from: https://github.com/aodn/imos-user-code-library/tree/master/MATLAB_R2011

It can be checked out using a Git client, or be downloaded as a zip file : https://github.com/aodn/imos-user-code-library/archive/master.zip

A tertiary toolbox, nctoolbox, needs to be installed. Please follow the description available at http://code.google.com/p/nctoolbox/.

1.2 Finding an IMOS NetCDF File

In order to find a dataset you are interested in, please refer to the portal help: http://help.aodn.org.au/help/?q=node/6. This is a how-to guide that can help users find netCDF files.

For users who are already familiar with IMOS facilities and datasets, IMOS NetCDF files are also directly accessible via an OPeNDAP catalog at : http://thredds.aodn.org.au/thredds/catalog/IMOS/catalog.html

Most of the examples in the following sections use the ‘Data URL’ of a dataset. If you have downloaded your dataset from the portal, the data URL is the file path to the file on your local machine. If you are using the THREDDS catelog, the file does not have to be downloaded to your local machine first – the OPeNDAP data URL can be parsed into MATLAB. The OPeNDAP data URL is found on the ‘OPeNDAP Dataset Access Form’ page (see http://help.aodn.org.au/help/?q=node/11), inside the box labelled ‘Data URL’ just above the ‘Global Attributes’ field.

Note: the list of URL’s generated by the IMOS portal when using that download option can be converted to a list of OPeNDAP data URL’s by replacing string http://data.aodn.org.au/IMOS/opendap with http://thredds.aodn.org.au/thredds/dodsC/IMOS.

2. General Features of the IMOS user code library

The first step consists in parsing a NetCDF file, whether this file is available locally or remotely on an OPeNDAP server.

Type in your MATLAB command window:

file_URL = 'http://thredds.aodn.org.au/thredds/dodsC/IMOS/eMII/demos/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/transect/2011/07/IMOS_SOOP-TMV_TSB_20110709T092744Z_VLST_FV02_transect-D2M_END-20110709T203700Z.nc' ;
dataset = ncParse(file_URL) ;`

2.1 Output structure

ncParse provides a Matlab structure with 2 fields variables and metadata. Metadata contains the NetCDF global attributes while variables contains the variable attributes and their values...

>> dataset

metadata: [1x1 struct]
variables: [1x1 struct]
dimensions: [1x1 struct]

2.2 Discover Metadata

In order to see all the global attributes available, type in your command window:

>> dataset.metadata

project: 'Integrated Marine Observing System (IMOS)'
conventions: 'IMOS-1.2'
title: [1x159 char]
institution: 'Environment Protection Authority Victoria (EPA Vic)'
date_created: '2011-08-02T10:51:05Z'
abstract: [1x330 char]
source: 'ship observation'
keywords: [1x192 char]
platform_code: 'VLST'
netcdf_version: '3.6'
naming_authority: 'IMOS'
quality_control_set: '1'
geospatial_lat_min: -41.177570000000003
geospatial_lat_max: -37.850310000000000
geospatial_lon_min: 1.446208400000000e+02
geospatial_lon_max: 1.463750700000000e+02
geospatial_vertical_min: 0
geospatial_vertical_max: 0
time_coverage_start: '2011-07-09T09:27:44Z'
time_coverage_end: '2011-07-09T20:37:00Z'
local_time_zone: 10
data_centre_email: 'info@emii.org.au'
data_centre: 'eMarine Information Infrastructure (eMII)'
principal_investigator: 'Lee, Randall EPA'
institution_references: 'http://imos.org.au/emii.html'
citation: [1x126 char]
acknowledgment: [1x385 char]
distribution_statement: [1x118 char]
netcdf_filename: 'IMOS_SOOP-TMV_TSB_20110709T092744Z_VLST_FV02_transect-D2M_END-20110709T203700Z.nc'

You can attach to the Matlab variable 'title' the corresponding NetCDF global attribute:

title = dataset.metadata.title

title =
Temperature, salinity, fluorescence and turbidity Data collected by the Spirit of Tasmania ferry 1 - Transect between Devonport and Melbourne on the09-Jul-2011

2.3 Discover Variables

In order to list all the variables available in each NetCDF file, type:

dataset.variables

TEMP: [1x1 struct]
TEMP_2: [1x1 struct]
COND: [1x1 struct]
PSAL: [1x1 struct]
CPHL: [1x1 struct]
TURB: [1x1 struct]
LATITUDE: [1x1 struct]
LONGITUDE: [1x1 struct]

To access the Temperature variable : data values

>> dataset.variables.TEMP.data

names of corresponding dimension variables

>> dataset.variables.TEMP.dimensions
'TIME'

quality control flags

>> dataset.variables.TEMP.flags

variable attributes

2.4 Discover Dimensions

The dimensions values attached to each variable, visible in dataset.variables.VARIABLE.dimensions, can be found in:

>> dataset.dimensions.TIME.data % 1st possibility

>> dataset.dimensions.(char(dataset.variables.TEMP.dimensions{1})).data % 2nd possibility to access to the data of the first dimension.

2.5 Harvest metadata only

It is possible to harvest the metadata (global attributes and variable attributes) only of a NetCDF files :

file_URL = 'http://thredds.aodn.org.au/thredds/dodsC/IMOS/eMII/demos/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/transect/2011/07/IMOS_SOOP-TMV_TSB_20110709T092744Z_VLST_FV02_transect-D2M_END-20110709T203700Z.nc' ;
metadata_subset = ncParse(file_URL, 'parserOption' , 'metadata')

2.6 Harvest a subset of variables

For the big dataset such as the gridded satellite products, a subset of variable is also possible. In this example below, only the data for the temperature variable will be harvested.

file_URL = 'http://thredds.aodn.org.au/thredds/dodsC/IMOS/eMII/demos/SOOP/SOOP-TMV/VLST_Spirit-of-Tasmania-1/transect/2011/07/IMOS_SOOP-TMV_TSB_20110709T092744Z_VLST_FV02_transect-D2M_END-20110709T203700Z.nc' ;
temp_subset = ncParse(file_URL,'varList' ,{'TEMP'})

3. Dataset examples – Using the NetCDF Parser for Plotting

3.1 AATAMS – Animal Tagging and Monitoring - non QC'd data

The Australian Animal Tagging And Monitoring System (AATAMS) is a coordinated marine animal tagging project. CTD Satellite Relay Data Loggers are used to explore how marine mammal behaviour relates to their oceanic environment.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/AATAMS/marine_mammal_ctd-tag/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot all the animal's dives as a single profile time-series of temperature, measured by a CTD tag.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/aatams.m

Example of a Temperature Profile Time-series from AATAMS data

Example of a Temperature profile from AATAMS data

3.2 DWM – Deep Water Mooring

3.2.1 Southern Ocean Time-series - non QC'd data

The Southern Ocean Time Series (SOTS) sub-facility provides high temporal resolution observations in sub-Antarctic waters. Observations are broad and include measurements of physical, chemical and biogeochemical parameters from multiple deep-water moorings.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/DWM/SOTS/catalog.html

In the example below, the ncParse function is used to extract temperature data from a Pulse mooring instrument and then produce a temperature time series plot.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/dwm.m

Time series plot of sea water temperature

3.3 ACORN – Ocean Radar - non QC'd data

The Australian Coastal Ocean Radar Network (ACORN) facility comprises a coordinated network of HF radars delivering real-time, non-quality controlled and delayed-mode, quality controlled surface current data into a national archive.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/ACORN/catalog.html

Monthly aggregated files are also available in the following folders: monthly_gridded_1h-avg-current-map_QC monthly_gridded_1h-avg-current-map_non-QC

In the example below, we demonstrate how to use the ncParse function to plot velocity data for one time value only in a latitude / longitude grid.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/acorn.m

Example of a Sea Water Speed gridded data with a Velocity Field from ACORN data

3.4 ANFOG – Ocean Gliders - QC'd good data

The Australian National Facility for Ocean Gliders (ANFOG), with IMOS/NCRIS funding, deploys a fleet of eight gliders around Australia.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/ANFOG/seaglider/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot salinity data as well as depth data in a same graph. Only the data points with a Quality Control flag greater than 1 (which means 'good data' , please refers to IMOS NetCDF User Manual for a description of the Quality Control, available at http://imos.org.au/facility_manuals.html)

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/anfog.m

Example of Sea Water Time-series taken during a SeaGlider Dive. Filtered to plot good data only

3.5 ANMN – National Mooring Network - QC'd good data

The Australian National Mooring Network Facility is a series of national reference stations and regional moorings designed to monitor particular oceanographic phenomena in Australian coastal ocean waters.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/ANMN/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot the U current variable measured with an ADCP instrument (in Western Australia).

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/anmn_adcp.m

Example of a Sea Water Velocity plot from ADCP data

3.6 AUV – Autonomous Underwater Vehicle - non QC'd data

The IMOS Autonomous Underwater Vehicle (AUV) Facility operates an ocean going AUV called Sirius capable of undertaking high resolution, geo-referenced survey work.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/AUV/catalog.html

In the example below, the ncParse function is used to extract depth, temperature, and time data and then produce a multiple time-series plot showing the variation of water temperature with depth and time during the robot’s dive.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/auv.m

Example of a Temperature Time-series plot during an AUV dive

3.7 Argo – Argo Floats Program - non QC'd data

Argo floats have revolutionised our understanding of the broad scale structure of the oceans to 2000 m depth. In the past 10 years more high resolution hydrographic profiles have been provided by Argo floats then from the rest of the observing system put together. Each Argo float is identified by a unique identification number called a WMO ID.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/Argo/aggregated_datasets/catalog.html

In the examples below, we demonstrate how to use the ncParse function to plot Argo data from an aggregated file ( One file per year per basin: Atlantic, Indian, Pacific North, Pacific South). All the surface locations of all the Argo floats are plotted, as well as one temperature profile only.

3.7.1 Example 1

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/argo1.m

Example of all Profile Locations of Argo Floats

Example of a Sea Water Temperature profile from an Argo float

3.7.2 Example 2

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/argo2.m

Example of a Sea Water Temperature Time-series profile from an Argo float with its location over time

3.8 FAIMMS – Wireless Sensor Networks - QC'd good data

The IMOS Facility for Intelligent Monitoring of Marine Systems is a sensor network established in the Great Barrier Reef off the coast of Queensland, Australia. A 'sensor network' is an array of small, wirelessly interconnected sensors that collectively stream sensor data to a central data aggregation point. Sensor networks can be used to provide spatially dense bio-physical measurements in real-time.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/FAIMMS/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot a temperature time-series. Only data points which have a flag value equal to 1 are used (which means 'good data', please refers to IMOS NetCDF User Manual for a description of the Quality Control, available at http://imos.org.au/facility_manuals.html).

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/faimms.m

Example of a Sea Water Temperature at 5m depth on the Great Barrier Reaf from FAIMMS data

3.9 SOOP – Ship Of Opportunities

3.9.1 XBT - expandable bathythermographs - QC'd data

IMOS Ship of Opportunity Underway Expandable Bathythermographs (XBT) group is a research and data collection project working within the IMOS Ship of Opportunity Multi-Disciplinary Underway Network sub-facility.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/SOOP/SOOP-XBT/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot a XBT temperature profile.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/soop_xbt.m

Example of Sea Water Temperature Profile from XBT data

Example of Sea Water Temperature Time-series Profile from XBT data with the profiles location

3.9.2 Sea Surface Temperature sub-facility – Bureau of Meteorology QC

The Sea Surface Temperature (SST) sub-facility aims to enable accurate, quality controlled, SST data to be supplied in near real-time (within 24 hours) from SOOPs and research vessels in the Australian region.

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/SOOP/SOOP-SST/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot a SOOP SST transect, by selecting only data which 'have passed all tests' according to BoM QC specifications.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/soop_sst.m

Example of SOOP SST transect

3.10 SRS – Satellite Remote Sensing

3.10.1 Bio-Optical database – Pigment data

The bio-optical database underpins the assessment of ocean colour products in the Australian region (e.g. chlorophyll a concentrations, phytoplankton species composition and primary production).

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/SRS/BioOptical/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot a Chlorophyll-a profile (High Performance Liquid Chromatography of pigments in discrete sea-water samples)

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/srs_BioOptical_pigment.m

Example of Pigment Data Profile from the BioOptical database dataset

3.10.2 Bio-Optical database – Absorption data

The bio-optical database underpins the assessment of ocean colour products in the Australian region (e.g. chlorophyll-a concentrations, phytoplankton species composition and primary production).

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/catalog/IMOS/SRS/BioOptical/catalog.html

In the example below, we demonstrate how to use the ncParse function to plot (1) the variation of Absorption coefficients of CDOM (gilvin) in discrete sea-water samples at different wavelengths and (2) the variation of absorption coefficients of CDOM at different wavelengths and different depths.

To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/srs_BioOptical_absorption.m

Example of Absorption Data plot from the BioOptical database dataset

Example of Absorption Data at different depth from the BioOptical database dataset

3.10.3 GHRSST – L3P mosaic

Please refer to the SRS product Help page : http://help.aodn.org.au/help/?q=node/67

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/dodsC/IMOS/SRS/GHRSST-SSTsubskin/

In the example below, we demonstrate how to use the ncParse function to plot the Sea Surface Temperature from a gridded data product.

Warning : this dataset can take a while to be loaded on your machine, or create some java memory heap space errors if your machine lacks off memory. To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/srs_l3p.m

Example of Sea Surface Temperature plot from a L3P product centered on the GBR

3.10.4 GHRSST – L3C – multi swath, single sensor - subset of variables

Please refer to the SRS product Help page : http://portalhelp.aodn.org.au/Portal2_help/?q=node/149

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/dodsC/IMOS/SRS/SRS-SST/L3C-01day/

In the example below, we demonstrate how to use the ncParse function to plot the Sea Surface Temperature from a gridded data product. Only the sea surface temperature and a land mask variable are harvested from the NetCDF file. More variables exist though. This is however quicker and reduces the needed memory by doing this way.

Warning: this dataset can take a while to be loaded on your machine, or create some java memory heap space errors if your machine lacks off memory. To paste the code in your MATLAB environment, please copy it from : https://github.com/aodn/imos-user-code-library/blob/master/MATLAB_R2011/demos/srs_l3c.m

Example of Sea Surface Temperature plot from a L3C product

3.10.5 GHRSST – L3S – multi swath, multi-sensor, one day - subset of variables

Please refer to the SRS product Help page: http://portalhelp.aodn.org.au/Portal2_help/?q=node/149

NetCDF files can be found at : http://thredds.aodn.org.au/thredds/dodsC/IMOS/SRS/SRS-SST/L3S-01day/