Skip to content

A MATLAB toolbox for interacting with bulk freely-available oceanographic data.

License

Notifications You must be signed in to change notification settings

lnferris/ocean_data_tools

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ocean_data_tools: a MATLAB toolbox for interacting with bulk freely-available oceanographic data

GitHub license GitHub stars GitHub forks View ocean_data_tools on File Exchange DOI

Copyright (c) 2020 lnferris

ocean_data_tools simplifies the process of extracting, formatting, and visualizing freely-available oceanographic data. While a wealth of oceanographic data is accessible online, some end-users may be dissuaded from utilizing this data due to the overhead associated with obtaining and formatting it into usable data structures. ocean_data_tools solves this problem by allowing the user to transform common oceanographic data sources into uniform structs, call generalized functions on these structs, easily perform custom calculations, and make graphics.

Find a bug, have a question, or want to chat about contributing? Open an issue or email lnferris@alum.mit.edu.

Getting Started

  1. Download bathymetry.
  2. Download nctoolbox. You will need to run the command setup_nctoolbox at the beginning of each MATLAB session.
  3. Add ocean_data_tools and nctoolbox to the path. Specifically, the following folders must be added to the path:
  • ocean_data_tools/ocean_data_tools
  • ocean_data_tools/ocean_data_tools/utilities
  • nctoolbox/
  1. Run each demonstration in demos/demos.m, which contains example usages for all functions. All required test data is included in data/.

Functions are named using a two-part system. The prefix (argo_, bathymetry_, general_, etc.) indicates the appropriate data source, while the suffix (\_build, \_profiles, \_section, etc.) indicates the action performed. Functions with the \_build suffix load raw data into uniform structs (e.g. argo, cruise, hycom, mercator, woa, wod). Uniform structs created by \_build functions are compatable with any general_ function.

Data sources currently supported:

Data Source DOI, Product Code, or Link
Argo floats doi:10.17882/42182
Smith & Sandwell bathymetry doi:10.1126/science.277.5334.1956
IOOS Glider DAC https://gliders.ioos.us/
MOCHA Climatology doi:10.7282/T3XW4N4M
HYbrid Coordinate Ocean Model https://hycom.org
CMEMS Global Ocean 1/12° Physics Analysis and Forecast GLOBAL_ANALYSIS_FORECAST_ PHY_001_024
CMEMS Global Ocean Waves Multi Year GLOBAL_REANALYSIS_WAV_001_032
GO-SHIP hydrographic cruises https://www.go-ship.org/
World Ocean Atlas 2018 https://www.ncei.noaa.gov/products/world-ocean-atlas
World Ocean Database https://www.ncei.noaa.gov/products/world-ocean-database

Main functions are located in ocean_data_tools/. Demonstrations are located in demos/. Test datas are located in data/. Shell scripts for batch downloading data are located in shell_scripts/. While shell scripts can be run directly in a macOS Terminal, running them in Windows requires Cygwin (and perhaps slight modification of commands). Python syntax examples are located in python/, which may be grow to become a module in the future.

Dependencies

The only true dependency is nctoolbox.

It is recommended to also download Gibbs-SeaWater (GSW) Oceanographic Toolbox. A benefit of ocean_data_tools is that neatly packs data into uniform structs; at which point a user can easily apply custom calculations or functions from other toolboxes such as GSW. See an example.

Accessing Help

To access help, run the command doc ocean_data_tools.

How to Contribute

  • Want to make changes or add a new function? (1) Fork the repository (make your own separate copy), (2) make changes, and (3) open a 'pull request'. Once approved, it can be merged into the master branch. If you wish to chat beforehand about your contribution, open an issue or email lnferris@alum.mit.edu.
  • Don't use git often and don't want to remember all the terminal commands? Download GitHub Desktop.
  • Find a bug in the code? Open an 'issue' to notify contributors and create an official record.

Before contributing, please see Contents and consider how your function fits into ocean_data_tools and its ethos of structure arrays. At a minimum, functions must be well-documented and address a specific freely-available oceanographic data source which can be accessed by anyone online.

Adding a new function isn't the only way to contribute. Python, Julia, etc. translations of existing Matlab functions are also welcomed!

If you are interested in becoming a formal collaborator (e.g. have direct access and co-manage this repository), please reach out.

Contents

Building uniform structs from data sources

argo_build searches the locally-stored Argo profiles matching the specified region & time period and builds a uniform struct

glider_build loads an archived glider survey (downloaded from gliders.ioos.us/erddap) and builds a uniform struct

mocha_build_profiles builds a uniform struct of profiles from the MOCHA Mid-Atlantic Bight climatology

model_build_profiles builds a uniform struct of profiles from HYCOM or Operational Mercator CMEMS GLOBAL_ANALYSIS_FORECAST_PHY_001_024

waves_build builds a uniform struct of timeseries from CMEMS Global Ocean Waves Multi Year product GLOBAL_REANALYSIS_WAV_001_032

whp_cruise_build builds a uniform struct of profiles from GO-SHIP cruise data in WHP-Exchange Format

woa_build_profiles builds a uniform struct of profiles from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields

wod_build builds a uniform struct of profiles from World Ocean Database data

Don't see a function yet for your preferred data source? Email lnferris@alum.mit.edu to request or contribute.

General functions for subsetting and plotting uniform structs

general_depth_subset subsets a uniform struct by depth

general_map plots coordinate locations in a uniform struct, with optional bathymetry contours

general_profiles plots vertical profiles in a uniform struct

general_region_subset subsets a uniform struct by polygon region

general_remove_duplicates removes spatially (or spatially and temporally) non-unique profiles from a uniform struct

general_section plots a data section from a uniform struct

Plotting gridded data without building structs

mocha_domain_plot plots a 3-D domain from the MOCHA Mid-Atlantic Bight climatology

mocha_simple_plot plots a 2-D layer from the MOCHA Mid-Atlantic Bight climatology

model_domain_plot plots a 3-D domain from HYCOM or Operational Mercator CMEMS GLOBAL_ANALYSIS_FORECAST_PHY_001_024

model_simple_plot plots a 2-D layer from HYCOM or Operational Mercator CMEMS GLOBAL_ANALYSIS_FORECAST_PHY_001_024

woa_domain_plot plots a 3-D domain from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields

woa_simple_plot plots a 2-D layer from World Ocean Atlas 2018 Statistical Mean for All Decades, Objectively Analyzed Mean Fields

Adding bathymetry to existing plots

bathymetry_extract extracts a region of Smith & Sandwell Global Topography and outputs as arrays

bathymetry_plot adds bathymetry to 2-D (latitude vs. longitude) or 3-D (latitude vs. longitude vs. depth) data plots

bounding_region finds the rectangular region around a uniform struct and/or list of coordinates to pass as an argument for other bathymetry functions

bathymetry_section adds Smith & Sandwell Global Topography to a section from plot using bathymetry data nearest to specified coordinates

Additional functions for inspecting Argo data

argo_platform_map plots locations of Argo profiles in a uniform struct, coloring markers by platform (individual Argo float)

argo_platform_subset subsets a uniform struct of Argo data to one platform (individual Argo float)

argo_profiles_map plots coordinate locations of Argo profiles in uniform struct argo, using colors corresponding to argo_profiles called on the same struct

argo_profiles plots vertical Argo profiles in uniform struct argo, using colors corresponding to argo_profiles_map called on the same struct

Miscellaneous utilities

region_select creates coordinate list (which represents vertices of a polygon region) by clicking stations on a plot

transect_select creates a coordinate list (which represents a virtual transect) by clicking stations on a plot

Finding Data

There two types of datasets: those that need to be downloaded manually1 and those that can be accessed remotely2 through OpenDAP (e.g. the data can be accessed directly on the the internet using a url).

argo1

Download Argo data directly from GDAC FTP servers using either the Coriolis selection tool, or the US GDAC. See the Argo User's Manual for more information.

Alternatively run shell_scripts/download_argo to download data via File Transfer Protocol.

bathymetry1

To get bathymetry data (for bathymetry_dir), download Smith & Sandwell under Global Topography V19.1 in netcdf form (topo_20.1.nc).

glider1

Vist gliders.ioos.us/erddap. Click "View a List of All 779 Datasets" or use the "Advanced Search". After choosing a dataset, navigate to the Data Access Form. To get started, select these variables:

Scroll to "File type:". In the drop-down menu, select ".nc". Click "Submit".

mocha2

The url for MOCHA Mid-Atlantic Bight climatology is embedded. See Rutgers Marine catalog.

model1,2

HYCOM data may be accessed remotely using OpenDAP. Get the data url by visiting the HYCOM website. For example, click Access Data Here -> GLBv0.08/expt_57.7 (Jun-01-2017 to Sep-30-2017)/ -> Hindcast Data: Jun-01-2017 to Sep-30-2017. Click on the OpenDAP link. Copy the url as and use this as the source in model_build_profiles.

Alteratively, download subsetted HYCOM data using NCSS. Get the data url by visiting the HYCOM website. For example, click Access Data Here -> GLBv0.08/expt_57.7 (Jun-01-2017 to Sep-30-2017)/ -> Hindcast Data: Jun-01-2017 to Sep-30-2017. Click on the NetcdfSubset link. Set constraints and copy the NCSS Request URL at the bottom of the page. Run shell_scripts/download_hycom_lite. To download multiple months or years, run shell_scripts/download_hycom_bulk_daily (partition files by day) or shell_scripts/download_hycom_bulk_monthly (partition files by month). Please use responsibly.

For Mercator, download Copernicus Marine data directly from FTP servers. First make a Copernicus account. Use the selection tool to download GLOBAL_ANALYSIS_FORECAST_PHY_001_024. Alternatively run shell_scripts/download_mercator. Before running the script, follow the instructions for modifying your ~/.netrc file in the comments of the script.

waves1

First make a Copernicus account. Use the selection tool to download CMEMS Global Ocean Waves Multi Year product GLOBAL_REANALYSIS_WAV_001_032.

whp_cruise1

For GO-SHIP data, get CTD data (for ctdo_dir) by choosing a GO-SHIP cruise and downloading the CTD data in whp_netcdf format. More information about whp_netcdf parameters is available here. Get LADCP data (for uv_dir, wke_dir) here. There is information about LACDP processing here.

woa2

Functions build the World Ocean Atlas url at maximum resolution based on arguments, but coarser resolutions and seasonal climatologies are available at the NODC website. Note NCEI is scheduled to update data urls in the near future. Functions will be updated as such.

wod1

Search the World Ocean Database and select products.

Citing ODT

Ferris, L., (2020). ocean_data_tools: A MATLAB toolbox for interacting with bulk freely-available oceanographic data. Journal of Open Source Software, 5(54), 2497. https://doi.org/10.21105/joss.02497