5. Data Formats

Format Standards and Resources

General Standards

• OGC (Open Geospatial Consortium) - International standards for geospatial data interoperability
• NOAA Data Formats - NOAA/NCEI format specifications
• Darwin Core - Biological occurrence data standard (used by OBIS)
• Ocean Data Standards (IODE) - International oceanographic data exchange standards

Common and Preferred Formats

Tabular Data

Common formats: CSV, NetCDF, Excel, HDF5

When to use what:

• CSV: Event logs, sample catalogs, simple sensor data, broad sharing
• NetCDF: Large multidimensional arrays, gridded data, archival submissions (follow CF conventions)
• Excel: Internal organization, manual entry (always export to CSV for archival)
• HDF5: Complex hierarchical data, high-performance computing workflows

Geospatial Data

Vector formats: GeoJSON, Shapefile, GeoPackage Raster formats: GeoTIFF, NetCDF

When to use what:

• GeoJSON: Dive tracks, sample locations, web mapping (single file, human-readable)
• Shapefile: Legacy GIS compatibility (multi-file format, being phased out)
• GeoPackage: Modern alternative to shapefiles (single file, no size limits)
• GeoTIFF: Bathymetry grids, classified habitat maps
• NetCDF: Gridded oceanographic data for modeling

Key tools:

• GDAL/OGR - Geospatial format conversion (raster and vector)
• QGIS - Open-source GIS software
• GeoJSON.io - Validate and visualize GeoJSON online

Specialized Oceanographic Formats

Multibeam Data

• GSF (Generic Sensor Format) - Widely used for multibeam bathymetry
• Proprietary formats: Kongsberg (.all, .kmall), Reson (.s7k)
• Processed grids: GeoTIFF, NetCDF, ASCII XYZ

Tools:

• MB-System - Open-source multibeam processing and format conversion

CTD Data

• Raw: Seabird (.cnv), RBR (.rsk)
• Processed: CSV, NetCDF

ADCP Data

• Raw: Teledyne binaries (.raw, .raw.log, .raw.log.bin)

Tools:

• CODAS - Open-source processing and visualization package for ADCP data

Video and Imagery

Video

Recommended archival format: MP4 with H.264 or H.265 codec

• Balance of quality, compression, and universal compatibility
• Avoid legacy formats (AVI) and editing codecs (ProRes, DNxHD) for archival

Tools:

• ffmpeg - Video format conversion and processing

Still Imagery

Archival master: TIFF or DNG (RAW) Distribution: High-quality JPEG (90-95%)

Metadata embedding:

• ExifTool - Read/write image metadata (GPS, depth, timestamp, annotations)

---

Contributing: Report format issues or suggest additional format resources.