AQUAPHYTO

Hyperspectral detection and classification of coastal phytoplankton species from PANTHYR radiometer data.

This repository provides a Python processing pipeline for the automated detection of Phaeocystis globosa, diatoms, and cyanobacteria in Belgian coastal waters using in-situ hyperspectral water reflectance (ρ_w) measured by the PANTHYR autonomous radiometer network.

The pipeline converts raw PANTHYR CSV files into a quality-controlled, ML-ready dataset containing per-day spectral indices and classification labels, designed to feed downstream machine learning models.

This pipeline has been developed with the OBAMA-NEXT project (WP2) and optimized within the ESA-AQUATIME project, including adaptation to detect Cyanobacteria (WP3).

---

Algorithms implemented

Index / method	Species target	Reference
MALH — Modified Astoreca Line Height	P. globosa	Lavigne et al. (2022)
Lubac D² classifier — second derivative of ρ_wN	P. globosa	Lubac et al. (2008)
CRAT — NIR-red chlorophyll-a	bloom biomass	Ruddick et al. (2001)

Water absorption uses the Buiteveld et al. (1994) look-up table with temperature correction (default T = 10 °C, Belgian coastal water).

---

Repository structure

AQUAPHYTO/
├── phytospec/                    # Core Python package
│   ├── __init__.py
│   ├── config.py                 # All paths and algorithm constants
│   ├── io.py                     # Raw CSV reader, datacube load/save
│   ├── algorithms.py             # MALH, CRAT, D², Lubac classifier
│   ├── qc.py                     # Quality control filters
│   └── dataset.py                # High-level orchestration functions
│
├── notebooks/
│   ├── 01_build_datacube.ipynb              # Raw CSV → QC datacube (.npz)
│   ├── 02_make_dataset.ipynb                # Datacube → ML-ready CSV
│   ├── 03_analysis_visualization.ipynb      # Exploratory spectral analysis
│   ├── 04_compare_chime_raw_vs_convolved.ipynb  # CHIME SRF comparison
│   ├── 05_cyano_normalization_sensitivity.ipynb # D² normalisation sensitivity (L. Võrtsjärv)
│   └── 06_LGBM_MALH_CS1.ipynb              # LightGBM MALH retrieval, RT1 + CPOWER
│
├── figures/                      # Output figures (generated by notebooks)
│
├── data/
│   ├── raw/                      # Raw PANTHYR CSV files (not tracked by git)
│   │   ├── RT1_2025/
│   │   ├── CPOWER_2025/
│   │   ├── DATA_FOR_ML/          # ML-ready datasets for Case Study 1
│   │   └── buiteveld_coeffs.csv
│   └── processed/                # Datacubes and datasets (not tracked by git)
│
├── pyproject.toml
├── requirements.txt
└── README.md

---

Notebooks

GitHub's built-in notebook renderer does not support large notebooks. Use the nbviewer links below to view rendered versions.

#	Notebook	Description
01	Build datacube	Raw PANTHYR CSV → QC datacube (.npz)
02	Make dataset	Datacube → ML-ready CSV with spectral indices
03	Analysis & visualization	Exploratory spectral analysis and figures
04	CHIME raw vs convolved	Algorithm transferability across CHIME spectral resolutions
05	Cyano normalisation sensitivity	D² normalisation sensitivity analysis for L. Võrtsjärv cyanobacteria
06	LightGBM MALH CS1	LightGBM MALH retrieval under CHIME revisit gaps, RT1 + CPOWER

---

Installation

git clone https://github.com/REMSEM/AQUAPHYTO.git
cd AQUAPHYTO
pip install -e .

Dependencies (automatically installed via pyproject.toml): numpy, scipy, pandas, matplotlib

---

Quick start

Step 1 — Build the datacube from raw files

Open notebooks/01_build_datacube.ipynb and set STATION and YEAR:

STATION = "RT1"
YEAR    = 2025

This reads all *QA_data.csv files from data/raw/RT1_2025/, applies quality control (sun-glint filter, NaN filter), and saves a compressed datacube to:

data/processed/datacube_RT1_2025.npz

Step 2 — Compute spectral indices and build the ML dataset

Open notebooks/02_make_dataset.ipynb. Running make_dataset() computes CHL, MALH, D²ρ_w, and the Lubac P. globosa label for each spectrum, selects one spectrum per day (closest to solar noon), and saves the result as:

data/processed/REFERENCE_DATASET_4_WP2_RT1_2025.csv

Step 3 — Exploratory analysis

notebooks/03_analysis_visualization.ipynb provides mean spectral signatures, interquartile ranges, and MALH vs Chl-a scatter plots by species class.

Step 4 — LightGBM MALH retrieval (Case Study 1)

notebooks/06_LGBM_MALH_CS1.ipynb trains a LightGBM regressor to predict MALH under CHIME satellite revisit gaps using ancillary data and the last known MALH value. Requires the prepared datasets in data/raw/DATA_FOR_ML/.

---

Configuration

All paths and algorithm constants are centralised in phytospec/config.py. Edit only that file when adapting the pipeline to a new station or year.

Key parameters:

Parameter	Default	Description
D2_NORM_WL	None (→ 442.5 nm)	Normalisation wavelength for D²
D2_DELTA	2.5 nm	Wavelength step for second derivative
BUITEVELD_T	10.0 °C	Water temperature for absorption correction
QC_SZA_MAX	75°	Maximum solar zenith angle
QC_LDEX_MAX	0.05 sr⁻¹	Maximum Ld/Ed ratio at 750 nm

---

Stations supported

Station	Longitude	Latitude	Description
RT1	2.9193 °E	51.2464 °N	Belgian Coastal Zone — Ostend reference tower
CPOWER	2.9255 °E	51.5368 °N	Belgian Coastal Zone — offshore wind farm

---

Data format

Input — raw PANTHYR CSV (*QA_data.csv)

Column	Description
wavelength	Wavelength [nm]
rhow_nosc	Water reflectance, sky-glint corrected [sr⁻¹]
lu	Upwelling radiance [W m⁻² nm⁻¹ sr⁻¹]
ed	Downwelling irradiance [W m⁻² nm⁻¹]
solar_zenith_angle	Solar zenith angle [°]

Output — ML-ready dataset CSV

Columns: date, CHL, MALH, P_LUB, rhow_355.0 … rhow_945.0, D2rhow_355.0 … D2rhow_945.0

---

References

• Lavigne, H. et al. (2022). Remote Sensing of Environment, 282, 113270.
• Lubac, B. et al. (2008). Journal of Geophysical Research: Oceans, 113.
• Ruddick, K. et al. (2001). Applied Optics, 40(9).
• Buiteveld, H. et al. (1994). Proc. SPIE — Ocean Optics XII.

---

License

This project is developed in the framework of the AQUATIME project (ESA Sentinel Users Preparation Initiative) by RBINS (Royal Belgian Institute of Natural Sciences) — REMSEM group.