MagicBathyNet: A Multimodal Remote Sensing Dataset for Benchmarking Learning-based Bathymetry and Pixel-based Classification in Shallow Waters

MagicBathyNet is a benchmark dataset made up of image patches of Sentinel-2, SPOT-6 and aerial imagery, bathymetry in raster format and seabed classes annotations. Dataset also facilitates unsupervised learning for model pre-training in shallow coastal areas.

Package for benchmarking MagicBathyNet dataset in learning-based bathymetry and pixel-based classification.

This repository contains the code of the paper MagicBathyNet: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-based Classification in Shallow Waters accepted for publication in the 2024 IEEE International Geoscience and Remote Sensing Symposium (IGARSS 2024).

Citation

If you find this repository useful, please consider giving a star ⭐.
If you use the code in this repository or the dataset please cite:

Agrafiotis, P., Janowski, L., Skarlatos, D. & Demir, B. (2024) MagicBathyNet: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-based Classification in Shallow Waters, arXiv preprint arXiv:2405.15477, 2024.

@article{agrafiotis2024magicbathynet,
      title={MagicBathyNet: A Multimodal Remote Sensing Dataset for Bathymetry Prediction and Pixel-based Classification in Shallow Waters}, 
      author={Panagiotis Agrafiotis and Łukasz Janowski and Dimitrios Skarlatos and Begüm Demir},
      year={2024},
      journal={arXiv preprint arXiv:2405.15477}
}

Getting started

Downloading the dataset

For downloading the dataset and a detailed explanation of it, please visit the MagicBathy Project website at https://www.magicbathy.eu/magicbathynet.html

Dataset structure

The folder structure should be as follows:

┗ 📂 magicbathynet/
  ┣ 📂 agia_napa/
  ┃ ┣ 📂 img/
  ┃ ┃ ┣ 📂 aerial/
  ┃ ┃ ┃ ┣ 📜 img_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┃ ┣ 📂 s2/
  ┃ ┃ ┃ ┣ 📜 img_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┃ ┣ 📂 spot6/
  ┃ ┃ ┃ ┣ 📜 img_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┣ 📂 depth/
  ┃ ┃ ┣ 📂 aerial/
  ┃ ┃ ┃ ┣ 📜 depth_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┃ ┣ 📂 s2/
  ┃ ┃ ┃ ┣ 📜 depth_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┃ ┣ 📂 spot6/
  ┃ ┃ ┃ ┣ 📜 depth_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┣ 📂 gts/
  ┃ ┃ ┣ 📂 aerial/
  ┃ ┃ ┃ ┣ 📜 gts_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┃ ┣ 📂 s2/
  ┃ ┃ ┃ ┣ 📜 gts_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┃ ┣ 📂 spot6/
  ┃ ┃ ┃ ┣ 📜 gts_339.tif
  ┃ ┃ ┃ ┣ 📜 ...
  ┃ ┣ 📜 [modality]_split_bathymetry.txt
  ┃ ┣ 📜 [modality]_split_pixel_class.txt
  ┃ ┣ 📜 norm_param_[modality]_an.txt
  ┃
  ┣ 📂 puck_lagoon/
  ┃ ┣ 📂 img/
  ┃ ┃ ┣ 📜 ...
  ┃ ┣ 📂 depth/
  ┃ ┃ ┣ 📜 ...
  ┃ ┣ 📂 gts/
  ┃ ┃ ┣ 📜 ...
  ┃ ┣ 📜 [modality]_split_bathymetry.txt
  ┃ ┣ 📜 [modality]_split_pixel_class.txt
  ┃ ┣ 📜 norm_param_[modality]_pl.txt

The mapping between RGB color values and classes is:

For the Agia Napa area:
0 : (0, 128, 0),   #seagrass
1 : (0, 0, 255),   #rock
2 : (255, 0, 0),   #macroalgae
3 : (255, 128, 0), #sand
4 : (0, 0, 0)}     #Undefined (black)

For the Puck Lagoon area:
0 : (255, 128, 0), #sand
1 : (0, 128, 0) ,  #eelgrass/pondweed
2 : (0, 0, 0)}     #Undefined (black)

Clone the repo

git clone https://github.com/pagraf/MagicBathyNet.git

Installation Guide

The requirements are easily installed via Anaconda (recommended):

conda env create -f environment.yml

After the installation is completed, activate the environment:

conda activate magicbathynet

Train and Test the models

To train and test the bathymetry models use MagicBathy_Benchmarking_Bathymetry.ipynb.

To train and test the pixel-based classification models use MagicBathy_Benchmarking_semsegm.ipynb.

Pre-trained Deep Learning Models

We provide code and model weights for the following deep learning models that have been pre-trained on MagicBathyNet for pixel-based classification and bathymetry tasks:

Pixel-based classification

Model Names	Modality	Area	Pre-Trained PyTorch Models
U-Net	Aerial	Agia Napa	unet_aerial_an.zip
SegFormer	Aerial	Agia Napa	segformer_aerial_an.zip
U-Net	Aerial	Puck Lagoon	unet_aerial_pl.zip
SegFormer	Aerial	Puck Lagoon	segformer_aerial_pl.zip
U-Net	SPOT-6	Agia Napa	unet_spot6_an.zip
SegFormer	SPOT-6	Agia Napa	segformer_spot6_an.zip
U-Net	SPOT-6	Puck Lagoon	unet_spot6_pl.zip
SegFormer	SPOT-6	Puck Lagoon	segformer_spot6_pl.zip
U-Net	Sentinel-2	Agia Napa	unet_s2_an.zip
SegFormer	Sentinel-2	Agia Napa	segformer_s2_an.zip
U-Net	Sentinel-2	Puck Lagoon	unet_s2_pl.zip
SegFormer	Sentinel-2	Puck Lagoon	segformer_s2_pl.zip

Learning-based Bathymetry

Model Name	Modality	Area	Pre-Trained PyTorch Models
Modified U-Net for bathymetry	Aerial	Agia Napa	bathymetry_aerial_an.zip
Modified U-Net for bathymetry	Aerial	Puck Lagoon	bathymetry_aerial_pl.zip
Modified U-Net for bathymetry	SPOT-6	Agia Napa	bathymetry_spot6_an.zip
Modified U-Net for bathymetry	SPOT-6	Puck Lagoon	bathymetry_spot6_pl.zip
Modified U-Net for bathymetry	Sentinel-2	Agia Napa	bathymetry_s2_an.zip
Modified U-Net for bathymetry	Sentinel-2	Puck Lagoon	bathymetry_s2_pl.zip

To achieve the results presented in the paper, use the parameters and the specific train-evaluation splits provided in the dataset. Parameters can be found here while train-evaluation splits are included in the dataset.

Example testing results

Example patch of the Agia Napa area (left), pixel classification results obtained by U-Net (middle) and predicted bathymetry obtained by MagicBathy-U-Net (right). For more information on the results and accuracy achieved read our paper.

Authors

Panagiotis Agrafiotis https://www.user.tu-berlin.de/pagraf/

Feedback

Feel free to give feedback, by sending an email to: agrafiotis@tu-berlin.de

Funding

This work is part of MagicBathy project funded by the European Union’s HORIZON Europe research and innovation programme under the Marie Skłodowska-Curie GA 101063294. Work has been carried out at the Remote Sensing Image Analysis group. For more information about the project visit https://www.magicbathy.eu/.