This repository includes code to generate 10 m spatial resolution flood and water masks that can be used to develop machine learning models to classify flooding in satellite images.
The flood and water masks are generated by combining various datasets including the ESA WorldCover 10m v100 and vector flood and water layers from the Copernicus Emergency Management System (EMS) Rapid Mapping Activation events.
In total, 321 flood and water mask layers were generated as GeoTIFF files with a 10 m spatial resolution. Each of these layers corresponds to a flood event and date (there can be many flood mask layers per-event which represent flooding on different days as the event evolves). Flood and water masks were generated for 65 flood events since 2018 in 26 countries spanning the tropics and sub-tropics.
Each flood and water mask is a raster image dataset where pixel values correspond to the following classes:
- 1: land and not flooded
- 2: flooding (rasterised from the observed events layers in the EMS Rapid Mapping Activation data)
- 3: permanent water (determined by the water class in the ESA WorldCover 10m v100 product or hydrography layers in the EMS Rapid Mapping Activation data)
Each flood and water mask has three dates associated with it:
- activation date: the date of the activation in the EMS Rapid Mapping Activation system.
- event date: the initial date of the flood event (this can be before the activation date).
- satellite date: the date of the latest input images and data used to generate the observed event data in EMS Rapid Mapping Activation.
The metadata for the flood and water mask files can be found in `/metadata/metadata.csv'.
The data is available for download from the Pacific Data Hub.
Downloads the latest version of vector products for all available flood events in tropical and sub-tropical countries sourced from the Copernicus Emergency Management System (EMS). Using functions from the ml4floods package, vector flood and water maps and metadata are generated.
Downloads the permanent water layer from European Commission's Joint Research Centre (JRC) and land cover from the European Space Agency (ESA) WorldCover 10m v100 product corresponding to each EMS activation event from Google Earth Engine (GEE). The data is spatially and temporally aligned to each EMS Rapid Mapping Activation event.
Note, the data is downloaded from GEE to a Google Cloud Storage bucket. This script should be run when authenticated to GEE and setting the gcs_bucket variable to a Google Cloud Storage bucket name.
Obtains the actual date of the flood events from the Copernicus event information page. The actual event date allows for accurate retrieval of pre-event images if required for subsequent machine learning and flood classification tasks.
Merges the static images that were downloaded from GEE for each event in 01-downlaod-images.py into one raster image using gdal_merge. This is necessary because some images from GEE are split during download.
This script generates a metadata table that contains the event date, the activation date and the satellite date (the date of the latest input images and data used to generate the observed event data in EMS Rapid Mapping Activation) for each image associated with an EMS Rapid Mapping Activation event.
Rasterises the flood and hydrography vector product generated for each EMS Rapid Mapping Activation event and combines this data with land and permanent water classes derived from the ESA WorldCover 10m v100 product.
Processes the bounding boxes of flood-water masks to ensure that all files corresponding to a specific Area of Interest (AOI) have the same bounding box dimension. This allows us to merge the flood-water masks, resulting in a single ground truth image per AOI that represents the maximumm extent of the floods.
Upload merged ground truth files with metadata to GEE.
Dockerfile contains all the Python packages needed to run the scripts above. These packages are also listed in requirements.txt. Follow the instructions below to set up the Docker environment.
-
Using a command line interface, set the current working directory to the project root directory. Build the docker image with:
docker build -t ccai-ground-truth . -
Once the build has completed, run the docker image with:
docker run --rm -p 8888:8888 -v "%cd%/ground-truth":/home/jovyan/work ccai-ground-truth -
You should now be able to access the notebook server via a URL.
This dataset was generated through a project funded by the Climate Change AI Innovation Grants Program. We would also like to acknowledge the ml4floods package and Google Earth Engine which provided some functions used to generate this dataset and the Copernicus Emergency Management Service and European Space Agency (ESA; WorldCover 10 m 2020) for providing access to the underlying data.