🚀 GeoFusion: A Hybrid CNN-Transformer Model for Pixel-Wise Boreal Peatland Classification

The software presented here is part of a high-spatial-resolution peatland database for Finland. Advances in Soil Information-MaaTi project produced methodology and software that distinguishes mire site types, nutrient status, and current land use. Since each country has different public data about natural resources, We publish the project software and synthetic data from three random channels.

This project aims to develop a standard model to handle multi-modal feature fusion for peatland classification. The GeoFusion architecture is a three-stream hybrid CNN-Transformer framework designed for pixel-wise peatland classification based on site types and fertility levels, leveraging both Sentinel-1 and Sentinel-2 optical imagery. While we present the architecture with three streams in this instance, GeoFusion is flexible and can support as many streams as necessary, depending on the number of available data sources.

The streams in this configuration are as follows:

Stream-1: This stream handles Sentinel-1 coherence data.

Stream-2: This stream processes Sentinel-1 intensity data.

Stream-3: This stream handles the Sentinel-2 rasters.

The overview of GeoFusion is as follows:

Get Started

• Installation
• Create Synthetic data to test the code
• Create a master file
• Create spatial window regions based on the annotation samples
• Train the model
• Pixel-wise classification
• Acknowledgements
• Citing

Details

• GeoFusion configuration
• How to use the code with own data

📝 Installation

Clone this Repo

git clone https://github.com/MaaTi-project/GeoFusion.git
cd Geofusion

Create the Conda environment

# move to requirement folder
cd /<ROOT FOLDER>/GeoFusion/Requirements
conda env create -f install_me.yml

Activate the Conda environment

You can activate the Conda environment by running the following command in the terminal.

conda activate maati_transformer

Create Synthetic data to test the code

We assume you might have several raster layers starting from satellite channels (SAR, Landsat) to aerial measurements (kalium gamma, electric conductivity of the surface). Each of these has originally had different raster sizes, so we assume a grid unification has been done (e.g., using bilinear interpolation). The synthetic data provided is randomly generated, but has the mean and variance from three channels.

# move to requirement folder
cd /<ROOT FOLDER>/GeoFusion/CreateSyntheticData

# write to create a synthetic raster  and a no-data raster(randomly generated)
python create_synthethic_rasters.py 

# write to create random annotations 
python create_synthetic_annotations.py

The raster is generated in the Datasets folder.

The output will look like this:

└── TestArea
    ├── Derived
    │   ├── derived_0.tif
    │   ├── derived_1.tif
    │   ├── derived_2.tif
    │   └── derived_3.tif
    ├── Forest
    │   ├── forest_0.tif
    │   ├── forest_1.tif
    │   ├── forest_2.tif
    │   └── forest_3.tif
    └── Optical
        ├── optical_0.tif
        ├── optical_1.tif
        ├── optical_2.tif
        ├── optical_3.tif
        ├── optical_4.tif
        └── optical_5.tif

Create a master file

The master file is a summary file that contains the paths of raster datasets, specifies which input of the GeoFusion each raster corresponds to, identifies the no-data value, and defines the replacement value for missing data.

# move to requirement folder
cd /<ROOT FOLDER>/GeoFusion/CreateSyntheticData

# write to generate a master file to handle patches of raster for other steps 
python create_masterfile.py

The masterfile output will look like this:

../Datasets/TestArea/Optical,channel_1,255,0,5
../Datasets/TestArea/Forest,channel_2,255,0,5
../Datasets/TestArea/Derived,channel_3,255,0,5

Create a summary file with all inputs and channels

# move to requirement folder
cd /<ROOT FOLDER>/GeoFusion/CreateSyntheticData

# write to generate a master file to handle patches of raster for other steps 
python create_summary_masterfiles.py -area TestArea

The script also contains another not required argument --path_to_raster if you want to override the default raster path.

Create spatial window regions based on the annotation samples

In this step, the algorithm will create a window with the annotation point in the center.

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/ManageDataset
# For linux user
source create_windows.sh

Prompt -> Insert area
Reply example -> TESTAREA

Prompt -> insert the path of the inputs
Reply example -> ../Configuration/Masterfiles/dataset_test_area.csv

Prompt-> Override windoiws dimension YES or NO
Reply example -> YES

or

# List files in a directory
cd /<ROOT FOLDER>/GeoFusion/ManageDataset

# from prompt 
python create_dataset.py -area name_of_your_zone \
                         -path_list path_to_your_masterfile \
                         -override_channel override current windiws dimension

Parameters guide:

Parameter	Values
area	Zone name of the analysis
path_list	Path to your masterfile
ovverride_channel	If you want to use different windows dimension instead of the ones in the configuration file

Train the model

To train GeoFusion, one has to:

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/ClassificationGeoFusion
# For Linux users
source launch_geofusion.sh

Prompt -> Write if you want to train drained or undrained data
Reply example -> drained

Prompt -> Write the name of the zone you want to study
Reply example -> TESTAREA

or

# List files in a directory
cd /<ROOT FOLDER>/GeoFusion/ClassificationGeoFusion

# from prompt 
python GeoFusion.py -types type of area you want to analyse \
                    -zone the zone you want to analyse

There is a third parameter called exclude that is used to exclude one input from the fusion. In the case that the launch script has to be used, it should look like this:

# List files in a directory
cd /<ROOT FOLDER>/GeoFusion/ClassificationGeoFusion

# from prompt 
python GeoFusion.py -types type of area you want to analyse \
                    -exclude input to exclude \
                    -zone the zone you want to analyse

Parameters guide:

Parameter	Values
types	Type of wetland of the analysis
exclude	input exclusion parameter
zone	Zone name of the analysis

This process will create a Results_GeoFusion folder that contains a CSV file with a confusion matrix and another one that includes the obtained accuracy for each fold.

Pixel-wise classification

In this work pixel-wise classification occurs in two phases:

Parallel processing

In this phase, the raster is divided into smaller subrasters in order to make computation faster. This is achieved by running the following script:

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/PixelWiseClassification

#run 
source prepare_the_command.sh

Prompt -> Write the area you want to analyse
Reply example ->  TESTAREA

Prompt -> Write pixel size in meter
read example -> 10

Prompt -> Write how many cpu / cores you have
Reply example -> 20

Prompt Write path to the dataset
Reply example -> path_to_dataset

or

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/PixelWiseClassification

python pixelwise_classification_create_commands.py -area Zone name of the analysis \
                                                   --opix Override the current spatial resolution \
                                                   --ocpu amount of cpus \
                                                   --path_to_dataset path to your rasters

Parameters list:

Parameter	Values
area	Zone name of the analysis
opix	Override the current spatial resolution
ocpu	write how many cores you have at your disposal to make the analysis in parallel
path_to_dataset	write the root path to your raster default value is ../Dataset/AREA_NAME

The generated list of commands is saved into the /<ROOT FOLDER>/GeoFusion/PixelWiseClassification/launch_commander/launch_commander_AREA_NAME.txt folder.

Classification map generation

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/PixelWiseClassification

source run_pixelwise_classification.sh

# prompt
Prompt -> "Enter the path of the launch commander"
Reply -> PixelWiseClassification/launch_commander/launch_commander_testarea.txt

Rebuilt final raster

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/Rebuilt

python create_array_from_workers.py -folder output folder of the pixelwise classification step.

Parameters list:

Parameter	Values
folder	Output folder of the pixelwise classification step

The output will be saved into the /<ROOT FOLDER>/GeoFusion/Rebuilt/saved_array.

In the same folder, create a raster from the rebuilt array:

# navigate to 
cd /<ROOT FOLDER>/GeoFusion/Rebuilt

python create_raster_from_array.py.

Acknowledgements

This work is part of the Advances in soil information- MaaTi project funded by the Ministry of Agriculture and Forestry of Finland (2021-2022, funding decision VN/27416/2020-MMM-2). The authors wish to acknowledge CSC – IT Center for Science, Finland, for computational resources, and the MaaTi project management and steering group for constructive comments during the work. The TerraSAR-X and RADARSAT-2 data were supplied through the European Space Agency's third-party mission proposals number. 36096 “Remote sensing as a tool for mapping and evaluating peatlands and peatland carbon stock in Northern Finland; Radarsat-2” and no. 36096 “Remote sensing as a tool for mapping and evaluating peatlands and peatland carbon stock in Northern Finland; Radarsat-2" and by Maarit Middleton in year 2017.

Citing

If you have used our code in your research, please cite our work.

[1] Fahimeh Farahnakiana, Luca Zelioli, Farshad Farahnakian, Maarit Middleton, Javad Sheikh, Jukka Heikkonen, "GeoFusion: Transformer-Based Fusion for Boreal Peatland Classification".