diff --git a/02-schedule.md b/02-schedule.md index 96ba2e1..fb4326a 100644 --- a/02-schedule.md +++ b/02-schedule.md @@ -16,7 +16,7 @@ AGU! | 9:25 AM | 10 minutes | **Break** | | | 9:35 AM | 60 minutes | [](modules/02-interactive-viz/index.ipynb) | Qiusheng | | 10:35 AM | 10 minutes | **Break** | | -| 10:45 AM | 60 minutes | [](modules/03-integrating-ai/index.md) | Qiusheng | +| 10:45 AM | 60 minutes | [](modules/03-integrating-ai/index.ipynb) | Qiusheng | | 11:45 AM | 15 minutes | **Q&A** | All | ## 🍽️ Lunch! (12:00 - 1:30) diff --git a/modules/03-integrating-ai/index.ipynb b/modules/03-integrating-ai/index.ipynb new file mode 100644 index 0000000..ea1352c --- /dev/null +++ b/modules/03-integrating-ai/index.ipynb @@ -0,0 +1,1910 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "83d7ede9", + "metadata": {}, + "source": [ + "# 🤖 3 - Integrating artificial intelligence with geospatial data analysis and visualization\n", + "\n", + "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/geojupyter/workshop-open-source-geospatial/blob/main/modules/03-integrating-ai/index.ipynb)\n", + "\n", + "## Introduction\n", + "\n", + "This notebook is for the workshop ([Open Source Geospatial Workflows in the Cloud](https://geojupyter.github.io/workshop-open-source-geospatial)) presented at the [AGU Fall Meeting 2025](https://agu.confex.com/agu/agu25/meetingapp.cgi/Session/252640).\n", + "\n", + "**GeoAI** represents the intersection of geospatial science and artificial intelligence, combining the power of machine learning with geographic information systems (GIS) to analyze, understand, and predict spatial patterns. This rapidly growing field enables us to extract meaningful insights from satellite imagery, aerial photos, and other geospatial datasets at unprecedented scales and accuracy levels.\n", + "\n", + "The **GeoAI Python package** (https://opengeoai.org) simplifies the application of deep learning models to geospatial data, making advanced AI techniques accessible to researchers, analysts, and practitioners in environmental science, urban planning, agriculture, and disaster management. The package provides a unified interface for:\n", + "\n", + "- **Data Preprocessing**: Automated handling of various geospatial data formats, coordinate systems, and multi-spectral imagery\n", + "- **Model Training**: Pre-configured deep learning architectures optimized for geospatial tasks like semantic segmentation and object detection\n", + "- **Feature Extraction**: Automated extraction of geographic features from satellite and aerial imagery\n", + "- **Visualization**: Interactive mapping and analysis tools for exploring results\n", + "\n", + "In this workshop, you will:\n", + "\n", + "- Discover the core capabilities of the GeoAI package, including data preprocessing, feature extraction, and geospatial deep learning workflows\n", + "- See live demonstrations on applying state-of-the-art AI models to satellite and aerial imagery\n", + "- Learn how to train custom segmentation models for surface water mapping using different data sources\n", + "- Explore real-world use cases in building footprint extraction and surface water mapping\n", + "\n", + "Additional Resources\n", + "\n", + "- GitHub: [GeoAI](https://github.com/opengeos/geoai)\n", + "- Book: [Introduction to GIS Programming: A Practical Python Guide to Open Source Geospatial Tools](https://gispro.gishub.org)\n", + "- YouTube: [Open Geospatial Solution](https://youtube.com/@giswqs)" + ] + }, + { + "cell_type": "markdown", + "id": "7188ad0f", + "metadata": {}, + "source": [ + "## Deep Learning Architectures and Encoders\n", + "\n", + "Before moving into the hands-on work, it’s important to first understand the key ideas behind **deep learning architectures** and **encoders**.\n", + "\n", + "A **deep learning architecture** is like the **blueprint of a factory**. It defines how the network is organized, how data flows through different components, and how raw inputs are transformed into meaningful outputs. Just as a factory blueprint specifies where materials enter, how they are processed, and where finished products come out, a neural network architecture lays out the arrangement of layers (neurons) that progressively extract and refine patterns from data—for example, detecting cats in images or translating between languages.\n", + "\n", + "Within this blueprint, an **encoder** functions as a **specialized assembly line**. Its role is to take messy raw materials (input data) and refine them into a compact, standardized representation that is easier for the rest of the system to work with. Some architectures also include a **decoder** assembly line, which reconstructs or generates the final output from the encoder’s compressed representation—for example, assembling a finished car from engine parts and panels.\n", + "\n", + "In short:\n", + "\n", + "- **Model architecture = the factory blueprint (overall design and flow)**\n", + "- **Encoder = the preprocessing line (condenses raw inputs into useful parts)**\n", + "- **Decoder = the finishing line (turns encoded parts into a final product)**\n", + "\n", + "### Types of Architectures\n", + "\n", + "Different blueprints are suited for different tasks:\n", + "\n", + "- **Feedforward Neural Networks**: simple, one-directional flow of data.\n", + "- **Convolutional Neural Networks (CNNs)**: specialized for images, capturing spatial patterns like edges and textures.\n", + "- **Recurrent Neural Networks (RNNs)**: designed for sequences, such as speech or time series.\n", + "- **Transformers**: powerful models for language and beyond, using attention mechanisms (e.g., ChatGPT).\n", + "\n", + "### What Does an Encoder Do?\n", + "\n", + "An **encoder** is the part of a neural network that takes an input (like an image or a sentence) and compresses it into a smaller, meaningful form called a **feature representation** or **embedding**. This process keeps the essential information while filtering out noise.\n", + "\n", + "For example, the sentence _“I love pizza”_ might be converted by an encoder into a vector of numbers that still reflects its meaning, but in a way that is easier for a computer to analyze and use.\n", + "\n", + "Encoders appear in many contexts:\n", + "\n", + "- **Autoencoders**: learn to compress and reconstruct data.\n", + "- **Transformer Encoders**: such as BERT, used for language understanding.\n", + "- **Encoder–Decoder Models**: such as translation systems, where the encoder reads one language and the decoder generates another.\n", + "\n", + "### Encoders and Architectures in Practice\n", + "\n", + "The [pytorch segmentation models library](https://github.com/qubvel-org/segmentation_models.pytorch) provides a wide range of pre-trained models for semantic segmentation. It separates **architectures** (the blueprint) from **encoders** (the feature extractors):\n", + "\n", + "- **Architectures**: `unet`, `unetplusplus`, `deeplabv3`, `deeplabv3plus`, `fpn`, `pspnet`, `linknet`, `manet`.\n", + "- **Encoders**: `resnet34`, `resnet50`, `efficientnet-b0`, `mobilenet_v2`, and many more.\n", + "\n", + "The **GeoAI** package builds on this library, offering a convenient wrapper so you can easily train your own segmentation models with a variety of architectures and encoders." + ] + }, + { + "cell_type": "markdown", + "id": "e5e64934", + "metadata": {}, + "source": [ + "## Environment Setup\n", + "\n", + "You can run this notebook locally or in Google Colab. You will need a GPU for training deep learning models. If you don't have a GPU, you can use the free GPU in Google Colab.\n", + "\n", + "To install the GeoAI package, it is recommended to use a virtual environment. Please refer to the [GeoAI installation guide](https://opengeoai.org/installation/) for more details.\n", + "\n", + "Here is a quick start guide to install the GeoAI package:\n", + "\n", + "```bash\n", + "conda create -n geo -c conda-forge mamba\n", + "conda activate geo\n", + "mamba install -c conda-forge python=3.12 geoai\n", + "```\n", + "\n", + "If you have a GPU, you can install the package with GPU support:\n", + "\n", + "```bash\n", + "mamba install -c conda-forge geoai \"pytorch=*=cuda*\"\n", + "```\n", + "\n", + "You can install the package using pip:\n", + "\n", + "```bash\n", + "pip install geoai-py\n", + "```" + ] + }, + { + "cell_type": "markdown", + "id": "ced6bc6d", + "metadata": {}, + "source": [ + "## Use Colab GPU\n", + "\n", + "To use GPU, please click the \"Runtime\" menu and select \"Change runtime type\". Then select \"T4 GPU\" from the dropdown menu. GPU acceleration is highly recommended for training deep learning models, as it can reduce training time from hours to minutes.\n", + "\n", + "## Install packages\n", + "\n", + "Uncomment the following cell to install the package. It may take a few minutes to install the package and its dependencies. Please be patient." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3adbcdc4", + "metadata": {}, + "outputs": [], + "source": [ + "# %pip install geoai-py" + ] + }, + { + "cell_type": "markdown", + "id": "2e1f76c5", + "metadata": {}, + "source": [ + "## Import libraries\n", + "\n", + "Let's import the GeoAI package." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "5f175b2c", + "metadata": {}, + "outputs": [], + "source": [ + "import geoai" + ] + }, + { + "cell_type": "markdown", + "id": "13df03ed", + "metadata": {}, + "source": [ + "## Surface water mapping with non-georeferenced satellite imagery\n", + "\n", + "**Surface water mapping** is one of the most important applications of GeoAI, as water resources are critical for ecosystem health, agriculture, urban planning, and climate monitoring. In this first demonstration, we'll work with **non-georeferenced satellite imagery** in standard image formats (JPG/PNG), which is often how satellite imagery is initially distributed or stored.\n", + "\n", + "**Why start with non-georeferenced imagery?**\n", + "\n", + "- Many datasets and online sources provide satellite imagery without embedded geographic coordinates\n", + "- It demonstrates the core computer vision aspects of GeoAI before adding geospatial complexity\n", + "- The techniques learned here can be applied to any imagery, regardless of coordinate system\n", + "- It's often faster to iterate and experiment with standard image formats\n", + "\n", + "We'll use **semantic segmentation**, a deep learning technique that classifies every pixel in an image. Unlike object detection (which draws bounding boxes), semantic segmentation provides precise pixel-level predictions, making it ideal for mapping natural features like water bodies that have irregular shapes.\n", + "\n", + "### Download sample data\n", + "\n", + "We'll use the [waterbody dataset](https://www.kaggle.com/datasets/franciscoescobar/satellite-images-of-water-bodies) from Kaggle, which contains 2,841 satellite image pairs with corresponding water masks. This dataset is particularly valuable because:\n", + "\n", + "- **Diverse geographic coverage**: Images from different continents and climate zones\n", + "- **Varied water body types**: Lakes, rivers, ponds, and coastal areas\n", + "- **Multiple seasons and conditions**: Different lighting conditions and seasonal variations\n", + "- **High-quality annotations**: Manually verified water body masks for training\n", + "\n", + "Credits to the author Francisco Escobar for providing this dataset.\n", + "\n", + "I downloaded the dataset from Kaggle and uploaded it to Hugging Face for easy access:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "af9e5ebb", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/waterbody-dataset.zip\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "67a06cc3", + "metadata": {}, + "outputs": [], + "source": [ + "out_folder = geoai.download_file(url)\n", + "print(f\"Downloaded dataset to {out_folder}\")" + ] + }, + { + "cell_type": "markdown", + "id": "f8230022", + "metadata": {}, + "source": [ + "The unzipped dataset contains two folders: `images` and `masks`. Each folder contains 2,841 images in JPG format. The `images` folder contains the original satellite imagery, and the `masks` folder contains the corresponding surface water masks in binary format (white pixels = water, black pixels = background).\n", + "\n", + "**Dataset characteristics:**\n", + "\n", + "- **Total image pairs**: 2,841 training examples\n", + "- **Image format**: RGB satellite imagery (3 channels)\n", + "- **Mask format**: Binary masks where 255 = water, 0 = background\n", + "- **Variable image sizes**: Ranging from small 256x256 patches to larger 1024x1024+ images\n", + "- **Global coverage**: Samples from diverse geographic regions and water body types" + ] + }, + { + "cell_type": "markdown", + "id": "4035c545", + "metadata": {}, + "source": [ + "### Train semantic segmentation model\n", + "\n", + "Now we'll train a semantic segmentation model using **U-Net** architecture with a **ResNet34** encoder and **ImageNet** pre-trained weights. Let's break down these important choices:\n", + "\n", + "**Architecture: U-Net**\n", + "\n", + "- **U-Net** is a convolutional neural network architecture specifically designed for semantic segmentation\n", + "- It has a \"U\" shape with an encoder (downsampling) path and a decoder (upsampling) path\n", + "- **Skip connections** between encoder and decoder preserve fine-grained spatial details\n", + "- Originally designed for medical image segmentation, it works exceptionally well for geospatial applications\n", + "\n", + "**Encoder: ResNet34**\n", + "\n", + "- **ResNet34** is a 34-layer Residual Network that serves as the feature extraction backbone\n", + "- **Residual connections** allow training of very deep networks without vanishing gradient problems\n", + "- Balances model complexity with computational efficiency (deeper than ResNet18, more efficient than ResNet50)\n", + "- Well-suited for satellite imagery feature extraction\n", + "\n", + "**Pre-trained weights: ImageNet**\n", + "\n", + "- **Transfer learning** from ImageNet provides a strong starting point for feature extraction\n", + "- ImageNet-trained models have learned to recognize edges, textures, and patterns relevant to natural imagery\n", + "- Significantly reduces training time and improves performance, especially with limited training data\n", + "- The encoder starts with knowledge of general image features, then specializes for water detection\n", + "\n", + "**Key training parameters:**\n", + "\n", + "- `num_channels=3`: RGB satellite imagery (red, green, blue bands)\n", + "- `num_classes=2`: Binary classification (background vs. water)\n", + "- `batch_size=32`: Process 32 images simultaneously for efficient GPU utilization\n", + "- `num_epochs=3`: Training iterations (limited for demo; real-world would use 20-50+ epochs)\n", + "- `learning_rate=0.001`: Controls optimization step size\n", + "- `val_split=0.2`: Reserve 20% of data for validation to monitor overfitting\n", + "- `target_size=(512, 512)`: Standardize all images to 512x512 pixels for consistent processing\n", + "\n", + "For more details on available architectures and encoders, please refer to https://smp.readthedocs.io/en/latest/encoders.html." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7cc72ff6", + "metadata": {}, + "outputs": [], + "source": [ + "# Test train_segmentation_model with automatic size detection\n", + "geoai.train_segmentation_model(\n", + " images_dir=f\"{out_folder}/images\",\n", + " labels_dir=f\"{out_folder}/masks\",\n", + " output_dir=f\"{out_folder}/unet_models\",\n", + " architecture=\"unet\", # The architecture to use for the model\n", + " encoder_name=\"resnet34\", # The encoder to use for the model\n", + " encoder_weights=\"imagenet\", # The weights to use for the encoder\n", + " num_channels=3, # number of channels in the input image\n", + " num_classes=2, # background and water\n", + " batch_size=16, # The number of images to process in each batch\n", + " num_epochs=3, # training for 3 epochs to save time, in practice you should train for more epochs\n", + " learning_rate=0.001, # learning rate for the optimizer\n", + " val_split=0.2, # 20% of the data for validation\n", + " target_size=(512, 512), # target size of the input image\n", + " verbose=True, # print progress\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0d69019c", + "metadata": {}, + "source": [ + "In the model output folder `unet_models`, you will find the following files:\n", + "\n", + "- `best_model.pth`: The best model checkpoint (highest validation IoU)\n", + "- `final_model.pth`: The last model checkpoint from the final epoch\n", + "- `training_history.pth`: Complete training metrics for analysis and plotting\n", + "- `training_summary.txt`: Human-readable summary of training configuration and results" + ] + }, + { + "cell_type": "markdown", + "id": "5ecbd461", + "metadata": {}, + "source": [ + "### Evaluate the model\n", + "\n", + "Model evaluation is crucial for understanding how well our trained model performs. We'll examine both the **training curves** and **quantitative metrics** to assess model quality and identify potential issues like overfitting or underfitting.\n", + "\n", + "**Key evaluation metrics for semantic segmentation:**\n", + "\n", + "1. **Loss**: Measures how far the model's predictions are from the ground truth\n", + "\n", + " - **Training loss**: How well the model fits the training data\n", + " - **Validation loss**: How well the model generalizes to unseen data\n", + " - **Ideal pattern**: Both should decrease, with validation loss closely following training loss\n", + "\n", + "2. **IoU (Intersection over Union)**: The most important metric for segmentation tasks\n", + "\n", + " - **Definition**: Area of overlap / Area of union between prediction and ground truth\n", + " - **Range**: 0.0 (no overlap) to 1.0 (perfect overlap)\n", + " - **Interpretation**: 0.69 IoU means ~69% accurate pixel-level water detection\n", + " - **Industry standard**: IoU > 0.7 is generally considered good performance\n", + "\n", + "3. **Dice (F-1) Score**: Alternative segmentation metric, closely related to IoU\n", + " - **Definition**: 2 × (Area of overlap) / (Total pixels in both prediction and ground truth)\n", + " - **Range**: 0.0 to 1.0, similar to IoU but slightly more lenient\n", + " - **Relationship**: Dice = 2×IoU / (1+IoU)\n", + "\n", + "IoU and Dice are monotonically related—optimizing one generally optimizes the other. However, Dice tends to give slightly higher values than IoU for the same segmentation.\n", + "\n", + "- **IoU** is stricter: it penalizes false positives and false negatives more heavily, making it less forgiving of small mismatches.\n", + "\n", + "- **Dice** is more sensitive to overlap and is often preferred in **medical image segmentation**, where the overlap between predicted and actual regions is more important than absolute boundaries.\n", + "\n", + "- **IoU** is often used in **object detection and computer vision challenges** (e.g., COCO benchmark), because it aligns with bounding box overlap evaluation.\n", + "\n", + "Let's examine the training curves and model performance:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ae17fcc5", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.plot_performance_metrics(\n", + " history_path=f\"{out_folder}/unet_models/training_history.pth\",\n", + " figsize=(15, 5),\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "0d8e2501", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/381ce436-3520-4706-9def-b0a7ae8244ac)" + ] + }, + { + "cell_type": "markdown", + "id": "a0c8f49c", + "metadata": {}, + "source": [ + "### Run inference on a single image\n", + "\n", + "**Inference** is the process of using our trained model to make predictions on new, unseen images. This is where we see the practical application of our trained model.\n", + "\n", + "**Note on testing approach:**\n", + "In this demo, we're using one of the training images for inference to demonstrate the workflow. In a real-world scenario, you should always test on completely independent images that were never seen during training to get an accurate assessment of model performance.\n", + "\n", + "You can run inference on a new image using the `semantic_segmentation` function:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "09baf6b6", + "metadata": {}, + "outputs": [], + "source": [ + "index = 3 # change it to other image index, e.g., 100\n", + "test_image_path = f\"{out_folder}/images/water_body_{index}.jpg\"\n", + "ground_truth_path = f\"{out_folder}/masks/water_body_{index}.jpg\"\n", + "prediction_path = f\"{out_folder}/prediction/water_body_{index}.png\" # save as png to preserve exact values and avoid compression artifacts\n", + "model_path = f\"{out_folder}/unet_models/best_model.pth\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "20d6891b", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.semantic_segmentation(\n", + " input_path=test_image_path,\n", + " output_path=prediction_path,\n", + " model_path=model_path,\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " num_channels=3,\n", + " num_classes=2,\n", + " window_size=512,\n", + " overlap=256,\n", + " batch_size=32,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b2e5876a", + "metadata": {}, + "outputs": [], + "source": [ + "fig = geoai.plot_prediction_comparison(\n", + " original_image=test_image_path,\n", + " prediction_image=prediction_path,\n", + " ground_truth_image=ground_truth_path,\n", + " titles=[\"Original\", \"Prediction\", \"Ground Truth\"],\n", + " figsize=(15, 5),\n", + " save_path=f\"{out_folder}/prediction/water_body_{index}_comparison.png\",\n", + " show_plot=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "bac01b07", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/00308228-0819-4161-9a35-6a98f4cefa93)" + ] + }, + { + "cell_type": "markdown", + "id": "ab76e685", + "metadata": {}, + "source": [ + "### Run inference on multiple images\n", + "\n", + "**Batch processing** is essential for operational applications where you need to process many images efficiently. The GeoAI package provides `semantic_segmentation_batch` for processing entire directories of images with consistent parameters.\n", + "\n", + "First, let's download a sample set of test images that the model has never seen:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d9bcdd1", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/waterbody-dataset-sample.zip\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3e87eef5", + "metadata": {}, + "outputs": [], + "source": [ + "data_dir = geoai.download_file(url)\n", + "print(f\"Downloaded dataset to {data_dir}\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7dca731d", + "metadata": {}, + "outputs": [], + "source": [ + "images_dir = f\"{data_dir}/images\"\n", + "masks_dir = f\"{data_dir}/masks\"\n", + "predictions_dir = f\"{data_dir}/predictions\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "388f10ed", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.semantic_segmentation_batch(\n", + " input_dir=images_dir,\n", + " output_dir=predictions_dir,\n", + " model_path=model_path,\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " num_channels=3,\n", + " num_classes=2,\n", + " window_size=512,\n", + " overlap=256,\n", + " batch_size=4,\n", + " quiet=True,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e86f622d", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.empty_cache()" + ] + }, + { + "cell_type": "markdown", + "id": "d05c07a6", + "metadata": {}, + "source": [ + "## Surface water mapping with Sentinel-2 imagery\n", + "\n", + "In the second part of this notebook, we'll demonstrate surface water mapping using **Sentinel-2 satellite imagery**, which provides **multispectral data** with much richer information than standard RGB imagery.\n", + "\n", + "**Why Sentinel-2 is ideal for water mapping:**\n", + "\n", + "**Sentinel-2** is a European Space Agency (ESA) satellite mission providing high-resolution optical imagery for land monitoring. Key advantages for water detection include:\n", + "\n", + "- **Multispectral capabilities**: 13 spectral bands covering visible, near-infrared, and short-wave infrared\n", + "- **High spatial resolution**: 10-20m pixels for detailed water body mapping\n", + "- **Frequent revisit time**: 5-day global coverage for monitoring temporal changes\n", + "- **Free and open access**: Available through Copernicus Open Access Hub and other platforms\n", + "- **Consistent quality**: Calibrated, atmospherically corrected imagery (Level 2A)\n", + "\n", + "**Spectral bands used in this analysis:**\n", + "\n", + "1. **Blue (490nm)**: Water absorption, atmospheric correction\n", + "2. **Green (560nm)**: Vegetation health, water clarity\n", + "3. **Red (665nm)**: Vegetation chlorophyll, land-water contrast\n", + "4. **Near-Infrared (842nm)**: **Critical for water detection** - water strongly absorbs NIR\n", + "5. **SWIR1 (1610nm)**: **Excellent water discriminator** - water has very low reflectance\n", + "6. **SWIR2 (2190nm)**: **Additional water detection** - separates water from wet soil/vegetation\n", + "\n", + "### Download sample data\n", + "\n", + "We'll use the [Earth Surface Water Dataset](https://zenodo.org/records/5205674#.Y4iEFezP1hE) from Zenodo, which contains Sentinel-2 imagery with 6 spectral bands and corresponding water masks. Credits to Xin Luo for creating this high-quality dataset.\n", + "\n", + "**Dataset characteristics:**\n", + "\n", + "- **Sensor**: Sentinel-2 Level 2A (atmospherically corrected)\n", + "- **Bands**: Blue, Green, Red, NIR, SWIR1, SWIR2 (6 channels total)\n", + "- **Spatial resolution**: 10-20 meters per pixel\n", + "- **Geographic coverage**: Multiple global locations with diverse water body types\n", + "- **Ground truth**: Expert-annotated water masks for training and validation" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "495d0ebd", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/dset-s2.zip\"\n", + "data_dir = geoai.download_file(url, output_path=\"dset-s2.zip\")" + ] + }, + { + "cell_type": "markdown", + "id": "7e80206d", + "metadata": {}, + "source": [ + "**Dataset structure:**\n", + "\n", + "In the unzipped dataset, we have four folders:\n", + "\n", + "- `dset-s2/tra_scene`: **Training images** - Sentinel-2 scenes for model training\n", + "- `dset-s2/tra_truth`: **Training masks** - Corresponding water truth masks\n", + "- `dset-s2/val_scene`: **Validation images** - Independent Sentinel-2 scenes for testing\n", + "- `dset-s2/val_truth`: **Validation masks** - Ground truth for performance evaluation\n", + "\n", + "We will use the training images and masks to train our model, then evaluate performance on the completely independent validation set." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fde41891", + "metadata": {}, + "outputs": [], + "source": [ + "images_dir = f\"{data_dir}/dset-s2/tra_scene\"\n", + "masks_dir = f\"{data_dir}/dset-s2/tra_truth\"\n", + "tiles_dir = f\"{data_dir}/dset-s2/tiles\"" + ] + }, + { + "cell_type": "markdown", + "id": "2c90ba2d", + "metadata": {}, + "source": [ + "### Create training data\n", + "\n", + "We'll create smaller training tiles from the large GeoTIFF images. Note that we have multiple Sentinel-2 scenes in the training and validation sets, we will use the `export_geotiff_tiles_batch` function to export tiles from each scene." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6dc599a4", + "metadata": {}, + "outputs": [], + "source": [ + "result = geoai.export_geotiff_tiles_batch(\n", + " images_folder=images_dir,\n", + " masks_folder=masks_dir,\n", + " output_folder=tiles_dir,\n", + " tile_size=512,\n", + " stride=128,\n", + " quiet=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "5ad4e20f", + "metadata": {}, + "source": [ + "### Train semantic segmentation model\n", + "\n", + "Now we'll train a semantic segmentation model specifically for **6-channel Sentinel-2 imagery**. The key difference from our previous model is the input channel configuration.\n", + "\n", + "**Important parameter changes:**\n", + "\n", + "- `num_channels=6`: Accommodate the 6 Sentinel-2 spectral bands (Blue, Green, Red, NIR, SWIR1, SWIR2)\n", + "- `num_epochs=5`: Slightly more training epochs to learn complex spectral relationships\n", + "- **Architecture remains U-Net + ResNet34**: Proven effective for multispectral imagery\n", + "\n", + "Let's train the model using the Sentinel-2 tiles:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "bce77601", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.train_segmentation_model(\n", + " images_dir=f\"{tiles_dir}/images\",\n", + " labels_dir=f\"{tiles_dir}/masks\",\n", + " output_dir=f\"{tiles_dir}/unet_models\",\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " encoder_weights=\"imagenet\",\n", + " num_channels=6,\n", + " num_classes=2, # background and water\n", + " batch_size=32,\n", + " num_epochs=3, # training for 3 epochs to save time, in practice you should train for more epochs\n", + " learning_rate=0.001,\n", + " val_split=0.2,\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "108a1202", + "metadata": {}, + "source": [ + "### Evaluate the model\n", + "\n", + "Let's examine the training curves and model performance:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a624b023", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.plot_performance_metrics(\n", + " history_path=f\"{tiles_dir}/unet_models/training_history.pth\",\n", + " figsize=(15, 5),\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "759368d5", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/61f675a7-ee67-4650-81c0-f754fe681f4d)" + ] + }, + { + "cell_type": "markdown", + "id": "b35771ac", + "metadata": {}, + "source": [ + "### Run inference\n", + "\n", + "Now we'll run inference on the validation set to evaluate the model's performance. We will use the `semantic_segmentation_batch` function to process all the validation images at once." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3a2894d3", + "metadata": {}, + "outputs": [], + "source": [ + "images_dir = f\"{data_dir}/dset-s2/val_scene\"\n", + "masks_dir = f\"{data_dir}/dset-s2/val_truth\"\n", + "predictions_dir = f\"{data_dir}/dset-s2/predictions\"\n", + "model_path = f\"{tiles_dir}/unet_models/best_model.pth\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "8c072f45", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.semantic_segmentation_batch(\n", + " input_dir=images_dir,\n", + " output_dir=predictions_dir,\n", + " model_path=model_path,\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " num_channels=6,\n", + " num_classes=2,\n", + " window_size=512,\n", + " overlap=256,\n", + " batch_size=32,\n", + " quiet=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "bef0fa3c", + "metadata": {}, + "source": [ + "### Visualize results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b4124da9", + "metadata": {}, + "outputs": [], + "source": [ + "image_id = \"S2A_L2A_20190318_N0211_R061\" # Change to other image id, e.g., S2B_L2A_20190620_N0212_R047\n", + "test_image_path = f\"{data_dir}/dset-s2/val_scene/{image_id}_6Bands_S2.tif\"\n", + "ground_truth_path = f\"{data_dir}/dset-s2/val_truth/{image_id}_S2_Truth.tif\"\n", + "prediction_path = f\"{data_dir}/dset-s2/predictions/{image_id}_6Bands_S2_mask.tif\"\n", + "save_path = f\"{data_dir}/dset-s2/{image_id}_6Bands_S2_comparison.png\"\n", + "\n", + "fig = geoai.plot_prediction_comparison(\n", + " original_image=test_image_path,\n", + " prediction_image=prediction_path,\n", + " ground_truth_image=ground_truth_path,\n", + " titles=[\"Original\", \"Prediction\", \"Ground Truth\"],\n", + " figsize=(15, 5),\n", + " save_path=save_path,\n", + " show_plot=True,\n", + " indexes=[5, 4, 3],\n", + " divider=5000,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "677d2ed0", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/53601ed7-2bd6-4e7e-b369-4d7bfc2ce120)" + ] + }, + { + "cell_type": "markdown", + "id": "14378f0b", + "metadata": {}, + "source": [ + "### Download Sentinel-2 imagery\n", + "\n", + "**Real-world data acquisition** is a crucial skill for operational GeoAI applications. Here we'll demonstrate how to:\n", + "\n", + "1. **Search for Sentinel-2 data** using STAC (SpatioTemporal Asset Catalog) APIs\n", + "2. **Apply quality filters** (cloud cover, date range, geographic bounds)\n", + "3. **Download specific spectral bands** needed for our analysis\n", + "4. **Prepare data** for inference with our trained model\n", + "\n", + "**STAC catalogs** provide a standardized way to search and access satellite imagery across different providers. The Earth Search STAC API aggregates Sentinel-2 data from AWS Open Data, making it easily accessible for analysis.\n", + "\n", + "**Search parameters:**\n", + "\n", + "- **Geographic bounds**: Define area of interest (bbox)\n", + "- **Temporal range**: Specify date range for imagery\n", + "- **Cloud cover filter**: Limit to images with <10% cloud cover\n", + "- **Collection**: Focus on Sentinel-2 Level 2A (atmospherically corrected)\n", + "- **Sorting**: Order by cloud cover (ascending) to get clearest images first\n", + "\n", + "Let's set up an interactive map to explore available Sentinel-2 data:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d3019fa5", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import leafmap" + ] + }, + { + "cell_type": "markdown", + "id": "61992521", + "metadata": {}, + "source": [ + "Set up the [TiTiler](https://developmentseed.org/titiler/) endpoint for visualizing raster data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "ebf8fb57", + "metadata": {}, + "outputs": [], + "source": [ + "os.environ[\"TITILER_ENDPOINT\"] = \"https://giswqs-titiler-endpoint.hf.space\"" + ] + }, + { + "cell_type": "markdown", + "id": "85fdbaf1", + "metadata": {}, + "source": [ + "Create an interactive map to explore available Sentinel-2 data." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a7c6c3f4", + "metadata": {}, + "outputs": [], + "source": [ + "m = leafmap.Map(center=[46.693725, -95.925399], zoom=12)\n", + "m.add_basemap(\"Esri.WorldImagery\")\n", + "m.add_stac_gui()\n", + "m" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3e0a80ac", + "metadata": {}, + "outputs": [], + "source": [ + "try:\n", + " print(m.user_roi_bounds())\n", + "except:\n", + " print(\"Please draw a rectangle on the map before running this cell\")" + ] + }, + { + "cell_type": "markdown", + "id": "fccbde61", + "metadata": {}, + "source": [ + "Use the drawing tool to draw a rectangle on the map. Click on the **Search** button to search Sentinel-2 imagery intersecting the rectangle." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "b3cff04c", + "metadata": {}, + "outputs": [], + "source": [ + "try:\n", + " display(m.stac_gdf)\n", + "except:\n", + " print(\"Click on the Search button before running this cell\")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1473ce41", + "metadata": {}, + "outputs": [], + "source": [ + "try:\n", + " display(m.stac_item)\n", + "except:\n", + " print(\"click on the Display button before running this cell\")" + ] + }, + { + "cell_type": "markdown", + "id": "6f10a1ec", + "metadata": {}, + "source": [ + "Search for Sentinel-2 data programmatically." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "376c7a59", + "metadata": {}, + "outputs": [], + "source": [ + "url = \"https://earth-search.aws.element84.com/v1/\"\n", + "collection = \"sentinel-2-l2a\"\n", + "time_range = \"2025-08-15/2025-08-31\"\n", + "bbox = [-95.9912, 46.6704, -95.834, 46.7469]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7c6f1aac", + "metadata": {}, + "outputs": [], + "source": [ + "search = leafmap.stac_search(\n", + " url=url,\n", + " max_items=10,\n", + " collections=[collection],\n", + " bbox=bbox,\n", + " datetime=time_range,\n", + " query={\"eo:cloud_cover\": {\"lt\": 10}},\n", + " sortby=[{\"field\": \"properties.eo:cloud_cover\", \"direction\": \"asc\"}],\n", + " get_collection=True,\n", + ")\n", + "search" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6b189fd0", + "metadata": {}, + "outputs": [], + "source": [ + "search = leafmap.stac_search(\n", + " url=url,\n", + " max_items=10,\n", + " collections=[collection],\n", + " bbox=bbox,\n", + " datetime=time_range,\n", + " query={\"eo:cloud_cover\": {\"lt\": 10}},\n", + " sortby=[{\"field\": \"properties.eo:cloud_cover\", \"direction\": \"asc\"}],\n", + " get_gdf=True,\n", + ")\n", + "search.head()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "51469688", + "metadata": {}, + "outputs": [], + "source": [ + "search = leafmap.stac_search(\n", + " url=url,\n", + " max_items=1,\n", + " collections=[collection],\n", + " bbox=bbox,\n", + " datetime=time_range,\n", + " query={\"eo:cloud_cover\": {\"lt\": 10}},\n", + " sortby=[{\"field\": \"properties.eo:cloud_cover\", \"direction\": \"asc\"}],\n", + " get_assets=True,\n", + ")\n", + "search" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "3052bd35", + "metadata": {}, + "outputs": [], + "source": [ + "bands = [\"blue\", \"green\", \"red\", \"nir\", \"swir16\", \"swir22\"]\n", + "assets = list(search.values())[0]\n", + "links = [assets[band] for band in bands]\n", + "for link in links:\n", + " print(link)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "06d88763", + "metadata": {}, + "outputs": [], + "source": [ + "out_dir = \"s2\"\n", + "leafmap.download_files(links, out_dir)" + ] + }, + { + "cell_type": "markdown", + "id": "805ae2ff", + "metadata": {}, + "source": [ + "### Stack image bands\n", + "\n", + "Uncomment the following cell to install GDAL on Colab." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "0fc0fbee", + "metadata": {}, + "outputs": [], + "source": [ + "# !apt-get install -y gdal-bin" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d72f0e48", + "metadata": {}, + "outputs": [], + "source": [ + "s2_path = \"s2.tif\"\n", + "\n", + "try:\n", + " if not os.path.exists(s2_path):\n", + " geoai.stack_bands(input_files=out_dir, output_file=s2_path)\n", + "except Exception as e:\n", + " print(e)\n", + " url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/s2-minnesota-2025-08-31-subset.tif\"\n", + " geoai.download_file(url, output_path=s2_path)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "e8335082", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(\n", + " s2_path, indexes=[4, 3, 2], vmin=0, vmax=5000, layer_name=\"Sentinel-2\"\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "d05e81bc", + "metadata": {}, + "source": [ + "### Run inference on a Sentinel-2 image" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "38c13853", + "metadata": {}, + "outputs": [], + "source": [ + "s2_mask = \"s2_mask.tif\"\n", + "model_path = f\"{tiles_dir}/unet_models/best_model.pth\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d9bffd0f", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.semantic_segmentation(\n", + " input_path=s2_path,\n", + " output_path=s2_mask,\n", + " model_path=model_path,\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " num_channels=6,\n", + " num_classes=2,\n", + " window_size=512,\n", + " overlap=256,\n", + " batch_size=32,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "dcc47f60", + "metadata": {}, + "source": [ + "### Visualize the results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "826a5615", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(\n", + " s2_mask,\n", + " no_data=0,\n", + " colormap=\"binary\",\n", + " layer_name=\"Water\",\n", + " basemap=s2_path,\n", + " basemap_args={\"indexes\": [4, 3, 2], \"vmin\": 0, \"vmax\": 5000},\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "98f3e0a0", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.empty_cache()" + ] + }, + { + "cell_type": "markdown", + "id": "76b22dc3", + "metadata": {}, + "source": [ + "## Surface water mapping with aerial imagery\n", + "\n", + "In this section, we'll demonstrate surface water mapping using **aerial imagery** from the USDA National Agriculture Imagery Program ([NAIP](https://naip-usdaonline.hub.arcgis.com/)). This represents the highest spatial resolution imagery commonly available for large-scale applications.\n", + "\n", + "**NAIP imagery characteristics:**\n", + "\n", + "**What is NAIP?**\n", + "\n", + "- **USDA Program**: National Agriculture Imagery Program providing high-resolution aerial photography\n", + "- **Coverage**: Continental United States with comprehensive coverage\n", + "- **Spatial resolution**: 1-meter pixels (compared to 10-20m for Sentinel-2)\n", + "- **Spectral bands**: Red, Green, Blue, Near-Infrared (4 channels)\n", + "- **Acquisition frequency**: Updated every 2-3 years for each area\n", + "- **Public availability**: Free access through USGS and other data portals\n", + "\n", + "### Download sample data\n", + "\n", + "If you are interested in downloading NAIP imagery for your area of interest, check out this notebook [here](https://opengeoai.org/examples/download_naip).\n", + "\n", + "To save time, we'll use a curated NAIP dataset with pre-processed training and testing imagery, including water body masks for model training and evaluation:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "78583715", + "metadata": {}, + "outputs": [], + "source": [ + "train_raster_url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip/naip_water_train.tif\"\n", + "train_masks_url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip/naip_water_masks.tif\"\n", + "test_raster_url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip/naip_water_test.tif\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4af3b74a", + "metadata": {}, + "outputs": [], + "source": [ + "train_raster_path = geoai.download_file(train_raster_url)\n", + "train_masks_path = geoai.download_file(train_masks_url)\n", + "test_raster_path = geoai.download_file(test_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f21f3880", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.print_raster_info(train_raster_path, show_preview=False)" + ] + }, + { + "cell_type": "markdown", + "id": "c774176d", + "metadata": {}, + "source": [ + "### Visualize sample data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a62f6cf4", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(train_masks_url, nodata=0, opacity=0.5, basemap=train_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "91f5fa18", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(test_raster_url)" + ] + }, + { + "cell_type": "markdown", + "id": "f278905c", + "metadata": {}, + "source": [ + "### Create training data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "4e74d86f", + "metadata": {}, + "outputs": [], + "source": [ + "out_folder = \"naip\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fd16cb75", + "metadata": {}, + "outputs": [], + "source": [ + "tiles = geoai.export_geotiff_tiles(\n", + " in_raster=train_raster_path,\n", + " out_folder=out_folder,\n", + " in_class_data=train_masks_path,\n", + " tile_size=512,\n", + " stride=256,\n", + " buffer_radius=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "b89ea6f7", + "metadata": {}, + "source": [ + "### Train segmentation model\n", + "\n", + "Similar to the previous example, we'll train a U-Net model on the NAIP dataset." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fcd9a2dc", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.train_segmentation_model(\n", + " images_dir=f\"{out_folder}/images\",\n", + " labels_dir=f\"{out_folder}/labels\",\n", + " output_dir=f\"{out_folder}/models\",\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " encoder_weights=\"imagenet\",\n", + " num_channels=4,\n", + " batch_size=8,\n", + " num_epochs=3,\n", + " learning_rate=0.005,\n", + " val_split=0.2,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "5091c2c3", + "metadata": {}, + "source": [ + "### Evaluate the model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "de2e3af1", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.plot_performance_metrics(\n", + " history_path=f\"{out_folder}/models/training_history.pth\",\n", + " figsize=(15, 5),\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "21a86b86", + "metadata": {}, + "source": [ + "### Run inference" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f7301d99", + "metadata": {}, + "outputs": [], + "source": [ + "masks_path = \"naip_water_prediction.tif\"\n", + "model_path = f\"{out_folder}/models/best_model.pth\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "200fe9b6", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.semantic_segmentation(\n", + " test_raster_path,\n", + " masks_path,\n", + " model_path,\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " encoder_weights=\"imagenet\",\n", + " window_size=512,\n", + " overlap=128,\n", + " batch_size=32,\n", + " num_channels=4,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "16f05ccb", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(\n", + " masks_path,\n", + " nodata=0,\n", + " layer_name=\"Water\",\n", + " basemap=test_raster_url,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "048ae031", + "metadata": {}, + "source": [ + "### Vectorize masks\n", + "\n", + "We can convert the raster predictions to vector features for further analysis." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "fd78e9f6", + "metadata": {}, + "outputs": [], + "source": [ + "output_path = \"naip_water_prediction.geojson\"\n", + "gdf = geoai.raster_to_vector(\n", + " masks_path, output_path, min_area=1000, simplify_tolerance=1\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "82ee5009", + "metadata": {}, + "outputs": [], + "source": [ + "gdf = geoai.add_geometric_properties(gdf)\n", + "len(gdf)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "49b84274", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_vector_interactive(gdf, tiles=test_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d84927ec", + "metadata": {}, + "outputs": [], + "source": [ + "gdf[\"elongation\"].hist()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "98ff7d32", + "metadata": {}, + "outputs": [], + "source": [ + "gdf_filtered = gdf[gdf[\"elongation\"] < 10]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "f1ae2cb9", + "metadata": {}, + "outputs": [], + "source": [ + "len(gdf_filtered)" + ] + }, + { + "cell_type": "markdown", + "id": "82803b3a", + "metadata": {}, + "source": [ + "### Visualize results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "331bc4c9", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_vector_interactive(gdf_filtered, tiles=test_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "28fca6ca", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.create_split_map(\n", + " left_layer=gdf_filtered,\n", + " right_layer=test_raster_url,\n", + " left_args={\"style\": {\"color\": \"red\", \"fillOpacity\": 0.2}},\n", + " basemap=test_raster_url,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "2f93b1f7", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.empty_cache()" + ] + }, + { + "cell_type": "markdown", + "id": "c0f4b4c1", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/a269b5a0-9f72-4ed8-8b2d-a175bbc45a23)\n", + "\n", + "## Building detection with aerial imagery\n", + "\n", + "### Download sample data\n", + "\n", + "If you are interested in downloading NAIP imagery and Overture Maps data for your area of interest, check out this notebook [here](https://opengeoai.org/examples/download_data).\n", + "\n", + "To save time, we'll use a curated NAIP dataset and building footprints for model training and evaluation:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1908669a", + "metadata": {}, + "outputs": [], + "source": [ + "train_raster_url = (\n", + " \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_rgb_train.tif\"\n", + ")\n", + "train_vector_url = \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_train_buildings.geojson\"\n", + "test_raster_url = (\n", + " \"https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_test.tif\"\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "723bc4af", + "metadata": {}, + "outputs": [], + "source": [ + "train_raster_path = geoai.download_file(train_raster_url)\n", + "train_vector_path = geoai.download_file(train_vector_url)\n", + "test_raster_path = geoai.download_file(test_raster_url)" + ] + }, + { + "cell_type": "markdown", + "id": "513eea91", + "metadata": {}, + "source": [ + "### Visualize sample data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "d2ed3a24", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.get_raster_info(train_raster_path)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "34b2b433", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_vector_interactive(train_vector_path, tiles=train_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "c43f3d5f", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(test_raster_url)" + ] + }, + { + "cell_type": "markdown", + "id": "b9852e33", + "metadata": {}, + "source": [ + "### Create training data\n", + "\n", + "We'll create the same training tiles as before." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "a31e3ebf", + "metadata": {}, + "outputs": [], + "source": [ + "out_folder = \"buildings\"\n", + "tiles = geoai.export_geotiff_tiles(\n", + " in_raster=train_raster_path,\n", + " out_folder=out_folder,\n", + " in_class_data=train_vector_path,\n", + " tile_size=512,\n", + " stride=256,\n", + " buffer_radius=0,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "7297c229", + "metadata": {}, + "source": [ + "### Train semantic segmentation model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "628b12d2", + "metadata": {}, + "outputs": [], + "source": [ + "# Train U-Net model\n", + "geoai.train_segmentation_model(\n", + " images_dir=f\"{out_folder}/images\",\n", + " labels_dir=f\"{out_folder}/labels\",\n", + " output_dir=f\"{out_folder}/unet_models\",\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " encoder_weights=\"imagenet\",\n", + " num_channels=3,\n", + " num_classes=2, # background and building\n", + " batch_size=8,\n", + " num_epochs=3,\n", + " learning_rate=0.001,\n", + " val_split=0.2,\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "e603dcf5", + "metadata": {}, + "source": [ + "### Evaluate the model\n", + "\n", + "Let's examine the training curves and model performance:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "11870233", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.plot_performance_metrics(\n", + " history_path=f\"{out_folder}/unet_models/training_history.pth\",\n", + " figsize=(15, 5),\n", + " verbose=True,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "109f730b", + "metadata": {}, + "source": [ + "![image](https://github.com/user-attachments/assets/ac7c997e-f8b7-404f-9ac7-b56c78fb34ed)\n", + "\n", + "### Run inference\n", + "\n", + "Now we'll use the trained model to make predictions on the test image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "20af0679", + "metadata": {}, + "outputs": [], + "source": [ + "masks_path = \"naip_test_semantic_prediction.tif\"\n", + "model_path = f\"{out_folder}/unet_models/best_model.pth\"" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "6bf1713e", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.semantic_segmentation(\n", + " input_path=test_raster_path,\n", + " output_path=masks_path,\n", + " model_path=model_path,\n", + " architecture=\"unet\",\n", + " encoder_name=\"resnet34\",\n", + " num_channels=3,\n", + " num_classes=2,\n", + " window_size=512,\n", + " overlap=256,\n", + " batch_size=4,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "f4a98209", + "metadata": {}, + "source": [ + "### Visualize raster masks" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "7f42c16b", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_raster(\n", + " masks_path,\n", + " nodata=0,\n", + " colormap=\"binary\",\n", + " basemap=test_raster_url,\n", + ")" + ] + }, + { + "cell_type": "markdown", + "id": "7f2b61ce", + "metadata": {}, + "source": [ + "### Vectorize masks\n", + "\n", + "Convert the predicted mask to vector format for better visualization and analysis." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9aab6787", + "metadata": {}, + "outputs": [], + "source": [ + "output_vector_path = \"naip_test_semantic_prediction.geojson\"\n", + "gdf = geoai.orthogonalize(masks_path, output_vector_path, epsilon=2)" + ] + }, + { + "cell_type": "markdown", + "id": "01534d0d", + "metadata": {}, + "source": [ + "### Add geometric properties" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "088d176f", + "metadata": {}, + "outputs": [], + "source": [ + "gdf_props = geoai.add_geometric_properties(gdf, area_unit=\"m2\", length_unit=\"m\")" + ] + }, + { + "cell_type": "markdown", + "id": "673a93f2", + "metadata": {}, + "source": [ + "### Visualize results" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "1d085acb", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_vector_interactive(gdf_props, column=\"area_m2\", tiles=test_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "befdbecb", + "metadata": {}, + "outputs": [], + "source": [ + "gdf_filtered = gdf_props[(gdf_props[\"area_m2\"] > 10)]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "9b02e151", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.view_vector_interactive(gdf_filtered, column=\"area_m2\", tiles=test_raster_url)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "99ea35ce", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.create_split_map(\n", + " left_layer=gdf_filtered,\n", + " right_layer=test_raster_url,\n", + " left_args={\"style\": {\"color\": \"red\", \"fillOpacity\": 0.2}},\n", + " basemap=test_raster_url,\n", + ")" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "id": "55abc329", + "metadata": {}, + "outputs": [], + "source": [ + "geoai.empty_cache()" + ] + }, + { + "cell_type": "markdown", + "id": "7257c3f8", + "metadata": {}, + "source": [ + "## Summary and Next Steps\n", + "\n", + "Congratulations! You've successfully completed a comprehensive introduction to the GeoAI package. Let's review what we accomplished and explore pathways for advancing your GeoAI skills.\n", + "\n", + "### What We Accomplished\n", + "\n", + "**1. Multi-scale Water Mapping Workflows:**\n", + "\n", + "- **RGB Imagery**: Trained models on standard satellite imagery (JPG format)\n", + "- **Multispectral Sentinel-2**: Leveraged 6 spectral bands for enhanced discrimination\n", + "- **High-resolution NAIP**: Utilized 1-meter aerial imagery for detailed mapping\n", + "\n", + "**2. Deep Learning Fundamentals:**\n", + "\n", + "- **U-Net Architecture**: Applied state-of-the-art segmentation models\n", + "- **Transfer Learning**: Leveraged ImageNet pre-trained weights for faster convergence\n", + "- **Multispectral Processing**: Handled various spectral configurations (3, 4, and 6 channels)\n", + "\n", + "**3. Operational Workflows:**\n", + "\n", + "- **Data Acquisition**: Downloaded and processed real satellite and aerial imagery\n", + "- **Model Training**: Trained custom models for different imagery types\n", + "- **Performance Evaluation**: Assessed model quality using IoU and Dice metrics\n", + "- **Batch Processing**: Applied models to multiple images efficiently\n", + "- **Vector Conversion**: Transformed predictions into GIS-ready polygon features\n", + "\n", + "**4. Real-world Applications:**\n", + "\n", + "- **Data Preprocessing**: Handled various geospatial data formats and projections\n", + "- **Quality Assessment**: Filtered results based on geometric properties\n", + "- **Interactive Visualization**: Created interactive maps for exploring results\n", + "\n", + "### Thank You!\n", + "\n", + "Thank you for participating in this **GeoAI workshop**! The techniques demonstrated here represent just the beginning of what's possible when combining artificial intelligence with geospatial analysis. The field of GeoAI is rapidly evolving, offering exciting opportunities to address real-world challenges in environmental monitoring, urban planning, agriculture, and climate science.\n", + "\n", + "**Keep exploring, keep learning, and keep pushing the boundaries of what's possible with GeoAI!**\n", + "\n", + "For questions, feedback, or collaboration opportunities, please visit the [GeoAI GitHub repository](https://github.com/opengeos/geoai)." + ] + } + ], + "metadata": { + "authors": [ + { + "affiliations": [ + "University of Tennessee, Knoxville" + ], + "email": "qwu18@utk.edu", + "github": "giswqs", + "name": "Qiusheng Wu", + "orcid": "0000-0001-5437-4073" + } + ], + "execute": false, + "jupytext": { + "default_lexer": "ipython3" + }, + "language_info": { + "name": "python" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +} diff --git a/modules/03-integrating-ai/index.md b/modules/03-integrating-ai/index.md deleted file mode 100644 index f36cdbc..0000000 --- a/modules/03-integrating-ai/index.md +++ /dev/null @@ -1,26 +0,0 @@ ---- -authors: - - name: "Qiusheng Wu" - affiliations: - - "University of Tennessee, Knoxville" - email: "qwu18@utk.edu" - orcid: "0000-0001-5437-4073" - github: "giswqs" ---- - -# 🤖 3 - Integrating artificial intelligence (AI) with geospatial data analysis with [GeoAI](https://geoai.gishub.org/) - -:::{note} 🛝 Slides -:icon: false -:class: dropdown - - -:::