DeepOT: Deep Learning-based Displacement Estimation for SAR Amplitude Images

DeepOT is a deep learning-based tool for estimating pixel-wise displacement between pairs of SAR (Synthetic Aperture Radar) amplitude images using U-Net and U-Net++ architectures.

Features

• U-Net Architecture: Encoder-decoder network with skip connections for accurate displacement estimation
• U-Net++ Architecture: Nested dense skip pathways for multi-scale feature learning
• Patch-based Processing: Handles images of any size through intelligent patch extraction and merging
• Smooth Blending: Weighted cosine blending eliminates artifacts at patch boundaries
• Cross-TF-Version Compatibility: Robust weight loading with automatic fallback for different TensorFlow versions
• Easy-to-Use API: Simple Python interface and command-line tool

Example Output

Displacement prediction for Slumgullion Landslide, Colorado using U-Net. From left to right: Reference amplitude, Secondary amplitude, Y-direction displacement, and X-direction displacement overlaid on amplitude basemap.

Installation

From PyPI

pip install deepot

See the deepot package on PyPI.

From GitHub

git clone https://github.com/smuinsar/DeepOT.git
cd DeepOT
pip install -e .

Requirements

• Python >= 3.8
• TensorFlow >= 2.10.0
• NumPy >= 1.19.0
• Matplotlib >= 3.3.0
• h5py >= 3.0.0

Quick Start

Python API

from deepot import DeepOTPredictor
import numpy as np

# Load your amplitude images (normalized to [0, 1])
ref_image = np.load('reference.npy')
sec_image = np.load('secondary.npy')

# U-Net prediction
predictor = DeepOTPredictor(model_path='model/UNet.h5')
disp_x, disp_y = predictor.predict(ref_image, sec_image)

# U-Net++ prediction
predictor = DeepOTPredictor(
    model_path='model/UNetPlusPlus.h5',
    model_type='unetpp',
)
disp_x, disp_y = predictor.predict(ref_image, sec_image)

# disp_x: X-direction displacement in pixels
# disp_y: Y-direction displacement in pixels

Command Line

# U-Net (default)
deepot-predict --ref-image inputs/reference.npy \
               --sec-image inputs/secondary.npy \
               --output-dir predictions

# U-Net++
deepot-predict --ref-image inputs/reference.npy \
               --sec-image inputs/secondary.npy \
               --model-type unetpp \
               --output-dir predictions

Visualization

from deepot import visualize_displacement, visualize_displacement_with_basemap

# Basic 2x2 grid visualization
visualize_displacement(
    disp_x, disp_y,
    ref_image=ref_image,
    sec_image=sec_image,
    vmin=-0.5, vmax=0.5,
    save_path='displacement.png'
)

# 1x4 layout with amplitude basemap overlay on displacement
visualize_displacement_with_basemap(
    disp_x, disp_y,
    ref_image=ref_image,
    sec_image=sec_image,
    vmin=-0.5, vmax=0.5,
    alpha=0.4,
    save_path='displacement_overlay.png'
)

Input Data Format

The model expects normalized amplitude images as 2D NumPy arrays:

• Shape: (height, width) - any size >= 256x256
• Data type: float32
• Value range: Normalized to [0, 1]
• File format: .npy (NumPy binary format)

Preparing Your Data

import numpy as np

# Load your amplitude data
amplitude = np.load('your_amplitude.npy')

# Normalize to [0, 1] range
amplitude = amplitude.astype(np.float32)
amplitude = (amplitude - amplitude.min()) / (amplitude.max() - amplitude.min())

# Save normalized data
np.save('normalized_amplitude.npy', amplitude)

Output

The predictor returns two displacement arrays:

Output	Description	Units
disp_x	X-direction displacement	pixels
disp_y	Y-direction displacement	pixels

Converting to Meters

from deepot.utils import pixels_to_meters

# Convert pixels to meters using your pixel spacing
pixel_spacing = 10.0  # meters per pixel
disp_x_meters = pixels_to_meters(disp_x, pixel_spacing)
disp_y_meters = pixels_to_meters(disp_y, pixel_spacing)

Examples

See the examples/ directory for Jupyter notebooks demonstrating complete workflows:

• displacement_prediction_Slum.ipynb: Slumgullion Landslide displacement prediction
• displacement_prediction_Kenn.ipynb: Kennicott Glacier displacement prediction

Both notebooks support switching between U-Net and U-Net++ by changing MODEL_TYPE.

Example Data

The npy/ directory contains example amplitude image pairs:

Dataset	Reference	Secondary	Description
Slumgullion	Slum_20130510.npy	Slum_20131025.npy	Slumgullion Landslide, Colorado
Kennicott	Kennicott_20180611.npy	Kennicott_20180810.npy	Kennicott Glacier, Alaska

Model Architectures

U-Net

Standard encoder-decoder with skip connections.

• Input: Two amplitude images concatenated along channel dimension
• Encoder: 5 levels with filters [64, 128, 256, 512, 1024]
• Decoder: Symmetric decoder with skip connections
• Output: 2-channel displacement map (x and y directions)

Input (256x256x2) → Encoder → Bottleneck → Decoder → Output (256x256x2)
                     ↓           ↓           ↑
                  Skip Connections ──────────┘

U-Net++

Nested dense skip pathways for multi-scale feature learning.

• Input: Two amplitude images concatenated along channel dimension
• Encoder: 6 levels with filters [32, 64, 128, 256, 512, 1024] (large)
• Decoder: Dense skip pathways connecting all encoder levels
• Output: 2-channel displacement map (x and y directions)

Input (256x256x2) → Encoder → Bottleneck → Decoder → Output (256x256x2)
                     ↓           ↓           ↑
                  Dense Skip Pathways ──────┘
                  (nested connections at each level)

Additional Model Architectures (share_models/)

The share_models/ directory contains reference implementations of other displacement/optical-flow architectures for comparison and research. These are standalone Keras model definitions and are not wired into DeepOTPredictor (which currently supports unet and unetpp only).

File	Model	Description
flownet2.py	FlowNet2	Full FlowNet 2.0 (Ilg et al., CVPR 2017) with correlation layer, image warping, FlowNet-CSS stacking, FlowNet-SD for small displacements, and a fusion network. Factory: create_flownet2_standard().
crn.py	CC-ResSiamNet	Cross-Correlation Residual Siamese Network: Siamese encoder-decoder with OTResBlock residual units that process both images jointly, cross-attention skip connections, and spectral + spatial attention. Factory: create_ccressiamnet().
displacedcn.py	DisplaceDCN	Efficient Deformable Convolutional Network: dual-path encoder-decoder with memory-efficient deformable convolutions, cross-connections, attention, and an output head supporting optional uncertainty estimation. Factory: create_displacedcn().

API Reference

DeepOTPredictor

DeepOTPredictor(
    model_path: str,           # Path to trained model weights
    model_type: str = 'unet',  # 'unet' or 'unetpp'
    patch_size: int = 256,     # Size of processing patches
    overlap: int = 128,        # Overlap between patches
    batch_size: int = 32,      # Batch size for inference
    filters: list = None,      # Filter config (auto-selected if None)
)

Default filters by model type:

• unet: [64, 128, 256, 512, 1024]
• unetpp: [32, 64, 128, 256, 512, 1024]

Methods

• predict(ref_image, sec_image) - Predict displacement from arrays
• predict_from_files(ref_path, sec_path) - Predict from .npy file paths

Visualization Functions

• visualize_displacement() - 2x2 grid with inputs and displacement outputs
• visualize_displacement_with_basemap() - 1x4 layout with displacement overlaid on amplitude basemap
• create_displacement_figure() - X, Y, and magnitude with optional basemap overlay
• plot_displacement_histogram() - Distribution of displacement values

Citation

If you use DeepOT in your research, please cite:

@software{deepot2026,
  title={DeepOT: Deep Learning-based Optical Flow Estimation for SAR Images},
  author={Kim, Jinwoo},
  year={2026},
  url={https://github.com/smuinsar/DeepOT}
}

License

This project is licensed under the MIT License - see the LICENSE file for details.

Author

• Jinwoo Kim: jinwook@smu.edu