AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis

Khiem Vuong, Anurag Ghosh, Deva Ramanan*, Srinivasa Narasimhan*, Shubham Tulsiani*

CVPR 2025

[arXiv] [Project Page] [3D Web Viewer] [Bibtex]

📢 News

• (Sep 5, 2025): We have moved the full dataset to HuggingFace to make it easier to download. Please refer to data_generation for full instructions.

Table of Contents

• Installation
• Quick Start
• Evaluation
• Data Generation
• Acknowledgement
• Citation
• Issues

📋 Before You Start...

Depending on what you're interested in:

• 🚀 If you want to download pretrained checkpoints and run the demo: you can continue reading below.

• 🛠️ If you want to access or generate the dataset yourself: please refer to data_generation for full instructions.

Installation

Below are the setup instructions (mostly following MASt3R repo). We follow the MASt3R setup since it encapsulates DUSt3R, but both should work similarly.

1. Clone the repository:

git clone --recursive https://github.com/kvuong2711/aerial-megadepth.git
cd aerial-megadepth/mast3r

# If you already cloned the repository, you can update the submodules:
# git submodule update --init --recursive

2. Create environment and install dependencies:

conda create -n aerialmd python=3.11 cmake=3.14.0
conda activate aerialmd 
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia  # use the correct version of cuda for your system
pip install -r requirements.txt
pip install -r dust3r/requirements.txt
pip install -r dust3r/requirements_optional.txt

3. Optional, compile the cuda kernels for RoPE (as in CroCo v2):

# DUST3R relies on RoPE positional embeddings for which you can compile some cuda kernels for faster runtime.
cd dust3r/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../

Quick Start

Our finetuned checkpoints are fully compatible with the original DUSt3R/MASt3R/MASt3R-SfM codebase - if you already have them set up, you can simply swap the checkpoint for aerial-ground scenarios!

Checkpoints

There are two options for downloading DUSt3R and MASt3R checkpoint finetuned on our AerialMegaDepth dataset:

1. You can use our huggingface_hub integration: the models (aerial-dust3r or aerial-mast3r) will be downloaded automatically.

2. Or, download the checkpoints manually (using gdown, install using pip install gdown):

# you are inside aerial-megadepth/mast3r
mkdir -p checkpoints/
gdown --fuzzy "https://drive.google.com/open?id=1wSGpYwWeGn99J8dVWNkfefwmWMAH7LFT" -O checkpoints/  # checkpoint-aerial-dust3r.pth
gdown --fuzzy "https://drive.google.com/open?id=1LrRNUQRQZcVzcioyYHYYx9ImypSZpUq2" -O checkpoints/  # checkpoint-aerial-mast3r.pth

Inference/Demo

We provide a few example images in the assets folder for quick testing, including images from ULTRRA challenge and Accenture-NVS1.

Each script (e.g., demo_dust3r_nongradio.py) contains predefined image paths to demonstrate typical use cases. You can modify the image_list variable inside the script to try different pairs. You can run inference using the following commands:

• DUSt3R demo code:

python demo_dust3r_nongradio.py --weights kvuong2711/checkpoint-aerial-dust3r

# or, if you downloaded the checkpoints manually:
# python demo_dust3r_nongradio.py --weights checkpoints/checkpoint-aerial-dust3r.pth

• MASt3R demo code:

python demo_mast3r_nongradio.py --weights kvuong2711/checkpoint-aerial-mast3r

# or, if you downloaded the checkpoints manually:
# python demo_mast3r_nongradio.py --weights checkpoints/checkpoint-aerial-mast3r.pth

Evaluation

• TODO: Evaluation data will be released soon.

Once data is downloaded, run the evaluation script:

python eval.py \
    --weights checkpoints/checkpoint-aerial-dust3r.pth \
    --eval_data_dir data/eval_data_release

This script (adapted from PoseDiffusion) reports RRA and RTA at various degree thresholds.

Data Generation

For instructions on how to download and/or generate the data, please refer to data_generation.

Acknowledgement

This codebase builds upon many excellent open-source projects, such as MegaDepth, DUSt3R, hloc, COLMAP, etc. We thank the respective authors for making their work publicly available.

Citation

If you find our work to be useful in your research, please consider citing our paper:

@inproceedings{vuong2025aerialmegadepth,
  title={AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis},
  author={Vuong, Khiem and Ghosh, Anurag and Ramanan, Deva and Narasimhan, Srinivasa and Tulsiani, Shubham},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2025},
}

Issues

If you have any problem/question/suggestion, feel free to create an issue or reach out directly to me via email.