Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation (CVPR 2026)

PyTorch implementation and pretrained models for the method DEO, from the paper Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation.

Authors: Filip Wolf, Blaž Rolih, Luka Čehovin Zajc.

[Paper] [Project Page] [BibTeX]

Update

[2-3-2026] We release pretrained weights and weight loading.
[26-2-2026] We release pretraining code for DEO.

Pretrained models

The Swin-b model used for the results in the paper can be found on OneDrive and Hugging Face. ViT-based and larger models are coming soon.

Installation

Our implementation requires Python 3.11+, PyTorch 2.4+ and xFormers 0.0.29+ and some other packages. Note that the code has only been tested with the specified versions and also expects a Linux environment. To setup the dependencies, please install via:

conda env create -f environment.yml

Data preparation

We use a mix of fMoW-Sentinel and fMoW-RGB. Refer to the paper for more details.

Training

To train our model, simply run:

python -m torch.distributed.run --nproc_per_node 4 --nnodes 1 --master_port 1234 DEO/main_dino.py --arch swin_t --batch_size 64 --data_split 500000 --local_crops_number 10 --eps 0.05 --output_dir output/[output_dir] --data_path ../pretraining_dataset.h5 --ms_arch --dist_arch dinov3 --dist_arch_size base

We compress our dataset into the .h5 format, and our dataloader expects such a class. However, adding your own dataset loader can be easily configured in utils/datasets.py

Evaluation

To evaluate DEO, simply load features from DEO_weights.py. The model takes as input either 3 channel RGB data or 10 channel S2 multispectral (no 60m atmospheric bands).

import torch

from DEO_weights import DEO

model = "swin_b"
path = "/path/to/features/"

model = DEO(model, path, "cuda")

image = torch.randn(1, 3, 256, 256, dtype=torch.float)
image = image.to("cuda")

features = model(image)
print([f.shape for f in features])

Citing DEO

If you like what we do, please consider citing us:

@article{wolf2026brewing,
  title={Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation},
  author={Wolf, Filip and Rolih, Blaž and Čehovin Zajc, Luka},
  journal={arXiv preprint arXiv:2602.19863},
  year={2026},
  url={https://arxiv.org/abs/2602.19863}
}

Acknowledgements

This project is largely built upon SimDINO, which was in turn built on the original DINO and DINOv2 projects.