Learning the Target Priors Before Image Translation (LTP-BIT)

A Decoupled Training Paradigm for Cross-Modal Image Translation in Remote Sensing

Keyan Hu^1*, Mingtao Wang^1*, Ziyu Zhou², Tiandong Shi¹, Haifeng Li¹, Ji Qi^3†, Chao Tao^1†

¹ Central South University    ² Wuhan University    ³ Guangzhou University

^* Equal contribution    ^† Corresponding authors

Overview

Remote-sensing cross-modal translation has to solve two different problems at once: learning what realistic target images look like, and learning how a source observation should control the generated result. Standard paired training learns both from the same limited set of aligned samples.

LTP-BIT separates them. It first learns a target-domain generative prior from scalable, unpaired RGB imagery. The pretrained backbone is then kept fixed while P-DART learns source-conditioned control from paired data. This prior-first design lets the model spend unpaired data on target realism and reserve paired samples for cross-modal correspondence.

Our experiments cover single-polarization SAR-to-RGB, multi-polarization SAR-to-RGB, and NIR-to-RGB translation. The final model uses 9.81% task-specific parameters. On QXS-SAROPT, it retains near-full-data instance fidelity with 25% of the paired training samples.

Model architecture

P-DART: controlling a fixed target prior

P-DART controls a pretrained DiT without rewriting the target prior learned from unpaired RGB images. It adds a trainable reference stream for the source image. At each block, asymmetric attention passes information from this stream into the frozen generation stream. During paired training, only P-DART and the backbone LoRA adapters are updated; the DiT backbone and DDT head stay fixed.

Component	Role during paired adaptation	Updated?
Target-prior backbone	Carries the target-domain generative field learned from unpaired RGB images	No
P-DART reference stream	Reads the source modality and injects source-dependent control	Yes
Backbone LoRA	Corrects the remaining mismatch between the pretrained prior and task target	Yes
DDT head	Decodes the adapted latent representation	No

Results

We evaluate LTP-BIT on three sensor pairs: single-polarization SAR to RGB, multi-polarization SAR to RGB, and NIR to RGB. Higher PSNR and SSIM indicate closer instance reconstruction. Lower LPIPS, FID, and CMMD indicate smaller perceptual or distributional differences.

Benchmark	Translation	PSNR ↑	SSIM ↑	LPIPS ↓	FID ↓	CMMD ↓
QXS-SAROPT	single-pol. SAR → RGB	15.956	0.351	0.446	16.54	0.201
SpaceNet6	multi-pol. SAR → RGB	18.924	0.353	0.293	43.76	0.808
Chesapeake	NIR → RGB	17.717	0.350	0.290	18.19	0.825
Bold values mark the best result among the methods compared in the paper.

_{Extended qualitative comparisons on QXS-SAROPT, grouped by scene type. See the paper for metric definitions and ablations.}

Getting started

Use Linux, Python 3.10, and NVIDIA GPUs with BF16 support. Our paper experiments ran on four NVIDIA RTX A6000 GPUs.

git clone https://github.com/KeyanHu-git/LTP-BIT.git
cd LTP-BIT

conda create -n ltp-bit python=3.10 -y
conda activate ltp-bit
pip install -r requirements.txt

The requirements file pins PyTorch 2.8.0 and torchvision 0.23.0. Before starting a multi-GPU run, check that your driver and CUDA runtime support this PyTorch build.

Data & checkpoints

The paired-data loader matches source and target images by relative path. If either side of a pair is missing, the loader raises an error instead of dropping the sample.

Dataset	Train	Test	Expected folders
QXS-SAROPT	16,000	4,000	{train,test}/{sar,opt}
SpaceNet6	20,168	5,048	{train,test}/{sar,opt}
Chesapeake	16,000	4,000	{train,test}/{nir,rgb}

Stage 2 uses RS-1M for target-prior pretraining. We selected one million 256 x 256 RGB images from Git-10M, screened low-quality samples, removed duplicates, and kept the final set diverse. The supplement describes the selection procedure.

Checkpoint files are not stored in Git. Download links will be added after the checkpoint package has been checked and uploaded; the expected folder names and paths are listed in the reproduction guide.

Project status

The code is public. Pretrained weights are not available yet. We will update this list as the repository changes.

• Training, inference, and evaluation code
• Reproduction configs for all three benchmarks
• Pretrained LTP-BIT checkpoints
• Additional examples and usage notes
• Compatibility fixes for future dependency updates

Training

Training runs in three steps: codec adaptation, target-prior pretraining, then paired P-DART adaptation. For QXS-SAROPT, the final step is launched with:

bash scripts/raev2/stage3/train.sh \
  configs/raev2/stage3/train/qxslab_saropt/igxl_s2git1m_ep100_mmdit_encdec_ep80.yaml \
  0,1,2,3

Stage 3 expects the adapted decoder and pretrained target prior named in its YAML file. See the reproduction guide for Stages 1 and 2, alternate datasets, and resume options.

Inference

Set CKPT to a trained Stage 3 checkpoint. The test script generates samples and computes the configured metrics.

CKPT=weights/stage3/qxslab_saropt/igxl_ltp_bit_ep80.pt \
bash scripts/raev2/stage3/test.sh \
  configs/raev2/stage3/test/qxslab_saropt/test_igxl_s2git1m_ep100_mmdit_encdec_ep80.yaml \
  0,1,2,3

SpaceNet6 and Chesapeake use the corresponding YAML files under configs/raev2/stage3/test/. Outputs are saved to the experiment directory set in the selected config.

Citation

If this code or paper helps your work, please cite:

@article{hu2026ltpbit,
  title   = {Learning the Target Priors Before Image Translation:
             A Decoupled Training Paradigm for Cross-Modal Image
             Translation in Remote Sensing},
  author  = {Hu, Keyan and Wang, Mingtao and Zhou, Ziyu and Shi, Tiandong and
             Li, Haifeng and Qi, Ji and Tao, Chao},
  journal = {arXiv preprint arXiv:2608.28517},
  year    = {2026}
}

Acknowledgements

Thanks to the authors of RAE, RAEv2, and DINOv3 for their models and code, and to the teams behind Git-10M, QXS-SAROPT, SpaceNet6, and Chesapeake Land Cover for making the datasets available.