TranSpace: Progressive Anchoring for Metric-Consistent Scene Synthesis

TranSpace: Progressive Anchoring for Metric-Consistent Scene Synthesis Hyeshim Kim, Taehei Kim*, Jihun Shin*, Hyeonjin Kim, Sung-Hee Lee — *equal contribution Korea Advanced Institute of Science and Technology (KAIST) ACM Multimedia 2026 (MM '26), Rio de Janeiro, Brazil doi:10.1145/3767308.3836316

Important

Non-commercial use only (research or evaluation). See LICENSE and NOTICE for details.

---

What this release contains

This release covers the iterative densification stage. The demo runs it on the playroom–drjohnson scene pair, whose keyframes and intermediate videos are already generated and shipped in the data pack, so the demo replays Progressive Anchoring without any generative model or API key:

1. Progressive Anchoring: bidirectional PnP localization with quality gating, iterative densification, and final closing
2. NVS reconstruction: trains a sparse-voxel model on the localized frames
3. Final shot: renders a smooth arc fly-through

Not included: keyframe synthesis stage, generation on new scenes, and the ScanNet++ scenes used in the paper's quantitative tables.

Requirements

Component	Requirement
OS	Linux (tested on Ubuntu 20.04)
GPU	NVIDIA, compute capability ≥ 8.0, ≥ 24 GB VRAM. Tested on RTX 3090
CUDA	11.7 (RTX 30xx/40xx) or 12.1 (A100/H100)
Disk	~19 GB for the data pack, plus ~10 GB working space per run
conda	Miniconda or Anaconda

Installation

cd transpace
conda create -n transpace python=3.10 -y
conda activate transpace

# PyTorch matched to your CUDA
pip install torch==2.0.1 torchvision==0.15.2 --index-url https://download.pytorch.org/whl/cu117

# CUDA toolkit inside the env (provides nvcc for the extension build)
conda install -c "nvidia/label/cuda-11.7.0" cuda-toolkit -y

# ffmpeg/ffprobe, for video frame extraction
conda install -c conda-forge ffmpeg -y

pip install -r requirements.txt

# Required before building the extensions: torch 2.0.1 imports pkg_resources,
# which setuptools >= 81 no longer ships.
pip install "setuptools<81"

# fused-ssim has no wheel; compile it against your torch
pip install git+https://github.com/rahul-goel/fused-ssim/ --no-build-isolation

# Build the two CUDA rasterization extensions
cd svraster/cuda
export CUDA_HOME=$CONDA_PREFIX
export PATH=$CUDA_HOME/bin:$PATH
python setup.py install            # svraster_cuda
python setup_trspace.py install    # svraster_cuda_trspace (space_id aware)
cd ../..

Verify:

python -c "import torch, svraster_cuda, svraster_cuda_trspace; \
print(torch.__version__, torch.cuda.get_device_name(0))"

On A100/H100, use the CUDA 12.1 build of PyTorch and the matching conda toolkit, and set CC/CXX to GCC 11 before building — CUDA's thrust headers do not compile under GCC 14+.

Data

The data pack is on the Hugging Face Hub: codeshim/transpace-playroom-drjohnson

pip install huggingface_hub

hf download codeshim/transpace-playroom-drjohnson --repo-type dataset \
    --local-dir data/playroom_drjohnson

It unpacks to:

data/playroom_drjohnson/
├── playroom_aligned/                   room A: images/, sparse/, checkpoints/
├── drjohnson_aligned/                  room B: images/, sparse/, checkpoints/
├── BASE_model/                         floor/threshold voxel model
├── trspace_padding_model_type{0,1,2}/  space_id mask model per variant
├── trspace_source_type{0,1,2}/         inpainted keyframes + intermediate videos
└── playroom_drjohnson_alignment_info_type{0,1,2}.json

The pack carries three connector variants for the same room pair: type0 (wing wall), type1 (pony wall), type2 (internal window). Only the RIGHT connector varies; LEFT is an internal window in every variant. A variant's source videos, alignment info and padding model must be used together.

Running the demo

python main.py --variant type0 --view_pair RIGHT	python main.py --variant type1 --view_pair RIGHT	python main.py --variant type2 --view_pair RIGHT	python main.py --variant type2 --view_pair LEFT

Each run starts fresh: outputs from a previous run of the same side are deleted first. Takes ~17 min on an RTX 3090.

Output

output/playroom_drjohnson_type2/
├── RIGHT_FRONT_RIGHT_REAR/     00001.png .. 00051.png
├── playroom_drjohnson_RIGHT_FRONT_RIGHT_REAR_prog_anchor.json
├── playroom_drjohnson_views.json
├── RIGHT_final_model/          trained voxel model
└── RIGHT_final.mp4             arc fly-through

RIGHT_FRONT_RIGHT_REAR/ is a front-to-rear ordered 1024×1024 RGB sequence — the training input for novel view synthesis. prog_anchor.json records the per-frame camera pose, inlier count and reprojection error.

Repository layout

main.py                  entry point; released settings as module constants
config.py                scene selection, variants, and all paths
constants.py             view/region IDs, camera intrinsics, voxel levels
scene/
├── localizer.py         Progressive Anchoring: PnP, quality gates, closing
├── stitcher.py          volumetric blending; NVS training
├── blueprinter.py       transition bounding boxes and keyframe view placement
├── compositor.py        sparse voxel composition
├── renderer.py          rasterization wrapper
├── generator.py         monocular metric depth estimation
└── scene_utils.py       voxel model save/load helpers
utils/
├── timeline.py          front-to-rear ordering and sequence export
├── finalshot.py         arc trajectory, rendering, video encoding
├── depth_alignment_utils.py   metric depth scale/shift alignment
├── io_utils.py          COLMAP, video frame extraction, alignment info
├── camera_utils.py      intrinsics
├── o3d_utils.py         Open3D visualization helpers
└── gpu_utils.py         memory cleanup
svraster/                vendored, MODIFIED SVRaster (see NOTICE)

Citation

@inproceedings{kim2026transpace,
  title     = {TranSpace: Progressive Anchoring for Metric-Consistent Scene Synthesis},
  author    = {Kim, Hyeshim and Kim, Taehei and Shin, Jihun and Kim, Hyeonjin and Lee, Sung-Hee},
  booktitle = {Proceedings of the 34th ACM International Conference on Multimedia (MM '26)},
  year      = {2026},
  publisher = {ACM},
  doi       = {10.1145/3767308.3836316}
}

Data provenance

The released pack derives from the Deep Blending scenes drjohnson and playroom (Hedman et al., ACM ToG 37(6), 2018).

The paper additionally evaluates on 13 ScanNet++ scenes (© Technical University of Munich, used under the ScanNet++ Terms of Use). ScanNet++ imagery and any voxel models trained from it are not redistributed here.

Acknowledgements

We are grateful to the authors of the open-source projects this work builds on.

• SVRaster (NVIDIA) — adaptive sparse voxel novel view synthesis; vendored and modified here, see NOTICE
• 3D Gaussian Splatting (Inria / MPII) — incorporated by SVRaster
• Depth-Anything-V2 — monocular metric depth
• Text2Room — parts of the source code used in this implementation
• DeeVid AI (V2.1) — intermediate frame densification for the released data pack
• Google Imagen 2 — keyframe inpainting for the released data pack