Sensen Gao*1, Zhaoqing Wang*2, Qihang Cao1, Dongdong Yu2, Changhu Wang2, Jia-Wang Bian📧1
1 Nanyang Technological University 2 AISphere | * Co-first authors 📧 Corresponding author
TL;DR: PixWorld is a single end-to-end pixel-space diffusion model that unifies 3D scene generation and reconstruction — it supervises a pixel-aligned 3D Gaussian field directly through differentiable rendering, with no VAE or RAE, and adds a geometry perception loss for 3D structural consistency.
- One unified model for generation and reconstruction. A single two-stream diffusion transformer processes posed multi-view inputs as a clean subset (→ reconstruction) and a noisy subset (→ generation, optionally text-conditioned), decoding a pixel-aligned 3D Gaussian scene in one forward pass — no task-specific branches.
- Pixel-space supervision, no VAE/RAE. A flow-matching loss is imposed directly on rendered multi-view images via differentiable rendering, so optimization is aligned with 3D scene fidelity instead of an intermediate latent target — removing the frozen VAE/RAE and its reconstruction ceiling.
- Geometry perception loss. Rendered views are aligned with ground truth in the geometry-aware feature space of a frozen 3D foundation model (π³ / VGGT), injecting 3D structural supervision beyond 2D photometric and perceptual losses.
- Real-time inference. After distillation, the 4-step model (
PixWorld-480P-4steps) generates a scene in ~0.6 s — up to ~1000× faster than diffusion-based world generators.
One unified model, three capabilities — each grid shows six explorable 3D Gaussian scenes.
▶️ Full-quality, playable videos on the project page.
grid_recon.mp4
grid_img2_3d.mp4
grid_txt2_3d.mp4
A single PixWorld model performs both 3D reconstruction and generation. After distillation, the 4-step model (PixWorld-480P-4steps) generates a scene in ~0.6 s — up to ~1000× faster than diffusion-based world generators (FantasyWorld 1041×, Gen3C 445×, Gen3R 148×, FlashWorld 5×).
We release NVS-Refined 🤗 — a curated, high-quality dataset for novel view synthesis, distilled from RealEstate10K, ACID, DL3DV, and SpatialVid. From these sources we keep only the clips that are:
- 🔍 Sharp & high-fidelity — visually clean frames, with blurry and heavily-compressed sequences filtered out.
- 🎥 Large in camera motion — sequences with substantial pose variation, so the data genuinely stresses view synthesis and 3D geometry.
- 🎨 High in aesthetic quality — scored and filtered for visual appeal.
For clips that are otherwise valuable but noticeably blurry, instead of discarding them we restore them with Streaming FlashVSR (streaming video super-resolution), recovering usable high-resolution detail.
👉 huggingface.co/datasets/Sensen02/NVS-Refined
- 📦 NVS-Refined dataset — released on Hugging Face 🤗
- ⚡
PixWorld-480P-4stepsdistilled model — the 4-step distilled weights + inference code, coming in a short time. Stay tuned!
If you find this repository useful, please consider citing PixWorld:
@misc{gao2026pixworld,
title={PixWorld: Unifying 3D Scene Generation and Reconstruction in Pixel Space},
author={Sensen Gao and Zhaoqing Wang and Qihang Cao and Dongdong Yu and Changhu Wang and Jia-Wang Bian},
year={2026},
eprint={2607.05373},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2607.05373},
}