Skip to content

DAGroup-PKU/PhysisForcing

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation 🔥

Peiwen Zhang*, Yufan Deng*, Shangkun Sun*, Juncheng Ma, Duomin Wang†, Jonas Du, Zilin Pan, Ye Huang, Hao Liang, Songyan Huang, Ruihua Zhang, Enze Xie†, Ming-Yu Liu, Daquan Zhou†‡

Peking University · NVIDIA

*Equal Contribution  †Co-Project Lead  ‡Corresponding Author

PhysisForcing is a training-time, plug-and-play framework that makes robotic video generation physically plausible. It focuses supervision on interaction-critical regions and aligns generation at two levels — a pixel-level trajectory loss and a semantic-level relational loss — on an intermediate DiT feature, dropping into existing video backbones (Wan, Cosmos) with zero extra inference cost. It ranks first on R-Bench, PAI-Bench, and EZS-Bench, and as a world model lifts the WorldArena-action planner closed-loop success rate from 16.0% → 24.0%.

PF_Cosmos = Cosmos3-Nano + PhysisForcing  |  PF_Wan = Wan + PhysisForcing

🔥 News

  • [2026.07.08] 🎉 We release the inference code & model weights for PF_Cosmos and PF_Wan! Training code will be open-sourced within a week.
  • [2026.06.29] 🔥 We release the arXiv paper and project page of PhysisForcing.

📑 Todo List

  • Inference code & checkpoints for PF_Wan & PF_Cosmos
  • Training code & auxiliary model checkpoints

🎥 Demo

The video below is a compressed preview. Full HD demos and the complete set of side-by-side qualitative comparisons (playable videos across embodiments and tasks) are best viewed on the Project Page.

demo.mp4

📈 Results

All scores are normalized percentages (higher is better, only the Avg. column shown). Bold = PhysisForcing variants (PF_Wan14B/PF_Wan5B on Wan, PF_Cosmos on Cosmos3-Nano). The first three are embodied video-generation benchmarks; the rightmost is the WorldArena Action Planner world-model evaluation (IDM closed-loop success rate).

R-BenchAvg.
Veo 3.156.3
Hailuo v256.5
Cosmos 3-super58.1
Seedance 1.5 Pro58.4
Wan 2.660.7
Abot-PhysWorld52.9
Wan2.2-A14B50.7
Wan2.2-A14B (ft)57.9
PF_Wan14B62.0
Cosmos 3-nano (ft)61.5
PF_Cosmos63.8
PAI-Bench (robot)Avg.
Wan 2.580.96
GigaWorld-080.87
Veo 3.180.45
WoW-Wan 14B79.53
Sora v2 Pro76.52
Abot-PhysWorld84.91
Wan2.2-A14B78.93
Wan2.2-A14B (ft)79.90
PF_Wan14B81.73
Cosmos 3-nano (ft)84.03
PF_Cosmos85.17
EZS-BenchAvg.
WoW-Wan 14B77.80
GigaWorld-075.49
Cosmos-Predict 2.573.94
UnifoLM-WMA-062.94
Kling 2.6-Pro79.39
Abot-PhysWorld80.30
Wan2.2-A14B77.16
Wan2.2-A14B (ft)79.04
PF_Wan14B80.54
Cosmos 3-nano (ft)80.29
PF_Cosmos81.08
Action PlannerAvg.
Genie Envisioner15.0
TesserAct18.0
RoboMaster14.0
Vidar10.5
WoW20.5
Wan2.2-5B (base)16.0
PF_Wan5B24.0

Full per-metric tables and ablations are in the paper.

⚙️ Usage

This repo hosts two self-contained inference bundles — each ships its own framework code, environment, weights, and example inputs. Pick the one you need and follow its README:

  • pf_cosmos/PF_Cosmos (Cosmos3-Nano + PhysisForcing), image-to-video.
  • pf_wan/PF_Wan (Wan2.2-A14B + PhysisForcing), image-to-video.
git clone https://github.com/DAGroup-PKU/PhysisForcing.git
cd PhysisForcing/pf_cosmos   # or: cd PhysisForcing/pf_wan
# then follow that bundle's README: environment -> weights -> run

The two bundles use different PyTorch/CUDA stacks, so set each up in its own environment.

🚧 Training code & auxiliary checkpoints: coming soon.

🙏 Acknowledgement

Our work builds on many excellent projects: Wan, Cosmos, CoTracker3, Depth-Anything-2, V-JEPA, and R-Bench / RoVid-X.

✏️ Citation

If you find PhysisForcing useful, please consider giving a ⭐ and citing:

@article{zhang2026physisforcing,
  title={PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation},
  author={Zhang, Peiwen and Deng, Yufan and Sun, Shangkun and Ma, Juncheng and
          Wang, Duomin and Du, Jonas and Pan, Zilin and Huang, Ye and Liang, Hao and
          Huang, Songyan and Zhang, Ruihua and Xie, Enze and Liu, Ming-Yu and Zhou, Daquan},
  journal={arXiv preprint arXiv:2606.28128},
  year={2026}
}

License

This project is released under the MIT License.

About

PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation

Resources

License

Stars

102 stars

Watchers

4 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages