Runyi Yu^1,2,*, Xiaoyi Lin^1,3,*, Ji Ma¹, Yinhuai Wang^2,✉, Koukou Luo², Jiahao Ji¹, Huayi Wang^1,4, Wenjia Wang^1,4, Runhan Zhang¹, Ping Tan², Ting Wu¹, Ruoli Dai¹, Qifeng Chen^2,✉, Lei Han^1,✉

¹Noitom Robotics   ²The Hong Kong University of Science and Technology   ³Wuhan University   ⁴The University of Hong Kong

^*Equal contributors    ^✉Corresponding authors

OmniContact is a contact-flow framework for long-horizon humanoid loco-manipulation. The system has two main pieces:

• CFgen generates task-space contact-flow references for skills such as carry, push, slide, relocate, kick, and chained meta-skills.
• CFtrack is the low-level policy that tracks either CFgen references or full .npz human-object interaction motions.

This repository provides two MuJoCo execution paths:

• deploy_omnicontact/run_skill_omnicontact.py: direct scripted execution for CFgen or NPZmotion tracking.
• deploy_omnicontact/deploy_omnicontact.py: interactive hot-switch execution with an Xbox joystick, designed to mirror the state switching pattern used by sim2real deployment.

Create and activate the Python environment:

conda create -n omnicontact python=3.11 -y
conda activate omnicontact
conda install pytorch==2.3.1 torchvision==0.18.1 torchaudio==2.3.1 pytorch-cuda=12.1 -c pytorch -c nvidia -y
pip install numpy onnx onnxruntime mujoco pyyaml scipy pygame

Run commands from the repository root:

cd /path/to/OmniContact_sim2sim

The direct runner resolves relative model paths under policy/omnicontact/model/, so --policy policy.onnx maps to policy/omnicontact/model/policy.onnx.

Use run_skill_omnicontact.py for scripted runs that start OmniContact immediately.

python deploy_omnicontact/run_skill_omnicontact.py \
  --reference-source CFgen \
  --policy policy.onnx \
  --task carrybox \
  --init-pos 1.0 0.0 \
  --goal-pos 2.5 0.5

Supported single-skill tasks:

--task pushbox is treated as an alias for pushbox-in.

python deploy_omnicontact/run_skill_omnicontact.py \
  --reference-source CFgen \
  --policy policy.onnx \
  --task-chaining carry-push \
  --init-pos 1.0 0.0 \
  --goal-pos 2.5 0.5

Common chain presets include push-carry, carry-push, push-relocate, carry-carry, carry-carry-carry, and carryheart.

Extra object initialization can be provided for chained tasks:

python deploy_omnicontact/run_skill_omnicontact.py \
  --reference-source CFgen \
  --policy policy.onnx \
  --task-chaining push-carry \
  --init-pos 1.0 0.0 \
  --init-pos-extra 2.2 -0.8 \
  --goal-pos 2.5 0.5

Use --reference-source NPZmotion to track a full .npz motion from data/.

python deploy_omnicontact/run_skill_omnicontact.py \
  --reference-source NPZmotion \
  --policy policy.onnx \
  --npz-dir data/relocateball/relocateball_motion_3_with_contact.npz \
  --start-frame 0

The runner infers XML assets from common data folders. Pass --xml-path to override automatic XML selection.

python deploy_omnicontact/run_skill_omnicontact.py \
  --reference-source NPZmotion \
  --xml-path g1_description/omnicontact_relocate_ball.xml \
  --policy policy.onnx \
  --npz-dir data/relocateball/relocateball_motion_3_with_contact.npz

deploy_omnicontact/deploy_omnicontact.py keeps the full FSM alive and allows hot-switching between policies with an Xbox joystick:

Passive / default state -> DefaultPose -> LocoMode -> OmniContact

This mirrors the sim2real pattern: bring the robot to a stable default pose, switch into locomotion, then switch into the OmniContact policy at the desired moment without restarting the controller.

python deploy_omnicontact/deploy_omnicontact.py \
  --task pushbox \
  --init-pos 1.0 1.0 \
  --goal-pos 3.0 0.0

deploy_omnicontact.py automatically selects the MuJoCo XML from --task. Pass --xml-path only when you want to override this automatic mapping.

The joystick mapping is defined in common/joystick.py.

When switching to OmniContact with L1 + A, the script resets the active object and table references from --init-pos, --goal-pos, and --box-half-dims, then enters the OmniContact FSM state.

The interactive deploy visualizes:

• wrist, torso, and ankle reference mocaps;
• contact-state color changes;
• table/object references;
• ghost robot visualization when the loaded XML contains g1_ghost.xml.

If the ghost robot is not visible in the MuJoCo viewer, enable group 1 rendering, since the ghost geoms are assigned to visual group 1.

@misc{yu2026omnicontactchainingmetaskillscontact,
      title={OmniContact: Chaining Meta-Skills via Contact Flow for Generalizable Humanoid Loco-Manipulation}, 
      author={Runyi Yu and Xiaoyi Lin and Ji Ma and Yinhuai Wang and Koukou Luo and Jiahao Ji and Huayi Wang and Wenjia Wang and Runhan Zhang and Ping Tan and Ting Wu and Ruoli Dai and Qifeng Chen and Lei Han},
      year={2026},
      eprint={2606.26201},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2606.26201}, 
}