Skip to content

v0.1.0 — sparse-wrist → whole-body models (1-wrist + 2-wrist)

Latest

Choose a tag to compare

@zxpeng2007 zxpeng2007 released this 01 Jul 09:08
· 7 commits to main since this release

Hand2Body — pretrained models

Hand2Body lifts sparse wrist tracking → whole-body SMPL motion (causal / streaming). Weights are
published here (kept out of git history). Two flavours of the same architecture (DiTDenoiser), selected
by hand_dim = 12 × #wrists:

diffusion_full.pt — 1 wrist (12D), table tennis

Left-wrist [pos(3) | lin_vel(3) | rot6D(6)] → whole-body SMPL (135-D). Held-out on 775 unseen
table-tennis sequences: 20.7 mm MPJPE, 8.5 mm wrist pos, 0.72° wrist orient.

m = DiTDenoiser(hidden=256, n_layers=4, hand_dim=12)

arctic_bimanual_30k.pt — 2 wrists (24D), bimanual manipulation

BOTH wrists [left12 | right12] → whole-body SMPL. Trained on ARCTIC (bimanual object manipulation),
30k steps. Held-out (30 val seqs): ≈63 mm MPJPE, 14 mm / 2.1° wrist, jitter 0.75. Wrist tracking is
tight; the ~63 mm MPJPE floor is the unconstrained lower body (legs/root are prior-driven — no wrist
observes them).

m = DiTDenoiser(hidden=256, n_layers=4, hand_dim=24)

Use

gh release download v0.1.0 -p diffusion_full.pt -D checkpoints         # 12D table-tennis model
gh release download v0.1.0 -p arctic_bimanual_30k.pt -D checkpoints    # 24D bimanual model
import torch
from h2b.models.diffusion import DiTDenoiser
m = DiTDenoiser(hidden=256, n_layers=4, hand_dim=24).eval()             # 12 for the 1-wrist model
m.load_state_dict(torch.load("checkpoints/arctic_bimanual_30k.pt", map_location="cpu"))

Both trained on internal / licensed data (table-tennis train.pkl; ARCTIC under its research license).
Arch: hidden=256, n_layers=4, n_heads=8, cosine schedule, DDIM. See the repo README for the full pipeline.