Skip to content

IROS 2026 Proprioceptive Transformer

Jump to bottom
hwoo.han edited this page Sep 7, 2026 · 2 revisions

IROS 2026 β€” Proprioceptive Transformer: Robust In-Hand Manipulation from Joint Sensing Only

Venue: IROS 2026 (Pittsburgh) Β· paper #3215 Β· ETH Zurich (Yao, Yang, Kim, Sympetheros, Katzschmann) Β· Best Paper / Best Student Paper Award session. Paper: arXiv 2605.21330. Representative of: in-hand dexterity (reorientation) β€” continuous cube rotation on a tendon-driven hand using joint sensing alone, no vision or tactile. Companions: Dexterous Manipulation Β· RL for VLA Β· IROS 2026 survey.

Proprioceptive Transformer β€” a teacher trained by RL with privileged object info is distilled into PT, which consumes only joint position/velocity histories (16 magnetic angle sensors on the ORCA hand) through a Transformer that extracts implicit object state from temporal joint patterns; no vision/tactile at inference (figure from Yao et al., arXiv 2605.21330, Β© the authors)

1. Problem

In-hand manipulation usually leans on vision or tactile to track object state, while joint sensing β€” the most readily available modality on any hand β€” is largely overlooked, especially for tendon-driven hands. How far can joint sensing alone go?

2. Method

The Proprioceptive Transformer (PT) is an exteroception-free policy for continuous cube rotation using only joint feedback:

  • A teacher policy is trained by RL with privileged object info, then distilled into PT, which operates solely on joint position + velocity histories.
  • A Transformer extracts implicit object-state information from temporal patterns in joint-sensor readings (proprioception β†’ implicit exteroception).

3. Results

  • On the real ORCA hand: 3.1Γ— higher rotation speed than baselines, and 23.4% lower RMSE for cube-pose estimation β€” i.e. joint-only sensing recovers object state better than baselines and controls faster, with no external perception.

4. Why it matters

PT is IROS 2026's in-hand-reorientation representative and a pointed result: proprioception alone can carry dexterous in-hand control, recovering implicit exteroceptive state via a Transformer over joint-history. It's the minimalist counterpoint to the tactile-heavy dexterity line (T-Rex, MrGrasp) β€” showing that the cheapest, always-present modality is underexploited. The teacher-distillation recipe echoes DreamMimic's privileged-teacher β†’ deployable-student pattern. (An award finalist β€” a rare "technically representative AND recognized" pick.)

Limitations (reviewer): demonstrated on cube rotation (the canonical in-hand benchmark) on one hand (ORCA); joint-only sensing may not extend to arbitrary object geometries / fine contact tasks where tactile is decisive; RL-teacher needs sim privileged info.

5. Links

← Back to IROS 2026 survey Β· Home

Navigation

πŸ“– Reviews

🏷 Model lineages

🧠 ML foundations

πŸ—“ Conferences

(each page indexes its per-paper pages)

πŸ“Œ Foundational

Clone this wiki locally