RSS 2026 LIBERO X

LIBERO-X — Robustness Litmus for Vision-Language-Action Models

Venue: RSS 2026 (Datasets & Benchmarks session) · Authors: Guodong Wang*, Chenkai Zhang*, Qingjie Liu, Jinjin Zhang, Jiancheng Cai, Junjie Liu, Xinmin Liu — Meituan × Beihang University · arXiv: 2602.06556 · code Category: VLA benchmark Trend tag: RSS 2026 thread 6 — the evaluation crisis Scale: 5-level difficulty hierarchy · training set of 2,520 teleoperated demonstrations / 600 tasks / 100 scenes

Compiled from the verified RSS 2026 abstract and the paper's Fig. 1.

Key figure

Figure 1 of the paper. (a) Original LIBERO: one scene → one task → homogeneous trajectories, with test data closely mirroring training — the memorization trap. (b) Current improvements (LIBERO-Plus/Pro): reuse the original trajectories and perturb single dimensions independently, with no systematic difficulty progression. (c) LIBERO-X: training becomes one scene → multiple tasks → diverse trajectories (the multi-colored trajectory bundle), and evaluation becomes a five-level pyramid — L1 local spatial perturbation → L2 extended spatial perturbation → L3 scene-topology reconstruction → L4 visual-attribute variation → L5 semantic-equivalent instruction reformulation — with the inset bar chart showing representative VLA success collapsing 39.4 → 29.6 → 17.0 → 11.5 → 8.2 as levels stack. The pyramid is the paper's argument: degradation is graded and diagnosable, not binary.

Problem

LIBERO-family benchmarks reward in-distribution pattern matching; existing protocols "provide limited or misleading assessments" because they don't capture real-world distribution shift — the same diagnosis as LIBERO-Plus and RobotManip §6.1.

Method

1. Hierarchical evaluation protocol with progressive difficulty levels targeting three capabilities: spatial generalization, object recognition, and task-instruction understanding — enabling fine-grained analysis of where performance degrades as complexity compounds.
2. High-diversity teleoperated training dataset in which each scene supports multiple fine-grained manipulation objectives, narrowing the train–eval distribution gap by construction rather than by perturbation alone.

Results (as reported)

• Representative VLAs show significant performance drops under cumulative perturbations, exposing persistent weaknesses in scene comprehension and instruction grounding.

Significance

The third generation of the LIBERO robustness lineage: LIBERO → LIBERO-Plus (seven perturbation axes) → LIBERO-X (hierarchical capability-targeted protocol + train-side diversity). Its instruction-understanding axis converges with RoboTwin-IF's language-grounding diagnosis — the field is standardizing on capability-decomposed OOD evaluation. Useful companion to PolaRiS: LIBERO-X stresses a fixed sim benchmark correctly; PolaRiS makes new benchmarks from reality.

← RSS 2026 survey · Home