ICLR 2026 RoboMD

RoboMD — Uncovering Robot Vulnerabilities through Semantic Potential Fields

Venue: ICLR 2026 Authors: Som Sagar · Jiafei Duan · Sreevishakh Vasudevan · Yifan Zhou · Heni Ben Amor · Dieter Fox · Ransalu Senanayake (Arizona State University · University of Washington · NVIDIA) arXiv: 2412.02818 Category: Robustness / security for VLA Trend tag: Vulnerability discovery · deep RL over VL embeddings · semantic potential fields

Approach diagram

flowchart LR
  subgraph EMB[Continuous vision-language embedding]
    S[Success regions]
    Fv[Vulnerable / failure regions]
  end
  DATA[Limited success-failure data] --> EMB
  EMB -->|treat as potential field| PF[Semantic potential field]
  PF --> RL[Deep RL vulnerability-prediction policy]
  RL -->|attracted to| Fv
  RL -->|repelled from| S
  RL --> VR[Virtual runs in simulation]
  VR --> OUT[Uncovered unique vulnerabilities<br/>up to +23% vs VL baselines]

Loading

Problem

Robot manipulation policies are highly vulnerable to external variations in the real world, but diagnosing these vulnerabilities is hard for two reasons: (i) the relevant variations to test against are often unknown a priori, and (ii) direct real-world testing is costly and unsafe. Heuristic testing tends to miss subtle failure modes.

Method

RoboMD learns a separate deep reinforcement learning policy for vulnerability prediction, run virtually rather than on hardware:

1. Build a continuous vision-language embedding trained from limited success-failure data — a space rich in semantic and visual variations.
2. Treat that embedding as a potential field: the RL policy is attracted toward vulnerable (failure) regions and repelled from success regions.
3. Explore the field via virtual runs, surfacing variations that break the target manipulation policy without risking the physical robot.

Results

Across simulation benchmarks and a physical robot arm, RoboMD uncovers up to 23% more unique vulnerabilities than state-of-the-art vision-language baselines, revealing subtle failure modes overlooked by heuristic testing. The discovered vulnerabilities can guide targeted, data-efficient fine-tuning to improve manipulation robustness.

Significance

RoboMD turns vulnerability discovery into a search problem over a semantic embedding, casting safe, simulation-based red-teaming of robot policies as RL on a potential field. This complements perturbation-robustness training: RoboMD finds which semantic variations matter, while methods like RobustVLA harden policies against given perturbation sets.

Links

• arXiv: https://arxiv.org/abs/2412.02818
• OpenReview: https://openreview.net/forum?id=Gsrw1vxq1G
• Code: https://github.com/somsagar07/RoboMD

Related pages

• RobustVLA — hardening against perturbations once found
• Survey: VLA & Manipulation

← Back to ICLR-2026