Skip to content

RoboRT v2.0 — FasterWAM GPU Inference

Choose a tag to compare

View all tags
@lw-liuwang lw-liuwang released this 29 Aug 10:25
· 2 commits to main since this release
RoboRTv2.0
c033cea

What's New

FasterWAM GPU Inference

Full GPU ggml graph inference for the FasterWAM flow-matching action diffusion model, with CUDA graph acceleration.

Performance on NVIDIA A10:

Scenario Latency vs. CPU baseline
Steady-state (text fixed, state updates) 7.2 ms/step ~70,000×
Text change (new task/instruction) 29.5 ms/step ~17,000×

Key optimizations:

  • Video KV cache — prefill skipped when text context is unchanged; only 7 ms denoise runs each step
  • Unrolled CUDA graph — all 20 denoising steps captured in a single CUDA graph, launched with one dispatch
  • G→G KV copy — video prefill results copied GPU→GPU directly into the denoise buffer, no host staging
  • Two-phase async interface (prepare / compute) — server can pipeline prepare(N+1) with compute(N)

robo:: Public API

New policy.h header provides a clean robo:: namespace over the existing vla:: implementation. All new code and architecture implementations should use robo::.

robo::Policy * m = robo::policy_load("", "policy.gguf");
std::vector<float> actions = robo::step(m, in);
robo::policy_free(m);

Build Target Rename

Old New
vla-pi05-server robort-server
vla-pi05-selfcheck robort-selfcheck

Old names remain valid as aliases.

Supported Models

Model Hardware Latency
π0.5 AGX Orin 104.3 ms/step
π0.5 A10 18.7 ms/step (Async+RTC)
FasterWAM A10 7.2 ms/step

Previous Release (v1.0)

  • π0.5 GGUF inference on AGX Orin: LIBERO 39/40 (97.5%), 104.3 ms/step
  • HY-VLA (Hy-Embodied-0.5-VLA) support
  • Async dual-thread pipeline + RTC prefix guidance
Assets 2
Loading