RoboRT v2.1 — π0.5 Engine Refactor

What's New

π0.5 Engine Refactor

The π0.5 inference engine (src/models/pi05/) has been fully refactored from a single 1640-line file into focused compilation units, matching the code quality standard of the rest of the codebase.

New file layout:

File	Responsibility
pi05.cpp	Top-level model: Pi05Model, pi05_create, predict
pi05_gguf.h/cpp	GGUF I/O: GgufReader — metadata queries, type conversion, random row access
pi05_graph.h/cpp	GGML graph builders: build_gemma_layer, build_adarms_gemma_layer, adarms_norm, gated_residual, build_embed_suffix
pi05_vispruner.h/cpp	Two-stage visual token pruning (importance + diversity)
pi05_weights.h	Weight structs: GemmaLayerW, AdaGemmaLayerW
pi05_utils.h	sinusoidal_time_emb

Key improvements:

• Extracted shared build_attention() helper eliminates ~60 lines of duplication between the prefix tower and action expert
• All files pass .clang-format and .clang-tidy (clang-18) with zero warnings
• Fully regression-tested: 100% success rate on libero_object/task_0 (5 episodes, seed 42)

Evaluation Metrics

VlaCppClient and AsyncRtcBroker now track and report two new per-episode metrics:

• Inference count — number of _predict_chunk calls per episode
• Server-side latency — latency_ms_inference from PredictResponse, reported as per-episode average

These appear in summary.txt and the per-episode console output.

Regression results (NVIDIA A10, libero_object/task_0, n=5, n_action_steps=10, rtc_horizon=10):

Metric	Value
Success rate	100% (5/5)
Avg wall-clock latency/step	18.3 ms
Avg server inference latency	143.5 ms/chunk
Avg inferences/episode	17.2
Avg steps/episode	139

Supported Models

Model	Hardware	Latency
π0.5	AGX Orin	104.3 ms/step
π0.5	A10	18.3 ms/step (Async+RTC)
FasterWAM	A10	7.2 ms/step (steady-state)
HY-VLA	A10	—

RoboRT v2.0 — FasterWAM GPU Inference

What's New

FasterWAM GPU Inference

Full GPU ggml graph inference for the FasterWAM flow-matching action diffusion model, with CUDA graph acceleration.

Performance on NVIDIA A10:

Scenario	Latency	vs. CPU baseline
Steady-state (text fixed, state updates)	7.2 ms/step	~70,000×
Text change (new task/instruction)	29.5 ms/step	~17,000×

Key optimizations:

• Video KV cache — prefill skipped when text context is unchanged; only 7 ms denoise runs each step
• Unrolled CUDA graph — all 20 denoising steps captured in a single CUDA graph, launched with one dispatch
• G→G KV copy — video prefill results copied GPU→GPU directly into the denoise buffer, no host staging
• Two-phase async interface (prepare / compute) — server can pipeline prepare(N+1) with compute(N)

robo:: Public API

New policy.h header provides a clean robo:: namespace over the existing vla:: implementation. All new code and architecture implementations should use robo::.

robo::Policy * m = robo::policy_load("", "policy.gguf");
std::vector<float> actions = robo::step(m, in);
robo::policy_free(m);

Build Target Rename

Old	New
vla-pi05-server	robort-server
vla-pi05-selfcheck	robort-selfcheck

Old names remain valid as aliases.

Supported Models

Model	Hardware	Latency
π0.5	AGX Orin	104.3 ms/step
π0.5	A10	18.7 ms/step (Async+RTC)
FasterWAM	A10	7.2 ms/step

Previous Release (v1.0)

• π0.5 GGUF inference on AGX Orin: LIBERO 39/40 (97.5%), 104.3 ms/step
• HY-VLA (Hy-Embodied-0.5-VLA) support
• Async dual-thread pipeline + RTC prefix guidance

RoboRT v0.1.0 — Real-time VLA Inference Engine

RoboRT v1.0

RoboRT 是一个面向 pi0.5 等视觉-语言-动作（VLA）策略的实时 C++ 推理引擎。v1.0 带来推理流水线重构、Jetson AGX Orin 端侧部署支持与多架构扩展能力。

✨ 亮点

• 实时推理大幅提速：异步双线程流水线 + RTC（Real-Time Chunking）实时动作引导，NVIDIA A10 上单步推理 377 ms → 18.7 ms（对比 LeRobot 基线 / 同步模式）
• Jetson AGX Orin 端侧部署：LIBERO 40 任务成功率 39/40（97.5%），平均推理 104.3 ms/step
• 多架构可扩展：按 checkpoint 元数据自动检测并调度架构，新增架构仅需注册一行

🚀 新功能

• 异步双线程 serving 流水线：接收/预处理与计算重叠，隐藏预处理耗时
• RTC 实时动作引导：利用上一步未执行的动作 chunk 前缀引导去噪，显著降低端到端延迟
• AGX Orin 交叉编译：aarch64 + sm_87 toolchain（local/agx-orin/toolchain-aarch64.cmake）、一键脚本 build-cross.sh、部署文档 DEPLOY.md
• HY-VLA 架构支持：单 GGUF 模型（视觉 + 语言 + 动作专家一体）
• 多架构构建选择：ROBORT_MODELS 编译选项 + 架构注册表 src/models/architectures.cmake

🔧 改进

• vla::Model 抽象为虚接口，按 checkpoint 的 GGUF KV 自动检测架构并分发（pi05 / hy_vla）
• 修正 .gitignore 的 models/ 规则，避免误伤 src/models/ 源码；local/agx-orin/ 交叉编译文件白名单

📖 文档

• 技术报告：《RoboRT: A Real-Time C++ Inference Engine for pi0.5 Vision-Language-Action Policies》（PDF）
• AGX Orin 交叉编译、部署与常见问题清单（local/agx-orin/DEPLOY.md）
• README 补充 AGX Orin 交叉编译说明与评估结果

📊 性能

平台	配置	成功率	平均推理
NVIDIA A10 (sm_86)	pi0.5, LIBERO object	10/10	18.7 ms/step（Async + RTC）
NVIDIA Jetson AGX Orin (sm_87)	pi0.5, LIBERO 4 suite / 40 任务	39/40（97.5%）	104.3 ms/step

交叉编译约束：必须使用 CUDA 12.6 的 nvcc 离线编译 sm_87 cubin（CUDA 13 产物要求驱动 ≥ 570，Orin 驱动 540.4.0 无法加载）。

🚀异步推理+RTC

新增异步推理和RTC，获得10倍加速🚀

RoboRT v0.1.0 — Real-time pi0.5 VLA Inference Engine

简介

RoboRT 是实时视觉-语言-动作（VLA）推理引擎，当前支持 pi0.5 策略。推理引擎为原生实现：手写 GGUF reader 直接加载权重，缓存计算图运行 flow-matching
去噪推理，仅依赖底层通用计算库（ggml 计算图 + CUDA 算子）与视觉编码库（SigLIP 视觉塔）。

核心特性

• 原生 pi0.5 引擎：手写 GGUF reader（F32/BF16/量化权重直读）+ 缓存计算图（仅当输入形状变化时重建）+ flow-matching 去噪
• C++ 推理 API：vla::Config / Inputs / predict / last_stats
• 无状态推理服务：ZeroMQ/Protobuf，强校验输入契约（state 长度、lang_tokens 范围、noise 形状、拒绝 NaN/Inf）
• 确定性自检工具：vla-pi05-selfcheck 输出 bit-exact dump，便于跨实现回归对比
• 模型转换脚本：HF safetensors → GGUF（主模型 + 视觉塔 mmproj）
• LIBERO 评估集成：seed-42 基线下 38/40，失败任务 {24, 29}

性能

LeRobot 基线	RoboRT
377 ms/step	161 ms/step

同一任务、同一观测、A10 GPU，约 2.3× 加速。