Repository navigation
What's New
π0.5 Engine Refactor
The π0.5 inference engine (src/models/pi05/) has been fully refactored from a single 1640-line file into focused compilation units, matching the code quality standard of the rest of the codebase.
New file layout:
| File | Responsibility |
|---|---|
pi05.cpp |
Top-level model: Pi05Model, pi05_create, predict |
pi05_gguf.h/cpp |
GGUF I/O: GgufReader — metadata queries, type conversion, random row access |
pi05_graph.h/cpp |
GGML graph builders: build_gemma_layer, build_adarms_gemma_layer, adarms_norm, gated_residual, build_embed_suffix |
pi05_vispruner.h/cpp |
Two-stage visual token pruning (importance + diversity) |
pi05_weights.h |
Weight structs: GemmaLayerW, AdaGemmaLayerW |
pi05_utils.h |
sinusoidal_time_emb |
Key improvements:
- Extracted shared
build_attention()helper eliminates ~60 lines of duplication between the prefix tower and action expert - All files pass
.clang-formatand.clang-tidy(clang-18) with zero warnings - Fully regression-tested: 100% success rate on libero_object/task_0 (5 episodes, seed 42)
Evaluation Metrics
VlaCppClient and AsyncRtcBroker now track and report two new per-episode metrics:
- Inference count — number of
_predict_chunkcalls per episode - Server-side latency —
latency_ms_inferencefromPredictResponse, reported as per-episode average
These appear in summary.txt and the per-episode console output.
Regression results (NVIDIA A10, libero_object/task_0, n=5, n_action_steps=10, rtc_horizon=10):
| Metric | Value |
|---|---|
| Success rate | 100% (5/5) |
| Avg wall-clock latency/step | 18.3 ms |
| Avg server inference latency | 143.5 ms/chunk |
| Avg inferences/episode | 17.2 |
| Avg steps/episode | 139 |
Supported Models
| Model | Hardware | Latency |
|---|---|---|
| π0.5 | AGX Orin | 104.3 ms/step |
| π0.5 | A10 | 18.3 ms/step (Async+RTC) |
| FasterWAM | A10 | 7.2 ms/step (steady-state) |
| HY-VLA | A10 | — |