A full-size VLA now runs end to end through PlugRL - inference, feedback and a training step - and the server's record of what happened reconciles exactly with the environment clients'.
The reinforcement learning result is negative, and is reported as negative.
E11: the experiment this release exists for
experiments/e11-vla-rl-libero was pre-registered on 2026-09-13, before any success rate had been measured, and its four predictions were judged against the data.
Stage A, the control. pi05_libero scored 99 of 100 on libero_spatial and 185 of 200 on libero_10, against openpi's published 98.8 and 92.4. Both cells reconcile to the unit: identical episode and step counts on the server and on the clients, zero missing-step-state warnings, zero reconnects.
Stage C, the fine-tuning. One FPO iteration - 4,096 transitions collected across the process boundary, 2,048 optimizer steps - produced a checkpoint that scores 0 of 50 where its baseline scored 26 of 50, on the same 50 initial states. P3 predicted the opposite and is recorded as falsified.
The run is incomplete, at 1 iteration of 10. Both attempts ran out of GPU memory at the second learn step, which has to fit beside the master weights and Adam moments the first one allocated on a 24 GB card.
| prediction | verdict |
|---|---|
| P1, the control is within 10 points of openpi's published number | confirmed, 0.990 against a threshold of 0.888 |
P2, libero_10 scores below libero_spatial |
cannot be separated - the intervals overlap, which the protocol says scores nothing |
| P3, fine-tuning raises success above the baseline interval | falsified - it lies below, and the two do not touch |
| P4, correctness holds with a real VLA behind the server | holds in all four cells that can test it; unavailable in the two training runs, whose crashes took the clients down before they wrote their summaries |
Six defects, each found by running it for real
None of these was reachable with a smaller policy, and each ships with a regression test.
| #14 | openpi's image masks are numpy scalars; batching turned them into Python lists, and the server died on its first inference against a real pi0.5 |
| #16 | FPO stored an observation as one float array, which a tree of images, masks and token ids is not. An FPO server for pi0 could not start |
| #17 | a bfloat16 value head cannot become a numpy array |
| #18 | an Adam step of 1e-5 rounds away in bfloat16 with nothing to keep it, so 95% of the action expert's linear weights never moved - no crash, nothing in the loss curve |
| #20 | a learn step of 3,490 s behind a 60 s feedback deadline looked like ten dead clients |
| #21 | a learn step's blocks stay in PyTorch's caching allocator, so the next collection cannot find 124 MiB |
#18 is the one worth reading about. It does not fail; it quietly trains almost nothing. It was found by comparing a checkpoint against the base weights parameter by parameter, and fixed by stepping float32 copies of the half-precision weights: 4.7% of those weights moved before the fix, 41.6% after, over the same 32 steps.
Known limits
- No VLA has been trained to completion here. One iteration of ten is what fits.
- Learning and inference are serialised by the model lock: during a learn step the env clients idle, which was 86% of that iteration's wall clock.
- Four fifths of a learn step recomputes the prefix of a frozen VLM, once per minibatch.
- Single machine. The Ray path caps workers at the local GPU count.
- One task, one seed, untuned hyperparameters in Stage C. The protocol said in advance that a negative P3 is a statement about those values on that task.
Everything else
The env client, the protocol and the documentation site are unchanged since v0.1.0 and keep that tag.