FLUX 3 Action is an open weights 7B world action model from Black Forest Labs. It takes camera frames, the robot's state and a text instruction, and returns the next chunk of actions, denoised together with the next video frames.
This repository is the standalone training and inference code for robots, simulators and games.
Weights · Documentation · Blog post · LeRobot integration
transformer.py, transformer_inf_bf16.py and transformer_inf_fp8r.py keep training and inference implementations separate so readers, especially coding agents, can follow each path directly without tracing inheritance or configuration branches.
The FLUX 3 Action collection contains separate DROID and SO-101 policy repositories. Their frozen encoders are shared through the base repository.
DROID offers BF16 and FP8r packages for base, guidance-distilled and single-step inference.
| Recipe | BF16 subfolder | FP8r subfolder |
|---|---|---|
| Base: 4 steps with guidance | root (omit subfolder) | variants/fp8r |
| Guidance-distilled: 4 steps | variants/gd |
variants/gd-fp8r |
| Step-distilled: 1 step | variants/sd |
variants/sd-fp8r |
Standalone FLUX Action supports all six packages. BF16 packages also include LeRobot configs;
native FP8r packages require FLUX Action. LeRobot reads config.json; FLUX Action reads
config.native.json when present, otherwise config.json.
See checkpoint selection for commands.
| Guide | What you’ll do |
|---|---|
| Setup & inference | Install dependencies, download weights and predict actions with DROID or SO-101 |
| DROID full fine-tuning | Prepare DROID data, run the reproduction recipe, resume and export |
| SO-101 LoRA | Adapt the prepared SO-101 checkpoint to your task with LeRobot |
| Custom tasks and embodiments | Define your own observations and actions, using games as a worked example |