Repository navigation
AutoE2E v1.0 is the first tagged release of the camera-first End-to-End driving model from the Autoware Robotaxi working group.
Highlights
- Reactive model. Runs at 10 Hz on six surround cameras, lifts them into a bird's-eye-view grid, optionally fuses a map raster, and plans the next 6.4 s of acceleration and curvature with a GRU planner. 79.9 M parameters. The World Action and Reasoning models are still in development.
- Checkpoints on Hugging Face.
nuplan-epoch-5is trained on nuPlan v1.1 for zero-shot driving and fine-tuning.kitscenes-epoch-5is fine-tuned on KITScenes-Multimodal. - Benchmarks. On KITScenes-Multimodal validation, the fine-tuned checkpoint reaches ADE 0.744 m and FDE 1.980 m at 3 s, second among the eight open-weights models compared. Issue #210 has the adapters and the full results.
- Speed. Inference reaches up to ~76 FPS (SwinV2-Tiny, feature-concat fusion, RTX 5080, batch 1). See BENCHMARKS.md.
- Tooling. The DataModelConsole dashboard replays camera scenes with ground-truth and predicted trajectories.
Platform/holds the EKS-based MLOps platform for training and evaluation, fully managed as IaC.
Get started
git clone --branch v1.0 https://github.com/autowarefoundation/auto_e2e.git
cd auto_e2e
make setupPython 3.12 is required. The documentation and academic paper cover the model in detail.
What's Changed
- bench: add RTX5090 results by @CodeName-Detective in #181
- bench: add RTX4070 Laptop GPU results by @CodeName-Detective in #180
- bench: add RTX5080 results by @CodeName-Detective in #179
- bench: add A40 results by @CodeName-Detective in #178
- fix(ci): correct setup-python input name in docs workflow by @apgeorg in #185
- Align KITScenes planning loss with rollout evaluation and composite checkpoint selection by @riita10069 in #176
- bench: add RTX4080 results by @CodeName-Detective in #183
- feat(map_encoder): add deformable cross-attention map BEV fusion mode by @intisar1020 in #184
- feat(training): expose map_fusion_mode as a train_il parameter by @ChristophBank in #192
- bench: add RTX 4060 8 GB inference speed results by @FLagbusted in #190
- fix(training): probe every planner parameter for optimizer progress by @ChristophBank in #201
- fix(model): vectorize egomotion normalization and stop mutating the caller by @ChristophBank in #202
- Add semantic occupancy dashboard by @riita10069 in #203
- docs: add AutoE2E academic paper artifacts by @riita10069 in #214
- docs(readme): link the Hugging Face v1.0 release by @riita10069 in #212
- Release/v1.0 by @riita10069 in #216
- feat(nuplan): pack held-out test scenes for Console playback by @riita10069 in #213
- feat(console): sync DataModelConsole with the production deployment by @riita10069 in #215
- docs(readme): add a Benchmarks section with the KITScenes 3 s ADE/FDE by @Arseni10Lk in #218
- docs(readme): animate the benchmark chart and match it to the table by @riita10069 in #219
- refactor(platform): single wf_autoe2e pipeline with MLflow LoggedModel lineage by @riita10069 in #225
- fix(platform): align main's IaC with the running production cluster by @riita10069 in #217
- feat(benchmark): add Drive-JEPA evaluation by @HassanAliAnsari in #224
- feat(benchmark): add METEOR evaluation scripts by @Arseni10Lk in #226
New Contributors
- @HassanAliAnsari made their first contribution in #224
Full Changelog: https://github.com/autowarefoundation/auto_e2e/commits/v1.0