METEOR: an open reference E2E model for surround-view driving, trained with zero human labels

[dan-dnn]

Hi all — TIER IV has open-sourced METEOR, a reference end-to-end (E2E) driving model built on Co-MLOps data, and published the weights and demo scenes under the Autoware Foundation on Hugging Face. Sharing it here as a base others can run, study and improve.

What it is

• 8 surround cameras (Co-MLOps DRS) + calibration + ego speed in; one 54M-parameter network out: BEV lane map, metric depth, 3D/2D detection with parked/stopped flags, 2D segmentation, 3D occupancy and flow, agent forecasting, traffic-light state, a near-range risk field, and a multimodal E2E trajectory (3 hypotheses + steering/accel/brake) under a deterministic guardrail.
• No HD map. Camera-only at deployment; LiDAR is an optional extra input on the same weights.
• Zero human labels: every supervision signal comes from an autolabel factory (CoMET) over raw LiDAR / ego-pose / panoptic recordings, plus NVIDIA Cosmos Transfer re-renderings for adverse conditions.
• Zero human-written code: the model, the label factory, the trainer, the TensorRT runtimes and the docs were written by an AI agent running the train–evaluate–improve loop; humans set goals and reviewed results.
• Runs on a Jetson AGX Orin at ~67 ms per 8-camera frame (INT8, 2:4 structured sparsity) with Python and C++ TensorRT runtimes.

What is released

• Source, training recipe, autolabel factory and runtimes (Apache-2.0): https://github.com/tier4/METEOR
• Weights (PyTorch checkpoint + plain ONNX) in the Autoware model layout: https://huggingface.co/AutowareFoundation/meteor
• Six anonymised 8-camera demo scenes (expressway, mountain road, arterial, city, night) — CC-BY-4.0: https://huggingface.co/datasets/AutowareFoundation/meteor-demo-scenes
• Background and the development story: TIER IV tech blog — https://tier4.co.jp/en/updates/technology/20260908-reference-e2e-model
• How the Co-MLOps dataset is paired with NVIDIA Cosmos: https://tier4.co.jp/en/updates/technology/20260807-comlops-dataset-foundation-for-autonomous-driving-with-nvidia-cosmos

Try it in a few minutes (no GPU needed for the first step)

pip install -U huggingface_hub onnxruntime opencv-python numpy
hf download AutowareFoundation/meteor --local-dir models
hf download AutowareFoundation/meteor-demo-scenes --repo-type dataset --local-dir data
git clone https://github.com/tier4/METEOR && cd METEOR
python3 hf/onnx_smoke_test.py --onnx ../models/meteor_v157c3Z.onnx --root ../data/valday --frame 40

deploy/build_engine_fp16.py then builds a plugin-free TensorRT engine on any NVIDIA GPU, and deploy/orin_realtime.py renders the 12-task demo video; the Orin INT8 path (lift plugin, on-device calibration) is documented in deploy/README.md.

Relation to Autoware

METEOR is published as a reference model with its own runtimes — an Autoware (ROS 2) node does not exist yet, and building one around the released ONNX is on the roadmap. The model card follows the layout of the other AutowareFoundation models (plain ONNX, param.yaml, engines built locally). Accuracy figures are measured on an internal validation split and are therefore not published; evaluation on a public benchmark is planned.

Feedback, questions and contributions are very welcome — especially on wrapping it as an Autoware node and on trying it with other camera rigs.