Agent-Aware Latent World Modeling for Long-Horizon Temporally Consistent Autonomous Driving
MomWorld is a research extension of MomAD for 6-second open-loop planning on nuScenes. It keeps the pretrained MomAD planner as a stable base and adds a mode-conditioned latent world model that rolls out multi-agent futures and predicts gated residual corrections to the ego plan.
- Mode-Conditioned Latent World Rollout (MC-LWR): encodes ego, map, detected-agent features and multi-modal agent forecasts, then performs a 12-step recurrent latent rollout for every command-conditioned planning mode.
- Temporal-Consistency-Aware Residual Fusion (TC-ARF): applies small corrections near the current time and stronger corrections at long horizons, while suppressing world-model updates that reduce temporal consistency.
Fresh evaluation on all 6,019 nuScenes validation samples with the included 6-second configuration:
| Metric | Result |
|---|---|
| Detection mAP | 0.4152 |
| NDS | 0.5257 |
| AMOTA | 0.3799 |
| Map mAP | 0.5616 |
| Planning L2 | 0.4945 m |
| Object collision | 1.002% |
| Box collision | 0.161% |
| Temporal consistency | 0.3995 |
The 4s / 5s / 6s planning values are:
| Horizon | L2 (m) | Box collision (%) | Consistency (m) |
|---|---|---|---|
| 4s | 1.2786 | 0.825 | 0.9271 |
| 5s | 1.7625 | 1.358 | 1.1855 |
| 6s | 2.3080 | 1.966 | 1.4593 |
- Final experiment config:
open_loop/projects/configs/MomAD_small_stage2_MomAD_World_model_6s_v2_oracle_mode_reg02_resume_repro.py - Latent world model:
open_loop/projects/mmdet3d_plugin/models/motion/latent_world_model_MomAD_World_model_6s.py - Planning head and residual fusion:
open_loop/projects/mmdet3d_plugin/models/motion/motion_planning_head_MomAD_World_model_6s_v2_oracle_reg_backup_20260628.py - World-state dataset pipeline:
open_loop/projects/mmdet3d_plugin/datasets/pipelines/world_model_pipeline_MomAD_World_model_6s.py - 6-second data converter:
open_loop/tools/data_converter/nuscenes_converter_MomAD_World_model_6s.py
nuScenes data, generated .pkl files, model checkpoints and experiment outputs are intentionally excluded from this repository. Put them under the relative locations referenced by the configuration, or adjust load_from and resume_from locally.
The open_loop/navsim_patch adapter is experimental. No official NAVSIM PDMS/EPDMS result is claimed yet.
This repository is derived from MomAD and retains its MIT license and attribution. See LICENSE and the upstream MomAD repository for details.