DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained in a diffusion world model. NeurIPS 2024 Spotlight.

Mastering Diverse Domains through World Models

Mastering Atari with Discrete World Models

Transformers are Sample-Efficient World Models. ICLR 2023, notable top 5%.

[CVPR 2024 Highlight] GenAD: Generalized Predictive Model for Autonomous Driving & Foundation Models in Autonomous System

Dream to Control: Learning Behaviors by Latent Imagination

A structured implementation of MuZero

World Model based Autonomous Driving Platform in CARLA 🚗

Deep Hierarchical Planning from Pixels

Efficient World Models with Context-Aware Tokenization. ICML 2024

World Models applied to the Open AI Sonic Retro Contest

Code for the ICLR 2024 spotlight paper: "Learning to Act without Actions" (introducing Latent Action Policies)

Code for "Planning Goals for Exploration", ICLR2023 Spotlight. An unsupervised RL agent for hard exploration tasks.

Transformer-based World Models

[NeurIPS 2022] SGAM: Building a Virtual 3D World through Simultaneous Generation and Mapping

[NeurIPS 2024] GenRL: Multimodal foundation world models allow grounding language and video prompts into embodied domains, by turning them into sequences of latent world model states. Latent state sequences can be decoded using the decoder of the model, allowing visualization of the expected behavior, before training the agent to execute it.

TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction. ICRA 2023. You may also want to check out the updated version: https://github.com/zhejz/TrafficBotsV1.5

Recall to Imagine, a model-based RL algorithm with superhuman memory. Oral (1.2%) @ ICLR 2024

Jax/Flax Implementation of TD-MPC2

[ICLR 2023] Choreographer: a model-based agent that discovers and learns unsupervised skills in latent imagination, and it's able to efficiently coordinate and adapt the skills to solve downstream tasks.