World Model Experiments

This project contains small games for testing how an agent can learn a world model only by interacting with an environment.

The intended setting is that the model has no prior explanation of the game rules. It receives observations, chooses actions, sees rewards, and must infer the transition dynamics from experience.

Project Structure

Each game lives in games/ and is split into two files:

• <game>.py: the executable Gym-style environment
• <game>.md: a human-readable game specification with examples

Each agent lives in agents/:

• <agent>.py: an agent implementation that exposes an Agent class

Current games:

• Word World: a one-dimensional string game where the player moves left or right, collects targets, and grows.

Current agents:

• random_agent: samples uniformly from the game's action space.

Environment API

Games should follow the Gymnasium-style API:

state, info = env.reset(seed=123)
state, reward, terminated, truncated, info = env.step(action)

The public loop should expose only the current state and reward to avoid giving away rule names through metadata.

Running Word World

Run the current game with:

python -m games.word_world

The command prints lines like:

state=-----+--------0------+---+---- reward=0.0
state=-----+-------0-------+---+---- reward=-1.0

Evaluating An Agent

Run an agent against a game with:

python evaluate.py random_agent word_world

The script prints an aggregate score:

agent=random_agent
game=word_world
episodes=20
score=-66.700
completed=7
completion_rate=0.350
best_reward=2.000
worst_reward=-118.000

Using Word World In Code

from games.word_world import LEFT, RIGHT, WordWorldEnv

env = WordWorldEnv()
state, info = env.reset()
state, reward, terminated, truncated, info = env.step(LEFT)

Seeded random initialization is available:

env = WordWorldEnv.random(width=30, food_count=3, seed=100)
state, info = env.reset(seed=42)

Adding A Game

To add a new game:

1. Add games/<game_name>.py.
2. Add games/<game_name>.md.
3. Keep the environment API compatible with reset and step.
4. Make the runnable game output only state and reward.
5. Add focused tests for reset, transitions, rewards, and episode endings.

Adding An Agent

To add a new agent:

1. Add agents/<agent_name>.py.
2. Define an Agent class.
3. Accept action_space and an optional seed in __init__.
4. Implement act(state, reward=0.0).
5. Optionally implement reset() for per-episode state.

Experiment Goal

The experiments compare how efficiently different agents learn an accurate world model. Useful measurements include:

• how many interaction steps are needed before predictions become accurate
• how well another algorithm can use the learned model to plan for reward
• how robust the learned model is across seeded game initializations

The first game is intentionally text-based so an LLM can perceive the full state directly. Later experiments can add more games while keeping the same interaction pattern.