{{ message }}

# sahandrez / gym_forestfire

Forest Fire Gym Environment

Switch branches/tags
Nothing to show

## Files

Failed to load latest commit information.
Type
Name
Commit time

# Gym Forest Fire

Forest fire simulation and Gym environment for tackling wildfires with reinforcement learning. The simulation largely follows the forest-fire model of Drossel and Schwabl (1992) which defines the forest fire model as a cellular automaton on a grid with Ld cells, where L is the sidelength of the grid and d is its dimension.

A cell can have three states: empty, occupied by a tree or burning. The Drossel and Schwabl (1992) model is then defined by four rules executed simultaneously:

1. A burning cell turns into an empty cell.
2. A tree will burn with probability q if at least one neighbor is burning.
3. A tree ignites with probability f even if no neighbor is burning.
4. An empty space fills with a tree with probability p.

The figure below shows snapshots of the forest fire simulation.

## Markov Decision Process Formulation

The problem of controlling the forest fire is modeled as a Markov Decision Process (MDP):

• States: An aerial image of the forest showing trees, fires, and empty cells. This is a `128x128` Numpy array with `0` as empty cells, `1` as trees and `10` as the fire cells.
• Actions: The coordinates `[x, y]` of the fire team. The action space is continuous and is normalized to `[0, 1]`. The fire team puts out any fire that exists within a certain radius of `[x, y]`.
• Rewards: At every step, if the fire team has put out any fire, the agent gets a reward of `1`, but if there is a fire in the forest and the fire team has not put out any fire, the agent gets a reward of `-1`. At the final step, if the total trees alive are more than 50% of the initial tree population, the agent gets a reward of `100`, otherwise it gets a reward of `-100`.
• Transition Probability: The environment is modeled following the Drossel and Schwabl (1992) forest-fire model as discussed above.

Note: A good policy should ideally learn when to let the fire burn trees and when to put it out.

## Dependencies

This code has very few dependencies and the simulation is fully vectorized for faster computation.

Requirements:

• Python 3.5+
• OpenAI Gym
• Numpy
• OpenCV (for rendering the environment)

## Instructions

### Setup

Clone the package and install it using `pip`. This install the package and all of its requirements.

``````From ~/gym_forestfire/
pip install -e .
``````

The environment will be registered in the Gym registry. So you can create an instance by simply calling `gym.make()`. You can pass `env_kwargs` for modifying the environment properties.

``````import gym

env = gym.make('gym_forestfire:ForestFire-v0')
env.reset()

for _ in range(500):
env.render()
a = env.action_space.sample()
s, r, d, _ = env.step(a)
env.close()
``````

### Training

The TD3 algorithm with CNN-based actor and critic has been implemented based on author's implementation. To train, simply run:

``````python main.py --env gym_forestfire:ForestFire-v0
``````

But don't expect great learning curves :) The actions saturate quickly even with the Orthogonal Initialization. If you have any ideas for improving the learning, please make a PR.

## Refrences

Forest Fire Gym Environment

## Releases

No releases published

## Packages 0

No packages published