-
Notifications
You must be signed in to change notification settings - Fork 0
MARLBEN Environments
In all environments, the map is a square or rectangular field, consisting of cells of different types. Each agent controls a character occupying one of the cells of the map. The character has numerical characteristics of health and two resources: water and food. Each turn, the agent loses one unit of both resources. If the level of one of the agent's resources is at zero, the agent loses health instead of that resource. The goal of each agent is to survive a fixed number of moves. If an agent's health drops to zero, their episode ends prematurely.
Cell types include:
- Grass - a base cell on which an agent can be placed.
- Forest - replenishes the food resource upon interaction. To interact, an agent must stand on a forest space. After the interaction, the forest is replaced by a bush (analogous to grass) for several turns, during which no agent can replenish the food resource on this cell.
- River - replenishes the resource of water when interacting with an agent. To interact, the agent must stand next to the river space.
- Rock - a base tile that cannot contain an agent.
- Lava - a cell that reduces the agent's health to zero.
Agents can move to neighboring cells in horizontal or vertical directions. Agents can only occupy grass, forest, or shrub spaces. When attempting to move onto a lava tile, the agent's health is reduced to zero. When trying to move towards river or stone tiles, the agent remains in place.
The information available to the agent in each state of the environment includes the type of each cell in a certain visibility radius around the agent, the characteristics of the agent itself (health and amount of resources), as well as the characteristics and location of other agents within its visibility radius.
An additional complication for reinforcement learning algorithms in our environments is the sparse reward. As a rule, the agent receives the only reward signal when the episode is terminated. This signal is positive (+1) if the agent successfully survives the fixed episode length and negative (-1) otherwise. It is worth noting that for algorithms from some other categories (for example, evolutionary), this feature does not play a role.