envs

Benchmark Environments

A set of cognitive neuroscience inspired navigation and decision making tasks in environments with either graph or grid structures.

These environments support the default gym interface which is commonly used in open source reinforcement learning packages. You can learn more about gym here.

GridEnv

GridEnv consists of a simple 2D grid environment with various topographies and observation types.

Topographies

The GridEnv class can generate a variety of different maze layouts by setting the topography. Below is a list of the topographies which are supported. Topographies can be set as an enum when initializing the environment. For example:

env = GridEnv(topography=GridTopography.empty)

To add your own topographies, edit grid_topographies.py.

Topography	Image Small (11x11)	Image Large (17 x 17)
empty
four_rooms
outer_ring
two_rooms
u_maze
t_maze
hallways
ring
s_maze
circle
hairpin
i_maze
detour
detour_block

Observation Types

The GridEnv class also supports a variety of observation types for the agent. These vary in the amount of information provided to the agent, and in their format.

Observation types can be set as an enum when initializing the environment. For example:

env = GridEnv(obs_type=GridObsType.index)

To add your own, edit grid_env.py.

Observation	Type	Shape (Fixed Orientation)	Shape (Dynamic Orientation)	Description
index	Tabular	[1]	[1]	An integer number representing the current state of the agent in the environment.
onehot	Function-Approx	[n * n]	[n * n * 4]	A one-hot encoding of the current state of the agent in the environment.
twohot	Function-Approx	[n + n]	[n + n + 4]	Two concatenated one-hot encodings of the agent's x and y coordinates in the environment.
geometric	Function-Approx	[2]	[3]	Two real-valued numbers between 0 and 1 representing the x and y coordinates of the agent in the environment.
boundary	Function-Approx	[n * 4]	[n * 4 + 4]	A matrix corresponding to the one-hot encodings of the distances of the agent from the nearest wall in the four cardinal directions
visual	Function-Approx	[n, n, 3]	[n, n, 3	A 3D tensor corresponding to the RGB image of the environment.
images	Function-Approx	[32, 32, 3]	[32, 32, 3	A 3D tensor corresponding to a unique CIFAR10 image per state.

• Where n is the length of the grid.

GraphEnv

GraphEnv consists of a simple graph environment with various layout structures and observation types.

Structures

Graph structures can be set as an enum when initializing the environment. For example:

env = GraphEnv(graph_structure=GraphStructure.neighborhood)

The graph structures can be added to by editing graph_structures.py

Structure	Image
two_step
linear
t_graph
neighborhood
ring
two_way_linear

Observation Types

The GraphEnv class also supports a variety of observation types for the agent. These vary in the amount of information provided to the agent, and in their format.

Observation types can be set as an enum when initializing the environment. For example:

env = GraphEnv(obs_type=GraphObsType.index)

To add your own, edit graph_env.py.

Observation	Type	Shape	Description
index	Tabular	[1]	An integer number representing the current state of the agent in the environment.
onehot	Function-Approx	[n]	A one-hot encoding of the current state of the agent in the environment.
images	Function-Approx	[32, 32, 3]	[32, 32, 3

• Where n is the number of nodes in the graph.