A development tool for evaluation and interpretability of reinforcement learning agents.

Installation

pip install rld

Usage

Firstly, calculate attributations for your rollout using:

rld attribute [--rllib] [--out <ROLLOUT>] config.py <INPUT_ROLLOUT>

This will take INPUT_ROLLOUT (possibly in the Ray RLlib format, if --rllib is set) and calculate attributations for each timestep in each trajectory, using the configuration stored in config.py. The output file will be stored as ROLLOUT. See the Config class for possible configuration.

Once the attributations are calculated, you can visualize them using:

rld start --viewer <VIEWER_ID> <ROLLOUT>

See the examples for reference.

Description

rld provides a set of tools to evaluate and understand behaviors of reinforcement learning agents. Under the hood, rld uses Captum to calculate attributations of observation components. rld is also integrated with Ray RLlib library and allows to load agents trained in RLlib.

Current limitations

rld is currently in its early development stage, thus the functionality is very limited.

RL algorithms

rld is algorithm-agnostic, but currently it is more suitable for policy-based methods. This is due to the fact that the Model is now expected to output logits for a given observation. This, however, will change in the future, and rld will support more algorithms.

Viewers

This is the list of viewers, which ship with rld:

• none
• cartpole
• atari

You can easily create your own viewer, for your own environment, but to make it visible for rld, you have to rebuild the project. This will be improved in the future.

Observation and action spaces

The table below presents currently supported observation and action spaces.

		Action space
		Discrete	MultiDiscrete
Obs space	Box	✔️	✔️
	Dict	✔️	✔️

Roadmap

See the issues page to see the list of features planned for the future releases. If you have your own ideas, you are encouraged to post them there.