Skip to content
Branch: master
Find file History
Robert Dadashi Copybara-Service
Robert Dadashi and Copybara-Service Add Value Function Polytope colab.
PiperOrigin-RevId: 246405745
Latest commit 5e3da5f May 2, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
AUTHORS Add Value Function Polytope colab. May 2, 2019
README.md Add Value Function Polytope colab. May 2, 2019
polytope.ipynb Add Value Function Polytope colab. May 2, 2019
requirements.txt Add Value Function Polytope colab. May 2, 2019

README.md

Visualization for the Value Function Polytope in Reinforcement Learning

Colab for generating the figures in the ICML 2019 paper: The Value Function Polytope in Reinforcement Learning (https://arxiv.org/abs/1901.11524)

Abstract:

We establish geometric and topological properties of the space of value functions in finite state-action Markov decision processes. Our main contribution is the characterization of the nature of its shape: a general polytope (Aigner et al., 2010). To demonstrate this result, we exhibit several properties of the structural relationship between policies and value functions including the line theorem, which shows that the value functions of policies constrained on all but one state describe a line segment. Finally, we use this novel perspective to introduce visualizations to enhance the understanding of the dynamics of reinforcement learning algorithms.

You can’t perform that action at this time.