Skip to content

Commit

Permalink
add plots to getting started guide
Browse files Browse the repository at this point in the history
  • Loading branch information
cpnota committed Jan 15, 2020
1 parent aabe564 commit 7705efd
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 14 deletions.
47 changes: 33 additions & 14 deletions docs/source/guide/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,26 @@ Prerequisites
-------------

The Autonomous Learning Library requires a recent version of Pytorch (>= 10.3).
Additionally, Tensorboard is required in order to enable logging,
which requires the installation of Tensorflow.
We recommond installing these through `link Conda <https://docs.conda.io/en/latest/>`.

We have also tested on Unix-based machines (Linux and MacOS).
However, there is no inherent reason why it shouldn
Additionally, Tensorboard is required in order to enable logging.
We recommond installing these through `Conda <https://docs.conda.io/en/latest/>`_.
We also strongly recommend using a machine with a fast GPU (a GTX 970 or better).

Installation
------------

Right now, the Autonomous Learning Library can only be installed by cloning the Github repository::
The ``autonomous-learning-library`` can be installed from PyPi using ``pip``:

.. code-block:: bash
pip install autonomous-learning-library
If you're lazy, you can install the prerequisites as well using:

.. code-block:: bash
pip install autonomous-learning-library[pytorch]
An alternate approach, that may be useful when following this tutorial, is to instead install by cloning the Github repository:

git clone https://github.com/cpnota/autonomous-learning-library.git
cd autonomous-learning-library
Expand All @@ -25,9 +34,7 @@ You can test your installation by running the tests::

make test

If you do not have Make installed, the tests can by run with::

python -m unittest discover -s all -p "*test.py"
This should also tell you if CUDA (the GPU driver) is unavailable.

Running a Preset Agent
----------------------
Expand All @@ -37,20 +44,32 @@ However, the library includes a number of "preset" agent configurations for easy
It also includes some useful scripts for running the presets.
For example, an A2C agent can be run on Cart-Pole as follows::

python scripts/classic.py CartPole-v0 a2c
python scripts/classic.py CartPole-v0 ppo

The results will be written to `runs/_a2c <id>`, where `<id>` is some some string generated by the library.
You can view these results and other information through `tensorboard`::
The results will be written to ``runs/_a2c <id>``, where ``<id>`` is some some string generated by the library.
You can view these results and other information through `tensorboard`:

tensorboard --logdir runs

By opening your browser to <http://localhost:6006>, you should see a dashboard that looks something like the following (you may need to adjust the "smoothing" parameter):

.. image:: tensorboard.png

If you want to compare agents in a nicer, format, you can use the `plot` script::

python scripts/plot.py runs

This should give you a plot similar to the following:

.. image:: plot.png

In this plot, each point represents the average of the episodic returns over the last 100 episodes.
The shaded region represents the standard deviation over that interval.

Finally, to watch the trained model in action, we provide a `watch` scripts for each preset module::

python scripts/watch_classic.py CartPole-v0 "runs/_a2c <id>"

You need to find the <id> by checking the ``runs`` directory.

Be sure to check out the `atari` and `continuous` presets for more fun!
You will need CUDA and a decent GPU in order to achieve good performance.
Binary file added docs/source/guide/plot.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/guide/tensorboard.png
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 7705efd

Please sign in to comment.