Merge branch 'master' of https://github.com/flow-project/flow into pl…

…otly_240

.coveragerc

@@ -13,6 +13,8 @@ omit =
     *aimsun/api.py
     *aimsun/generate.py
     *aimsun/load.py
+    *flow/multiagent_*
+    *flow/singleagent_*
 
 exclude_lines =
     if __name__ == .__main__.:

README.md

@@ -21,7 +21,7 @@ See [our website](https://flow-project.github.io/) for more information on the a
 
 # Technical questions
 
-Please direct your technical questions to [Stack Overflow](https://stackoverflow.com) using the [flow-project](https://stackoverflow.com/questions/tagged/flow-project) tag.
+If you have a bug, please report it. Otherwise, join the [Flow Users group](https://forms.gle/CuVBu6QtX3dfNaxz6) on Slack! You'll recieve an email shortly after filling out the form. 
 
 # Getting involved
 

docs/source/examples.rst

@@ -43,11 +43,10 @@ In this scenario, human drivers placed on a ring develop a travelling wave that
 decelerations and lowers the average velocity of the system. The goal is to train a single
 autonomous vehicle to eliminate the shockwave.
 
-The pre-built run script for running this in human-only mode is `examples/sumo/sugiyama.py`.
-The run scripts in rllib and aimsun are `examples/rllib/stabilizing_the_ring.py` and
-`examples/aimsun/stabilizing_the_ring.py`. These control environments are partially observed:
-the autonomous vehicle only observes its own distance to the leading vehicle, its speed,
-and the speed of the leading vehicle.
+The experiment config for this in human-only mode (non-RL) is `examples/exp_configs/non_rl/ring.py`.
+The experiment config for RL-based is `examples/exp_configs/rl/singleagent/singleagent_ring.py`. 
+These control environments are partially observed: the autonomous vehicle only observes its own 
+distance to the leading vehicle, its speed, and the speed of the leading vehicle.
 
 To make this task more difficult, the environment has a configurable parameter, `ring_length`, which
 can be set to a list containing the minimum and maximum ring-size. The autonomous vehicle must

docs/source/flow_setup.rst

@@ -110,8 +110,8 @@ Note that, if the above commands did not work, you may need to run
 *Troubleshooting*:
 If you are a Mac user and the above command gives you the error ``FXApp:openDisplay: unable to open display :0.0``, make sure to open the application XQuartz.
 
-Testing your installation
-~~~~~~~~~~~~~~~~~~~~~~~~~
+Testing your SUMO and Flow installation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Once the above modules have been successfully installed, we can test the
 installation by running a few examples. Before trying to run any examples, be
@@ -124,17 +124,17 @@ sure to enter your conda environment by typing:
 Let’s see some traffic action:
 
 ::
-    
-    python examples/sumo/sugiyama.py
+
+    python examples/simulate.py ring
 
 Running the following should result in the loading of the SUMO GUI.
 Click the run button and you should see unstable traffic form after a
 few seconds, a la (Sugiyama et al, 2008). This means that you have Flow
-properly configured with SUMO and Flow!
+properly configured with SUMO!
 
 
 (Optional) Installing Aimsun
------------------
+----------------------------
 
 In addition to SUMO, Flow supports the use of the traffic simulator "Aimsun".
 In order setup Flow with Aimsun, you will first need to install Aimsun. This
@@ -178,15 +178,15 @@ The latter command should return an output similar to:
 
     /path/to/envs/aimsun_flow/bin/python
 
-Copy the path up until right before /lib (i.e. /path/to/envs/aimsun_flow/bin/python) and
+Copy the path up until right before /lib (i.e. /path/to/envs/aimsun_flow) and
 place it under the `AIMSUN_SITEPACKAGES` variable in your bashrc, like this:
 
 ::
 
-    export AIMSUN_SITEPACKAGES="/path/to/envs/aimsun_flow/bin/python"
+    export AIMSUN_SITEPACKAGES="/path/to/envs/aimsun_flow"
 
-Testing your installation
-~~~~~~~~~~~~~~~~~~~~~~~~~
+Testing your Aimsun installation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 To test that you installation was successful, you can try running one of the
 Aimsun examples within the Flow main directory. In order to do so, you need
@@ -196,8 +196,10 @@ to activate the `flow` env. Type:
 
     source deactivate aimsun_flow
     source activate flow
-    python examples/aimsun/sugiyama.py
+    python examples/simulate.py ring --aimsun
 
+*Troubleshootig for Ubuntu users with Aimsun 8.4*: when you run the above example, you may get a subprocess.Popen error ``OSError: [Errno 8] Exec format error:``.  
+To fix this, go to the `Aimsun Next` main directory, open the `Aimsun_Next` binary with a text editor and add the shebang to the first line of the script ``#!/bin/sh``.
 
 (Optional) Install Ray RLlib
 ----------------------------
@@ -226,22 +228,8 @@ required libraries as specified at
 <http://ray.readthedocs.io/en/latest/installation.html> and
 then follow the setup instructions.
 
-(Optional) Install Stable Baselines
-----------------------------
-
-An additional library that Flow supports is the fork of OpenAI's Baselines, Stable-Baselines.
-First visit <https://stable-baselines.readthedocs.io/en/master/guide/install.html> and
-install the required packages and pip install the stable baselines package as described in their
-installation instructions.
-
-You can test your installation by running
-
-::
-
-    python examples/stable_baselines/stabilizing_the_ring.py
-
-Testing your installation
-~~~~~~~~~~~~~~~~~~~~~~~~~
+Testing your RLlib installation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 See `getting started with RLlib <http://ray.readthedocs.io/en/latest/rllib.html#getting-started>`_ for sample commands.
 
@@ -255,11 +243,34 @@ In order to test run an Flow experiment in RLlib, try the following command:
 
 ::
 
-    python examples/rllib/stabilizing_the_ring.py
+    python examples/train_rllib.py singleagent_ring
+
 
 If it does not fail, this means that you have Flow properly configured with
 RLlib.
 
+(Optional) Install Stable Baselines
+-----------------------------------
+
+An additional library that Flow supports is the fork of OpenAI's Baselines, Stable-Baselines.
+First visit <https://stable-baselines.readthedocs.io/en/master/guide/install.html> and
+install the required packages and pip install the stable baselines package as described in their
+installation instructions.
+
+Testing your Stable Baselines installation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can test your installation by running
+
+::
+
+    python examples/train_stable_baselines.py singleagent_ring
+
+
+
+(Optional) Visualizing with Tensorboard
+---------------------------------------
+
 To visualize the training progress:
 
 ::
@@ -283,6 +294,8 @@ jobs from there.
     ray teardown scripts/ray_autoscale.yaml
 
 
+
+
 (Optional) Direct install of SUMO from GitHub
 ---------------------------------------------
 

docs/source/index.rst

@@ -9,7 +9,7 @@ Welcome to Flow
 `Flow <https://flow-project.github.io>`_ is a computational framework for deep RL and control experiments for traffic microsimulation. Visit `our website <https://flow-project.github.io>`_ for more information.
 
 Flow is a work in progress - input is welcome. Available documentation is limited for now.
-`Tutorials <https://github.com/flow-project/flow/tree/master/tutorials/exercises>`_ are available in iPython notebook format.
+`Tutorials <https://github.com/flow-project/flow/tree/master/tutorials>`_ are available in iPython notebook format.
 
 *If you are looking for Akvo Flow, their documentation can be found at* http://flowsupport.akvo.org.
 

docs/source/multiagent.rst

@@ -50,4 +50,4 @@ A brief example of a multiagent environment:
 
 
 For further details look at our
-`multiagent examples <https://github.com/flow-project/flow/tree/master/examples/rllib/multiagent_exps>`_.
+`multiagent examples <https://github.com/flow-project/flow/tree/master/examples/exp_configs/rl/multiagent>`_.

environment.yml

@@ -3,7 +3,7 @@ name: flow
 dependencies:
     - python==3.6.8
     - scipy==1.1.0
-    - lxml==4.2.4
+    - lxml==4.4.1
     - six==1.11.0
     - path.py
     - python-dateutil==2.7.3

examples/README.md

@@ -1,40 +1,79 @@
 # Flow Examples
 
 Before continuing to the Flow examples, we recommend **installing Flow** by 
-executing the following [installation instructions](
+following the [installation instructions](
 https://flow.readthedocs.io/en/latest/flow_setup.html).
 
 The **examples** folder provides several examples demonstrating how 
-both simulation and RL-oriented experiments can be setup and executed within 
-the Flow framework on a variety of traffic problems. These examples are .py 
-files that may be executed  either from terminal or via an editor. For example,
-in order to execute the  sugiyama example in *examples/sumo*, we run:
+both non-RL simulation and RL-oriented simulatons can be setup and executed 
+within the Flow framework on a variety of traffic problems. These examples are 
+python files that may be executed either from terminal or via a text editor. 
+For example, in order to execute the non-RL Ring example we run:
 
 ```shell
-python <flow-path>/examples/sumo/sugiyama.py
+python simulate.py ring
 ```
 
-The examples are distributed into the following sections:
+The examples are categorized into the following 3 sections:
 
-**examples/sumo/** contains examples of transportation network with vehicles
+**non-RL examples** contains examples of transportation network with vehicles
 following human-dynamical models of driving behavior using the traffic 
-micro-simulator sumo.
+micro-simulator sumo and traffic macro-simulator Aimsun.
 
-**examples/aimsun/** contains examples of transportation network with vehicles
-following human-dynamical models of driving behavior using the traffic 
-micro-simulator Aimsun.
+To execute these examples, run
+
+```shell
+python simulate.py EXP_CONFIG 
+```
+where `EXP_CONFIG` is the name of the experiment configuration file, as located in 
+`exp_configs/non_rl.`
+
+There are several *optional* arguments that can be added to the above command:
+
+```shell
+ python simulate.py EXP_CONFIG --num_runs n --render --aimsun --gen_emission
+```
+where `--num_runs` indicates the number of simulations to run (default of `n` is 1), `--render` indicates whether to run the simulation during runtime (default is False), `--aimsun` indicates whether to run the simulation using the simulator Aimsun (the default simulator is SUMO), and `--gen_emission` indicates whether to generate an emission file from the simulation.
+
+**RL examples based on RLlib** provides similar networks as those presented in 
+the first point, but in the present of autonomous vehicle (AV) or traffic light agents 
+being trained through RL algorithms provided by *RLlib*.
+
+To execute these examples, run
+
+```shell
+ python train_rllib.py EXP_CONFIG
+```
+where `EXP_CONFIG` is the name of the experiment configuration file, as located in 
+`exp_configs/rl/singleagent` or  `exp_configs/rl/multiagent.`
 
-**examples/rllib/** provides similar networks as those presented in the 
-previous point, but in the present of autonomous vehicle (AV) or traffic light 
-agents being trained through RL algorithms provided by *RLlib*.
 
+**RL examples based on stable_baselines** provides similar networks as those 
+presented in the first point, but in the present of autonomous vehicle (AV) or traffic 
+light agents being trained through RL algorithms provided by OpenAI *stable 
+baselines*.
+
+To execute these examples, run
+
+```shell
+ python train_stable_baselines.py EXP_CONFIG
+```
+where `EXP_CONFIG` is the name of the experiment configuration file, as located in 
+`exp_configs/rl/singleagent.`
+
+There are several *optional* arguments that can be added to the above command:
+
+```shell
+ python train_stable_baselines.py EXP_CONFIG --num_cpus n1 --num_steps n2 --rollout_size r
+```
+where `--num_cpus` indicates the number of CPUs to use (default of `n1` is 1), `--num_steps` indicates the total steps to perform the learning (default of `n2` is 5000), and `--rollout_size` indicates the number of steps in a training batch (default of `r` is 1000)
 
 ## Simulated Examples
 
-The following networks are available for simulation within flow, and 
-specifically the **examples/sumo** folder. Similar networks are available with 
-trainable variants in the examples/rllib and examples/aimsun folders; however, 
-they may be under different names.
+The following networks are available for simulation within flow. These examples are 
+all available as non-RL examples, while some of them are also available (with 
+trainable variants) as RL examples, with RLlib or Stable Baselines.
+
 
 ### bay_bridge.py \& bay_bridge_toll.py
 
@@ -45,7 +84,7 @@ only on the toll booth and sections of the road leading up to it.
 
 ![](https://raw.githubusercontent.com/flow-project/flow/master/docs/img/bay_bridge.gif)
 
-### bottlenecks.py
+### bottleneck.py
 
 Example demonstrating formation of congestion in bottleneck
 
@@ -73,9 +112,15 @@ Example of an open multi-lane network with human-driven vehicles.
 
 ![](https://raw.githubusercontent.com/flow-project/flow/master/docs/img/highway.gif)
 
+### highway_ramps.py
+
+Example of a highway section network with on/off ramps
+
+![](picture to be added)
+
 ### merge.py
 
-Example of a merge network with human-driven vehicles.
+Example of a straight road with an on-ramp merge.
 
 In the absence of autonomous vehicles, the network exhibits properties of
 convective instability, with perturbations propagating upstream from the merge
@@ -85,15 +130,16 @@ point before exiting the network.
 
 ### minicity.py
 
-Example of modified minicity of University of Delaware network with 
-human-driven vehicles.
+Example of modified mini city developed under a 
+[collaboration with University of Delaware](https://sites.google.com/view/iccps-policy-transfer),
+with human-driven vehicles.
 
 ![](https://raw.githubusercontent.com/flow-project/flow/master/docs/img/minicity.gif)
 
-### sugiyama.py
+### ring.py
 
-Used as an example of sugiyama experiment.
+Used as an example of a ring experiment.
 
-This example consists of 22 IDM cars on a ring road creating shockwaves.
+This example consists of 22 IDM cars driving on a ring road creating shockwaves.
 
 ![](https://raw.githubusercontent.com/flow-project/flow/master/docs/img/sugiyama.gif)