Update tutorials to match with the new way of running experiments (#802)

* unused parameter

* Update tutorials according to the new example folder

* Adding final changes to the tutorials

* Correct the env_name and network in flow_params

* bug fix

Co-authored-by: Aboudy Kreidieh <akreidieh@gmail.com>

.coveragerc

@@ -13,6 +13,8 @@ omit =
     *aimsun/api.py
     *aimsun/generate.py
     *aimsun/load.py
+    *flow/multiagent_*
+    *flow/singleagent_*
 
 exclude_lines =
     if __name__ == .__main__.:

docs/source/index.rst

@@ -9,7 +9,7 @@ Welcome to Flow
 `Flow <https://flow-project.github.io>`_ is a computational framework for deep RL and control experiments for traffic microsimulation. Visit `our website <https://flow-project.github.io>`_ for more information.
 
 Flow is a work in progress - input is welcome. Available documentation is limited for now.
-`Tutorials <https://github.com/flow-project/flow/tree/master/tutorials/exercises>`_ are available in iPython notebook format.
+`Tutorials <https://github.com/flow-project/flow/tree/master/tutorials>`_ are available in iPython notebook format.
 
 *If you are looking for Akvo Flow, their documentation can be found at* http://flowsupport.akvo.org.
 

examples/exp_configs/non_rl/aimsun_template.py

@@ -7,8 +7,6 @@
 from flow.networks import Network
 import os
 
-# inflow rate at the highway
-FLOW_RATE = 2000
 
 # no vehicles in the network
 vehicles = VehicleParams()
@@ -31,7 +29,7 @@
     # simulator that is used by the experiment
     simulator='aimsun',
 
-    # sumo-related parameters (see flow.core.params.SumoParams)
+    # Aimsun-related parameters
     sim=AimsunParams(
         sim_step=0.1,
         render=True,

tutorials/README.md

@@ -20,10 +20,10 @@ cd <flow-path>/tutorials
 jupyter-notebook
 ```
 
-Instructions are written in each file. To do each exercise, first run all of
+Instructions are written in each file. To do each tutorial, first run all of
 the cells in the Jupyter notebook. Then modify the ones that need to be
 modified in order to prevent any exceptions from being raised. Throughout these
-exercises, you may find the
+tutorials, you may find the
 [Flow documentation](https://flow.readthedocs.io/en/latest/) helpful.
 
 > **Common error:** if, when running a notebook, you run into an error of the form
@@ -45,7 +45,7 @@ exercises, you may find the
 > ```
 > (`cd /` is to make sure that `flow` is not in the folder you run `python` from). Then, in the Python interface that opens, run `import flow`. If you get an `ImportError`, this means you haven't installed Flow in your environment. Go back to the [installation instructions](https://flow.readthedocs.io/en/latest/flow_setup.html), especially the part where you do `pip install -e .` after having done `source activate flow`.
 
-The content of each exercise is as follows:
+The content of each tutorial is as follows:
 
 **Tutorial 0:** High-level introduction to Flow.
 

tutorials/tutorial01_sumo.ipynb

@@ -6,15 +6,15 @@
    "source": [
     "# Tutorial 01: Running Sumo Simulations\n",
     "\n",
-    "This tutorial walks through the process of running non-RL traffic simulations in Flow. Simulations of this form act as non-autonomous baselines and depict the behavior of human dynamics on a network. Similar simulations may also be used to evaluate the performance of hand-designed controllers on a network. This tutorial focuses primarily on the former use case, while an example of the latter may be found in `exercise07_controllers.ipynb`.\n",
+    "This tutorial walks through the process of running non-RL traffic simulations in Flow. Simulations of this form act as non-autonomous baselines and depict the behavior of human dynamics on a network. Similar simulations may also be used to evaluate the performance of hand-designed controllers on a network. This tutorial focuses primarily on the former use case, while an example of the latter may be found in `tutorial09_controllers.ipynb`.\n",
     "\n",
-    "In this exercise, we simulate a initially perturbed single lane ring road. We witness in simulation that as time advances the initially perturbations do not dissipate, but instead propagates and expands until vehicles are forced to periodically stop and accelerate. For more information on this behavior, we refer the reader to the following article [1].\n",
+    "In this tutorial, we simulate a initially perturbed single lane ring road. We witness in simulation that as time advances the initially perturbations do not dissipate, but instead propagates and expands until vehicles are forced to periodically stop and accelerate. For more information on this behavior, we refer the reader to the following article [1].\n",
     "\n",
     "## 1. Components of a Simulation\n",
     "All simulations, both in the presence and absence of RL, require two components: a *network*, and an *environment*. Networks describe the features of the transportation network used in simulation. This includes the positions and properties of nodes and edges constituting the lanes and junctions, as well as properties of the vehicles, traffic lights, inflows, etc. in the network. Environments, on the other hand, initialize, reset, and advance simulations, and act the primary interface between the reinforcement learning algorithm and the network. Moreover, custom environments may be used to modify the dynamical features of an network.\n",
     "\n",
     "## 2. Setting up a Network\n",
-    "Flow contains a plethora of pre-designed networks used to replicate highways, intersections, and merges in both closed and open settings. All these networks are located in flow/networks. In order to recreate a ring road network, we begin by importing the network `RingNetwork`."
+    "Flow contains a plethora of pre-designed networks used to replicate highways, intersections, and merges in both closed and open settings. All these networks are located in `flow/networks`. In order to recreate a ring road network, we begin by importing the network `RingNetwork`."
    ]
   },
   {
@@ -150,15 +150,15 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Importing the `ADDITIONAL_NET_PARAMS` dict from the ring road network, we see that the required parameters are:\n",
+    "Importing the `ADDITIONAL_NET_PARAMS` dictionary from the ring road network, we see that the required parameters are:\n",
     "\n",
     "* **length**: length of the ring road\n",
     "* **lanes**: number of lanes\n",
     "* **speed**: speed limit for all edges\n",
     "* **resolution**: resolution of the curves on the ring. Setting this value to 1 converts the ring to a diamond.\n",
     "\n",
     "\n",
-    "At times, other inputs may be needed from `NetParams` to recreate proper network features/behavior. These requirements can be found in the network's documentation. For the ring road, no attributes are needed aside from the `additional_params` terms. Furthermore, for this exercise, we use the network's default parameters when creating the `NetParams` object."
+    "At times, other inputs may be needed from `NetParams` to recreate proper network features/behavior. These requirements can be found in the network's documentation. For the ring road, no attributes are needed aside from the `additional_params` terms. Furthermore, for this tutorial, we use the network's default parameters when creating the `NetParams` object."
    ]
   },
   {
@@ -198,7 +198,7 @@
    "source": [
     "### 2.5 TrafficLightParams\n",
     "\n",
-    "`TrafficLightParams` are used to describe the positions and types of traffic lights in the network. These inputs are outside the scope of this tutorial, and instead are covered in `exercise06_traffic_lights.ipynb`. For our example, we create an empty `TrafficLightParams` object, thereby ensuring that none are placed on any nodes."
+    "`TrafficLightParams` are used to describe the positions and types of traffic lights in the network. These inputs are outside the scope of this tutorial, and instead are covered in `tutorial10_traffic_lights.ipynb`. For our example, we create an empty `TrafficLightParams` object, thereby ensuring that none are placed on any nodes."
    ]
   },
   {
@@ -218,7 +218,7 @@
    "source": [
     "## 3. Setting up an Environment\n",
     "\n",
-    "Several envionrments in Flow exist to train autonomous agents of different forms (e.g. autonomous vehicles, traffic lights) to perform a variety of different tasks. These environments are often network or task specific; however, some can be deployed on an ambiguous set of networks as well. One such environment, `AccelEnv`, may be used to train a variable number of vehicles in a fully observable network with a *static* number of vehicles."
+    "Several envionrments in Flow exist to train autonomous agents of different forms (e.g. autonomous vehicles, traffic lights) to perform a variety of different tasks. These environments are often network- or task-specific; however, some can be deployed on an ambiguous set of networks as well. One such environment, `AccelEnv`, may be used to train a variable number of vehicles in a fully observable network with a *static* number of vehicles."
    ]
   },
   {
@@ -234,7 +234,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Although we will not be training any autonomous agents in this exercise, the use of an environment allows us to view the cumulative reward simulation rollouts receive in the absence of autonomy.\n",
+    "Although we will not be training any autonomous agents in this tutorial, the use of an environment allows us to view the cumulative reward simulation rollouts receive in the absence of autonomy.\n",
     "\n",
     "Envrionments in Flow are parametrized by three components:\n",
     "* `EnvParams`\n",
@@ -264,7 +264,7 @@
    "source": [
     "### 3.2 EnvParams\n",
     "\n",
-    "`EnvParams` specify environment and experiment-specific parameters that either affect the training process or the dynamics of various components within the network. Much like `NetParams`, the attributes associated with this parameter are mostly environment specific, and can be found in the environment's `ADDITIONAL_ENV_PARAMS` dictionary."
+    "`EnvParams` specify environment and experiment-specific parameters that either affect the training process or the dynamics of various components within the network. Much like `NetParams`, the attributes associated with this parameter are mostly environment-specific, and can be found in the environment's `ADDITIONAL_ENV_PARAMS` dictionary."
    ]
   },
   {
@@ -317,7 +317,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "These objects may be used to simulate rollouts in the absence of reinforcement learning agents, as well as acquire behaviors and rewards that may be used as a baseline with which to compare the performance of the learning agent. In this case, we choose to run our experiment for one rollout consisting of 3000 steps (300 s).\n",
+    "This object may be used to simulate rollouts in the absence of reinforcement learning agents, as well as acquire behaviors and rewards that may be used as a baseline with which to compare the performance of the learning agent. In this case, we choose to run our experiment for one rollout consisting of 3000 steps (300 s).\n",
     "\n",
     "**Note**: When executing the below code, remeber to click on the    <img style=\"display:inline;\" src=\"img/play_button.png\"> Play button after the GUI is rendered."
    ]
@@ -338,11 +338,26 @@
     "# create the environment object\n",
     "env = AccelEnv(env_params, sumo_params, network)\n",
     "\n",
-    "# create the experiment object\n",
-    "exp = Experiment(env)\n",
+    "flow_params = dict(\n",
+    "    exp_tag='ring_example',\n",
+    "    env_name=AccelEnv,\n",
+    "    network=RingNetwork,\n",
+    "    simulator='traci',\n",
+    "    sim=sumo_params,\n",
+    "    env=env_params,\n",
+    "    net=net_params,\n",
+    "    veh=vehicles,\n",
+    "    initial=initial_config,\n",
+    "    tls=traffic_lights,\n",
+    ")\n",
+    "\n",
+    "# number of time steps\n",
+    "flow_params['env'].horizon = 3000\n",
+    "exp = Experiment(flow_params)\n",
+    "exp.env - env\n",
     "\n",
-    "# run the experiment for a set number of rollouts / time steps\n",
-    "_ = exp.run(1, 3000, convert_to_csv=True)"
+    "# run the sumo simulation\n",
+    "_ = exp.run(1, convert_to_csv=True)"
    ]
   },
   {

tutorials/tutorial02_aimsun.ipynb

@@ -12,7 +12,7 @@
     "\n",
     "Simulations of this form act as non-autonomous baselines and depict the behavior of human dynamics on a network. Similar simulations may also be used to evaluate the performance of hand-designed controllers on a network. This tutorial focuses primarily on the former use case, while an example of the latter may be found in `tutorial10_controllers.ipynb`.\n",
     "\n",
-    "In this exercise, we simulate an initially perturbed single lane ring road. We witness in simulation that as time advances, the initial perturbations do not dissipate, but instead propagate and expand until vehicles are forced to periodically stop and accelerate. For more information on this behavior, we refer the reader to the following article [1].\n",
+    "In this tutorial, we simulate an initially perturbed single lane ring road. We witness in simulation that as time advances, the initial perturbations do not dissipate, but instead propagate and expand until vehicles are forced to periodically stop and accelerate. For more information on this behavior, we refer the reader to the following article [1].\n",
     "\n",
     "## 1. Components of a Simulation\n",
     "All simulations, both in the presence and absence of RL, require two components: a *network*, and an *environment*. Networks describe the features of the transportation network used in simulation. This includes the positions and properties of nodes and edges constituting the lanes and junctions, as well as properties of the vehicles, traffic lights, inflows, etc. in the network. Environments, on the other hand, initialize, reset, and advance simulations, and act the primary interface between the reinforcement learning algorithm and the network. Moreover, custom environments may be used to modify the dynamical features of an network.\n",
@@ -147,7 +147,7 @@
     "* **resolution**: resolution of the curves on the ring. Setting this value to 1 converts the ring to a diamond.\n",
     "\n",
     "\n",
-    "At times, other inputs may be needed from `NetParams` to recreate proper network features/behavior. These requirements can be founded in the network's documentation. For the ring road, no attributes are needed aside from the `additional_params` terms. Furthermore, for this exercise, we use the network's default parameters when creating the `NetParams` object."
+    "At times, other inputs may be needed from `NetParams` to recreate proper network features/behavior. These requirements can be founded in the network's documentation. For the ring road, no attributes are needed aside from the `additional_params` terms. Furthermore, for this tutorial, we use the network's default parameters when creating the `NetParams` object."
    ]
   },
   {
@@ -187,7 +187,7 @@
    "source": [
     "### 2.5 TrafficLightParams\n",
     "\n",
-    "`TrafficLightParams` are used to desribe the positions and types of traffic lights in the network. These inputs are outside the scope of this tutorial, and instead are covered in `exercise06_traffic_lights.ipynb`. For our example, we create an empty `TrafficLightParams` object, thereby ensuring that none are placed on any nodes."
+    "`TrafficLightParams` are used to desribe the positions and types of traffic lights in the network. These inputs are outside the scope of this tutorial, and instead are covered in `tutorial10_traffic_lights.ipynb`. For our example, we create an empty `TrafficLightParams` object, thereby ensuring that none are placed on any nodes."
    ]
   },
   {
@@ -242,7 +242,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Although we will not be training any autonomous agents in this exercise, the use of an environment allows us to view the cumulative reward simulation rollouts receive in the absence of autonomy.\n",
+    "Although we will not be training any autonomous agents in this tutorial, the use of an environment allows us to view the cumulative reward simulation rollouts receive in the absence of autonomy.\n",
     "\n",
     "Envrionments in Flow are parametrized by three components:\n",
     "* `EnvParams`\n",
@@ -347,11 +347,26 @@
     "# create the environment object\n",
     "env = AccelEnv(env_params, sim_params, network, simulator='aimsun')\n",
     "\n",
-    "# create the experiment object\n",
-    "exp = Experiment(env)\n",
+    "flow_params = dict(\n",
+    "    exp_tag='test_network',\n",
+    "    env_name=AccelEnv,\n",
+    "    network=RingNetwork,\n",
+    "    simulator='aimsun',\n",
+    "    sim=sim_params,\n",
+    "    env=env_params,\n",
+    "    net=net_params,\n",
+    "    veh=vehicles,\n",
+    "    initial=initial_config,\n",
+    "    tls=traffic_lights\n",
+    ")\n",
+    "\n",
+    "# number of time steps\n",
+    "flow_params['env'].horizon = 3000\n",
+    "exp = Experiment(flow_params)\n",
+    "exp.env = env\n",
     "\n",
-    "# run the experiment for a set number of rollouts / time steps\n",
-    "_ = exp.run(1, 3000)"
+    "# run the Aimsun simulation\n",
+    "_ = exp.run(1)"
    ]
   },
   {