Tutorial 0 - Introduction to Flow (#656)

* introduction draft

* wrote part 1

* add notes to introduction

* add transition at end of part 2

* structured tutorial

* quick fix

* add environments section

* modify environments according to better-file-names PR

* tutorial0

* add image

* fix old transition

* Minor changes to tutorial

* Language fix

Co-Authored-By: Eugene Vinitsky <eugenevinitsky@users.noreply.github.com>

* Improve tutorial 0

* Add tutorial 0 in README

* Replace subsections titles by horizontal lines

* Update tutorial00_flow.ipynb

* Last modifications to tutorial 0

* Correction in spelling

Changed "refere" to "refer"

tutorials/README.md

@@ -47,6 +47,8 @@ exercises, you may find the
 
 The content of each exercise is as follows:
 
+**Tutorial 0:** High-level introduction to Flow.
+
 **Tutorial 1:** Running SUMO simulations in Flow.
 
 **Tutorial 2:** Running Aimsun simulations in Flow.

tutorials/tutorial00_flow.ipynb

@@ -0,0 +1,117 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Tutorial 0: Flow\n",
+    "\n",
+    "This tutorial gives you a high-level introduction to and a better understanding about what Flow is and how it works. Whether you want to be serious about using Flow or wish to contribute to the project, it may be helpful to understand the basics about how the Flow code is organized and what it does. This tutorial will introduce you to the core concepts used in Flow and is a highly recommended read before you dive into the next tutorials.\n",
+    "\n",
+    "**How to get help:** If you happen, throughout the tutorials or when building your own code, to have any general or technical question related to Flow, don't hesitate to have a look on [Stack Overflow](https://stackoverflow.com/questions/tagged/flow-project) and see if it has been answered already, or otherwise to post it using the tag `flow-project`. We will be happy to help you!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 1. High-level of Flow\n",
+    "\n",
+    "<img src=\"img/flow_venn_diagram.png\" alt=\"Flow Venn Diagram\" width=\"50%\"/>\n",
+    "\n",
+    "Flow acts as a bridge between traffic simulators (e.g. Sumo, Aimsun, ...) and reinforcement learning (RL) libraries (e.g. RLlib, Open AI, ...). It provides you with an interface that lets you train RL agents on a custom road network without having to worry about integration with the traffic simulator and the RL library. Flow creates this connection automatically. Flow also provides you with tools to analyze the trained policies.\n",
+    "\n",
+    "### Running Flow without training\n",
+    "\n",
+    "All you need to run Flow is a scenario. \n",
+    "\n",
+    "- <u>**a scenario**</u>: this is basically the term we use to talk about a road network. A scenario is a class that contains information about the road network on which your agents will be trained. It describes the roads (position, size, number of lanes, speed limit, ...), the connections between the roads (junctions, intersections, ...), and possibly other information (traffic lights, ...).\n",
+    "\n",
+    "Once you have defined this class, the next step is to set up the parameters of the simulation. These include, non-exhaustively, name of the simulation, the scenario to use, the simulator to use _(SUMO, Aimsun, ...)_, the vehicles and/or traffic lights to add to the network, etc.\n",
+    "\n",
+    "You can then run a simulation on this scenario without doing any training, in this case you won't need any \"RL environment\" (explained in next section). The next tutorials will show you how to do just that. Once you have gone through this tutorial, the next tutorials will walk you through the process of creating your own scenarios and RL environments, setting up a simulation so as to train your own agents, and finally visualizing the results.\n",
+    "\n",
+    "### Running Flow with training\n",
+    "\n",
+    "In order to get started and train your own agent on your own road network, you will need: \n",
+    "\n",
+    "- <u>**a scenario**</u>: explained above.\n",
+    "\n",
+    "- <u>**an environment**</u>: this is the RL environment _(**not to be confused** with the physical environment, that we refer to as **scenario**)_. It is a class that allows you to control how the RL agent will be trained. To creat an environment, you will need to specify\n",
+    "    - a **state space** that describes the states of the system that are available to observe. For example, for a vehicle, a state space could be the positions and velocities of all nearby vehicles, as well as its own speed. \n",
+    "    - an **action space** describing how the agent can act in the environment. For example, a standard action for a vehicle would be an acceleration, whereas a standard action for a traffic light would be to switch the traffic light color. \n",
+    "    - a **reward function** describing what the agent should try to maximize. Common rewards include maximizing the speed of the traffic system, the average flow of the traffic system, or the negative of the fuel emissions (a negative is used here to denote the penalty, so that total fuel emissions are minimized). \n",
+    "\n",
+    "Once you have defined these two classes, the last step is to set up the parameters of the simulation. These include, non-exhaustively, name of the simulation, the scenario and environment to use, the simulator to use _(SUMO, Aimsun, ...)_, the RL algorithm to use _(PPO, TRPO, ...)_ and its parameters _(number of iterations, number of CPUs/GPUs to use, discount rate, ...)_, the vehicles and/or traffic lights to add to the network, decision to render the simulation _(not rendering makes training much faster)_, etc.\n",
+    "\n",
+    "### Tools\n",
+    "\n",
+    "During the training or after it has ended, you can use Flow's visualization tools in order to visualize the data saved in the checkpoint files generated during the training. You can see how well your agent is doing by running a new simulation in the simulator, that will used the trained policy (this time, the simulation will be rendered). You can also plot the reward or return functions, time-space diagrams, capacity diagrams etc.\n",
+    "\n",
+    "To ease the process of getting started, Flow comes pre-built with over a dozen scenarios and RL environments that you can use as a starting point. Flow also has a lot of examples that set up simulations, with or without training, using these scenarios and environments in various ways. You can use them as a starting point and modify them according to your needs, or use them as templates to create your own code.\n",
+    "\n",
+    "In the next section, we will give an overview of how Flow's codebase is organized, so that you can have some reference points when you go through the tutorials."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## 2. Codebase structure\n",
+    "\n",
+    "The `flow` codebase directory is structured as follows:\n",
+    "\n",
+    "```python\n",
+    "flow\n",
+    "├── docs  # some random documents, don't worry about it\n",
+    "├── examples  # a lot of example codes using Flow -- this is where you want to head once you're done with the tutorials and want to start doing some real code\n",
+    "│   ├── aimsun  # examples using just Aimsun, without training \n",
+    "│   ├── rllab  # examples of training with rllab (avoid rllab, we are going to deprecate it soon!)\n",
+    "│   ├── rllib  # examples of training with RLlib\n",
+    "│   └── sumo  # examples using just SUMO, without training. You can run these right away to see what the scenarios look like!\n",
+    "├── flow\n",
+    "│   ├── benchmarks  # several custom networks and configurations on which you can evaluate and compare different RL algorithms\n",
+    "│   ├── controllers  # implementations of controllers for the vehicles (IDM, Follower-Stopper...)\n",
+    "│   ├── core  # the core logic of the code -- where the magic happens\n",
+    "│   │   └── kernel\n",
+    "│   │       ├── scenario  # logic for the scenario\n",
+    "│   │       ├── simulation  # where the simulation is created and managed \n",
+    "│   │       ├── traffic_light  # logic for the traffic lights\n",
+    "│   │       └── vehicle  # logic for the vehicles\n",
+    "│   ├── envs  # environments (where states, actions and rewards are handled)\n",
+    "│   ├── multiagent_envs  # multi-agent environments\n",
+    "│   ├── renderer  # pyglet renderer\n",
+    "│   ├── scenarios  # scenarios (ie road networks)\n",
+    "│   ├── utils  # the files that don't fit anywhere else\n",
+    "│   └── visualize  # scripts to replay policies, analyse reward functions etc.\n",
+    "├── scripts  # mostly installation scripts\n",
+    "├── tests  # unit tests\n",
+    "└── tutorials  # <-- you are here\n",
+    "```\n",
+    "\n",
+    "Don't hesitate to go and read the code files directly! We try to keep everything documented and understandable. However if something remains unclear, even after reading all the tutorials and going through the examples, you can ask us on [Stack Overflow](https://stackoverflow.com/questions/tagged/flow-project) using the tag `flow-project` (make sure your question wasn't already asked before!)."
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.7.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}