DMAS group A1:
- Daniel Bick (s3145697)
- Thijs Eker (s2576597)
- Ewout Bergsma (s3441423)
- Thomas Bakker (s2468360)
Dear assessors, welcome to the GitHub page of our project. The file you are currently reading is meant to explain all that is required to train, visualize and obtain results of our project. All other README.md
files are from the original repository and not our work, excluding the ones found in the checkpoints
directory.
By using and adjusting the above mentioned orignial GitHub repository we constructed a multi agent traffic simulation, using reinforcement learning. The following files with code were created by us for creating the RL agents:
examples/rllib/multiagent_exps/multiagent_bottleneck.py
flow/envs/multiagent/bottleneck.py
The multiagent_bottleneck.py
file is the main file to run, if one would want to train the model. The bottleneck.py
file holds the environment classes that are used in the aforementioned main file, thus holding code for the state, action and reward.
The following files hold code that were created by us in order to obtain results:
flow/visualize/visualizer_rllib.py
<- Main file to visualize resultsflow/benchmarks/baselines/our_bottleneck.py
<- Baseline variant offlow/benchmarks/our_bottleneck.py
(file can be run)flow/benchmarks/our_bottleneck.py
<- The main benchmark fileflow/benchmarks/our_bottleneck_10_perc_av.py
<- Same as our_bottleneck.py but with 10% AVsflow/benchmarks/rllib/our_ppo_runner.py
<- The file to run the benchmark with PPO agents
Additionally, our report can also be found on this page: Report.pdf
.
In order to train a model Google Colab can be used. Following these instructions:
- Download the
Google_Colab.ipynb
found on this page - Go to the Google Colab website: https://colab.research.google.com
- Log in to a Google account (if not logged in already)
- Upload the file that was uploaded in step 1 (file > upload notebook > navigate to downloaded file and select)
- Follow the instructions in the file that has just opened in the browser
Before installing the project that there are two videos uploaded to Youtube, for visualization purposes. Additionally one can train the model using Google Colab. If one would still want to install the project locally, follow these steps:
- Git clone https://github.com/gitaar9/groupA1-coop-av-flow.git
- Navigate to the base directory of git repo
- Follow the instructions in the
Install_locally.sh
file (may want to open it in a text editor)
For visualization of the simulation there are two options. Firstly, we have uploaded two visualization videos to Youtube:
- Reward function rvelocity with 50% AV: https://youtu.be/O65Y8ObD3qI
- Reward function rtime with 10% AV: https://youtu.be/np_Wf7nnfUo
The second option is to render visualizations locally. This requires to install the project locally first. Afterwards follow these instructions:
- Navigate to the root of this project
- Activate conde env:
conda activate flow
-
Example command to visualize the 160th checkpoint of the r_velocity model:
python3 flow/visualize/visualizer_rllib.py checkpoints/r_velocity/PPO_BottleneckThijsMultiAgentEnv-v0_0_2019-10-24_11-42-2561p4zgee 160
- In sumo click the green play button 3 times (we think this has to do with the fact we use 4 cores)
- When the play button is pressed the 4th time, the simulation will run
- With the step button next to the play button, you can step through the simulation to get a better view of what the cars are doing.
The red cars are the cars controlled by the trained model, white cars are controlled by a simple algorithm designed to mimick human behaviour (provided by the flow library).
Example of command in order to run the code for obtaining results, running from root of this project:
python flow/benchmarks/rllib/our_ppo_runner.py --checkpoint CHECKPOINT_FILE --benchmark_name our_bottleneck --num_rollouts 8 --num_cpus 2
This would append the results for every completed simulation to a file called result. This result file can be summarized using python flow/benchmarks/rllib/summary.py result
Obtaining the baseline results is a lot easier and can be done by running python flow/benchmarks/baselines/our_bottleneck.py