Skip to content

poudel-bibek/EnduRL

Repository files navigation

EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning

Alt Text
Our RVs in the Ring


Alt Text
Our RVs in the Bottleneck

Appendix section (appendix.pdf file)

  I. Car Following Filter
  II. Model Based Robot Vehicles
  III. Heuristic Based Robot Vehicles
  IV. Reinforcement Learning (RL) Benchmarks
  V. Congestion Stage Classifier


This work was done on top of FLOW framework obtained on Jan 3, 2023.

Installation instructions

Developed and tested on Ubuntu 18.04, Python 3.7.3

conda env create -f environment.yml
conda activate flow
python setup.py develop
pip install -U pip setuptools
pip install -r requirements.txt

Part 1: Training the Congestion Stage Classifier

If you want to use the trained CSCs, see Data section below.

Part 2: Training RL based RVs

Go to the respective folders for the environment ring/Ours or bottleneck/Ours and enter the command:

# Train our policy in Ring (at 5% penetration) 
python train.py singleagent_ring

# Train our policy in Ring (at penetrations >5%)
python train.py multiagent_ring

# Train our policy in Bottleneck (for all penetrations)
python train.py multiagent_bottleneck

Make sure to change the exp_config files accordingly to specify the RV penetration rate and Safety + Stability or Efficiency mode.

To view tensorboard while training:

tensorboard --logdir=~/ray_results/

Part 3: Generate rollouts for RL based RVs or Heuristic and Model based RVs and save as csv files.

All scripts related to this part are consolidated Evaluate Ring and Evaluate Bottleneck Jupyter Notebooks.

I. RL based RVs:

Replace the method name to be one of: ours, wu

python test_rllib.py [Location of trained policy] [checkpoint number] --method wu --gen_emission --num_rollouts [no_of_rollouts] --shock --render --length 260

II. Heuristic and Model based RVs:

For all (replace the method_name to be one of: bcm, lacc, piws, fs, idm)

python classic.py --method [method_name] --render --length 260 --num_rollouts [no_of_rollouts] --shock --gen_emission

For stability tests where a standard perturbation is applied by a leading HV, include --stability to the line above

Part 4: Evaluate the generated rollouts

To evaluate the generated rollouts into Safety, Efficiency and Stability metrics: Replace the method name to be one of: bcm, idm, fs, piws, lacc, wu, ours

python eval_metrics.py --method [method_name] --num_rollouts [no_of_rollouts]

To add plots to the metrics, include --save_plots

For Stability plots

python eval_plots.py --method [method_name]

Data


Cite


@INPROCEEDINGS{poudel2024endurl,
  title={EnduRL: Enhancing Safety, Stability, and Efficiency of Mixed Traffic Under Real-World Perturbations Via Reinforcement Learning},
  author={Poudel, Bibek and Li, Weizi and Heaslip, Kevin},
  journal={2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year={2024}
}

About

Official code for the EnduRL paper

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published