NeuroDynamicProgramming-MFD [PDF]

Reference

This repository contains the source code of the following research paper:

• Z.C. Su, Andy H.F. Chow, N. Zheng, Y.P. Huang, E.M. Liang, R.X. Zhong, Neuro-dynamic programming for optimal control of macroscopic fundamental diagram systems, Transportation Research Part C: Emerging Technologies, Volume 116, 2020, 102628, ISSN 0968-090X, [Link]

If our paper helps, please cite it as:

@article{ZCSU2020,
title = "Neuro-dynamic programming for optimal control of macroscopic fundamental diagram systems",
journal = "Transportation Research Part C: Emerging Technologies",
volume = "116",
pages = "102628",
year = "2020",
issn = "0968-090X",
author = "Z.C. Su and Andy H.F. Chow and N. Zheng and Y.P. Huang and E.M. Liang and R.X. Zhong",
}

Introduction

The macroscopic fundamental diagram (MFD) can effectively reduce the spatial dimension involved in dynamic optimization of trafc performance for large-scale networks. Solving the Hamilton-Jacobi-Bellman (HJB) equation takes center stage in yielding solutions to the optimal control problem. At the core of solving the HJB equation is the value function that represents choosing a sequence of actions to optimize the system performance. However, this problem generally becomes intractable for possible discontinuities in the solution and the curse of dimensionality for systems with all but modest dimension. To address these challenges, a neural network is used to approximate the value function to obtain the optimal controls through policy iteration. Furthermore, a saturated operator is embedded in the neural network approximator to handle the difculty caused by the control and state constraints. This policy iteration can be implemented as an iterative data-driven technique that integrates with the model-based optimal design based on real-time observations. Numerical experiments are conducted to show that the neuro-dynamic programming approach can achieve optimization goals while stabilizing the system by regulating the traffic state to the desired uncongested equilibrium.

Quick Start

Scenario A: Minimizing system energy (set-point control)

1. Open the main file <Main_NeuroDynamicMFD.m>

2. Check the simulation settings:

   • Initial states of two regions
   • Set points of two regions
   • Demand profile (static) of each directions
   • Control time span
   • Weighted matrix for value function
   • Sample time of Monte-carlo sampling
   • The number of iterations of the Policy Iteration

3. Choose the appropriate kernel function:

   • Open the function file <Calculate_dPHI.m>
   • Select one of those provided kernel functions, and comment others

4. Run the code.

Contact

If you have any question, please feel free to send an E-mail to zichengsu2-c@my.cityu.edu.hk.