Zeroth-Order Actor-Critic: An Evolutionary Framework for Sequential Decision Making

Prerequisites

The results in the paper is based on the following suggested environment:

• Python == 3.8.18
• SUMO == 2.3.7
• traci == 1.21.0
• gops == 1.1.0
• nevergrad == 0.10.0
• gymnasium == 0.28.1
• torch == 2.0.1
• casadi == 3.6.3
• ray == 2.9.1
• hydra-core == 1.3.2

How to Use

1. We use hydra to configure the hyperparameters of ZOAC. See conf/config_zoac.yaml for details;
2. Simply run python main.py to evaluate ZOAC;
3. You may also run python run_baseline.py to evaluate evolutionary baselines from nevergrad.

Reference

@article{lei2025zeroth,
  title={Zeroth-Order Actor-Critic: An Evolutionary Framework for Sequential Decision Making},
  author={Lei, Yuheng and Lyu, Yao and Zhan, Guojian and Zhang, Tao and Li, Jiangtao and Chen, Jianyu and Li, Shengbo Eben and Zheng, Sifa},
  journal={IEEE Transactions on Evolutionary Computation},
  year={2025},
  publisher={IEEE}
}