Skip to content
Bayesian Q-learning
Python Dockerfile
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Bayesian Q-learning with Explicit Uncertainty Measures : Assumed Density Filtering Q-learning (ADFQ)

This repository contains the ADFQ algorithms from the following paper. See the paper for more technical details.

  • Assumed Density Filtering Q-learning ( : H. Jeong, C. Zhang, D. D. Lee, and G. J. Pappas, “Assumed Density Filtering Q-learning,” the 28th International Joint Conference on Artificial Intelligence (IJCAI), Macao, China, 2019


The ADFQ codes for the finite state and action spaces (directly under the ADFQ directory) work both in python 2.7.x and python 3. For deep ADFQ, python 3 (>=3.5) and tensorflow-gpu are recommended.


Clone the repo

git clone
cd ADFQ && source setup

Use the Dockerfile or install the dependencies individually. We include a part of codes from OpenAI baselines (due to the repository stability issue, we are not directly using the current version of the OpenAI baseline git repo). You may need some packages mentioned in the installation guidelines at

Example for running ADFQ algorithm

Classic environments:

python --env loop

And running ADFQ in Cartpole-v0


set callback=None in line 78 if you don't want it to end its training after reaching a maximum time step of a task (e.g. 199 for CartPole).\ Running ADFQ in an atari game, for example, Asterix-v0

python --env AsterixNoFrameskip-v4 --act_policy bayesian

Usage with Target Tracking Environment

This repository also contains example codes to run the presented RL algorithms in the target tracking environments ( Please install the ttenv repository separately in order to use deep_adfq/ or deep_adfq/baselines0/deepq/ The related work is presented in the following paper:

  • Learning Q-network for Active Information Acquisition ( : H. Jeong, B. Schlotfeldt, H. Hassani, M. Morari, D. D. Lee, and G. J. Pappas, “Learning Q-network for Active Information Acquisition,”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macao, China, 2019


If you use this repo in your research, you can cite it as follows:

    author = {Heejin Jeong, Clark Zhang, Daniel D. Lee, George J. Pappas},
    title = {ADFQ_open_source},
    year = {2018},
    publisher = {GitHub},
    journal = {GitHub repository},
    howpublished = {\url{}},
You can’t perform that action at this time.