Addressing reward bias in GAIL

Implementation of the paper Addressing reward bias in Adversarial Imitation Learning with neutral reward functions in Tensorflow.

To run the experiments, do the following:

1. Install the repository (instructions below).
2. Save trajectories using PPO agents.
3. Use various reward functions to run GAIL agents (with keywords positive_reward, negative_reward, neutral, and neutral_b).

Imitation Learning Baseline Implementations

This project aims to provide clean implementations of imitation learning algorithms. Currently we have implementations of AIRL and GAIL, and intend to add more in the future.

To install:

conda create -n imitation python=3.8  # python 3.7 and virtualenv are also okay.
conda activate imitation
pip install -e '.[dev]'  # install `imitation` in developer mode

Optional Mujoco Dependency:

Follow instructions to install mujoco_py v1.5 here.

To run:

# Train PPO2 agent on cartpole and collect expert demonstrations
python -m imitation.scripts.expert_demos with cartpole
# Train AIRL on from demonstrations
python -m imitation.scripts.train_adversarial with cartpole airl

View Tensorboard with tensorboard --logdir output/.

Contributing

• Follow the Google Python Style Guide. Examples of Google-style docstrings can be found here.
• Add units tests covering any new features, or bugs that are being fixed.
• PEP8 guidelines with line width 80 and 2-space indents are enforced by ci/lint.sh, which is automatically run by Travis CI.
• Static type checking via pytype is automatically run in ci/type_check.sh.
• Code coverage is automatically enforced by CodeCov. The exact coverage required by CodeCov depends on the previous code coverage %. Files in imitation/{examples,scripts}/ have no coverage requirements.

Citation

If you like this work and want to use it in your research, consider citing us:

@article{jena2020addressing,
  title={Addressing reward bias in Adversarial Imitation Learning with neutral reward functions},
  author={Jena, Rohit and Agrawal, Siddharth and Sycara, Katia},
  journal={arXiv preprint arXiv:2009.09467},
  year={2020}
}