Skip to content

AndrejOrsula/drl_air_hockey

Repository files navigation

Air Hockey with Deep Reinforcement Learning

Docker

This is a participant repository for the Robot Air Hockey Challenge 2023.

drl_air_hockey_self_play.mp4

Overview

The implemented approach focuses on applying a model-based deep reinforcement learning algorithm DreamerV3 to acquire a policy capable of playing air hockey with continuous observations and actions.

  • Additional details about the approach are presented in the report here.
  • Models of pre-trained agents can be downloaded from here.

Instructions

Quick Test

As a quick test, you can try evaluating a pre-trained agent in a self-play mode by running .docker/run.bash directly via curl as shown below.

curl -sSfL "https://raw.githubusercontent.com/AndrejOrsula/drl_air_hockey/main/.docker/run.bash" | DOCKER_RUN_OPTS="--rm" bash -s -- drl_air_hockey/scripts/eval_dreamerv3.py -r

Python

Installation

Install air_hockey_challenge and drl_air_hockey Python modules with pip.

pip3 install git+https://github.com/AndrejOrsula/air_hockey_challenge.git
pip3 install git+https://github.com/AndrejOrsula/drl_air_hockey.git

Warning

Not all dependencies have their versions pinned, e.g. transitive dependencies of dreamerv3. Therefore, the functionality of this repository may be affected in future builds. Consider using the pre-built Docker image for a stable environment.

Training and Evaluation

To train a new agent, you can run the included train_dreamerv3.py script.

scripts/train_dreamerv3.py

To configure the training process, consider modifying any of these files directly:

Once you are satisfied with the training progress, you can evaluate the agent by adjusting the included eval_dreamerv3.py script.

scripts/eval_dreamerv3.py

Docker

To install Docker on your system, you can run .docker/host/install_docker.bash to configure Docker with NVIDIA GPU support.

.docker/host/install_docker.bash

Build Image

To build a new Docker image from Dockerfile, you can run .docker/build.bash as shown below. Note that only the pre-built Docker image already contains models of pre-trained agents.

.docker/build.bash ${TAG:-latest} ${BUILD_ARGS}

Run Container

To run the Docker container, you can use .docker/run.bash as shown below.

.docker/run.bash ${TAG:-latest} ${CMD:-bash}

Run Dev Container

To run the Docker container in a development mode (source code mounted as a volume), you can use .docker/dev.bash as shown below.

.docker/dev.bash ${TAG:-latest} ${CMD:-bash}

As an alternative, VS Code users familiar with Dev Containers can modify the included .devcontainer/devcontainer.json to their needs. For convenience, .devcontainer/open.bash script is available to open this repository as a Dev Container in VS Code.

.devcontainer/open.bash

Join Container

To join a running Docker container from another terminal, you can use .docker/join.bash as shown below.

.docker/join.bash ${CMD:-bash}

Citation

@article{orsula2023learning,
  title   = {{Learning to Play Air Hockey with Model-Based Deep Reinforcement Learning}},
  author  = {Andrej Orsula},
  journal = {NeurIPS 2023 --- The Robot Air Hockey Challenge},
  year    = {2023},
}