This repository contains my submission for Project 1: Navigation of the Udacity Deep Reinforcement Learning Nanodegree.
The assignment is to train an agent that solves the Unity ML-Agents "Banana" environment.
The solution implements a Deep Q-Network based on [1] to solve the environment. For implementation and algorithm details, please see Report.md.
The trained agent is able to effectively navigate the environment, collecting yellow bananas (positive rewards) while avoiding blue bananas (negative rewards).
(The below description is replicated from the udacity/deep-reinforcement-learning repository.)
A reward of +1 is provided for collecting a yellow banana, and a reward of -1 is provided for collecting a blue banana. Thus, the goal of your agent is to collect as many yellow bananas as possible while avoiding blue bananas.
The state space has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around agent's forward direction. Given this information, the agent has to learn how to best select actions. Four discrete actions are available, corresponding to:
0- move forward.1- move backward.2- turn left.3- turn right.
The task is episodic, and in order to solve the environment, your agent must get an average score of +13 over 100 consecutive episodes.
- Python >= 3.6
- A GPU is NOT required; the agent is simple enough to be trained on a standard CPU 🎉
- Clone the repository.
git clone https://github.com/JunShern/dqn-navigation.git- Create a virtual environment to manage your dependencies.
cd dqn-navigation/
python3 -m venv .venv
source .venv/bin/activate # Activate the virtualenv- Install python dependencies
cd dqn-navigation/python
pip install .- Add your virtual environment to jupyter's kernels.
python -m ipykernel install --user --name .venv --display-name "deeprl"-
Download the Banana environment from one of the links below:
- Linux: click here
- Mac OSX: click here
- Windows (32-bit): click here
- Windows (64-bit): click here
Place the file in the root of this repository, and unzip (or decompress) the file.
The project is intended to be run from Navigation.ipynb.
- Run jupyter-notebook
cd dqn-navigation/
source .venv/bin/activate # Activate the virtualenv
jupyter-notebook- Open
Navigation.ipynb. - Follow the instructions in the notebook.
- [1] Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." nature 518.7540 (2015): 529-533.