Skip to content
Branch: master
Find file History
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
images [snake-dqn] Add graphics and inference page (#270) May 6, 2019
.babelrc [snake-dqn] Initial commit (#265) Apr 23, 2019
.gitignore [snake-dqn] Add graphics and inference page (#270) May 6, 2019
README.md [DOC] Add hosted tensorboard summaries (tensorboard.dev links) to REA… Nov 12, 2019
agent.js [snake-dqn] fix: typo (#325) Aug 23, 2019
agent_test.js
dqn.js [snake-dqn] Fix a bug in copyWeights(); Upgrade to tfjs 1.2.7 (#323) Aug 19, 2019
dqn_test.js [snake-dqn] Fix a bug in copyWeights(); Upgrade to tfjs 1.2.7 (#323) Aug 19, 2019
index.html [snake-dqn] Add graphics and inference page (#270) May 6, 2019
index.js [snake-dqn] Add graphics and inference page (#270) May 6, 2019
package.json Upgrade all tfjs-node-based examples to v1.3.2 (#341) Dec 1, 2019
replay_memory.js [snake-dqn] Initial commit (#265) Apr 23, 2019
replay_memory_test.js [snake-dqn] Initial commit (#265) Apr 23, 2019
run_tests.js
snake_game.js [snake-dqn] Add graphics and inference page (#270) May 6, 2019
snake_game_test.js [snake-dqn] Add graphics and inference page (#270) May 6, 2019
snake_graphics.js [snake-dqn] Add graphics and inference page (#270) May 6, 2019
train.js [snake-dqn] Add graphics and inference page (#270) May 6, 2019
train_test.js [snake-dqn] Initial commit (#265) Apr 23, 2019
utils.js [snake-dqn] Initial commit (#265) Apr 23, 2019
utils_test.js [snake-dqn] Initial commit (#265) Apr 23, 2019
yarn.lock Upgrade all tfjs-node-based examples to v1.3.2 (#341) Dec 1, 2019

README.md

Using Deep Q-Learning to Solve the Snake Game

DQN Snake Game

See this example live!

Deep Q-Learning is a reinforcement-learning (RL) algorithm. It is used frequently to solve arcade-style games like the Snake game used in this example.

The Snake game

The Snake game is a grid-world action game in which the player controls a virtual snake that keeps moving on the game board (9x9 by default). At each step, there are four possible actions: left, right, up, and down. To achieve higher scores (rewards), the player should guide the snake to the fruits on the screen and "eat" them, while avoiding

  • its head going off the board, and
  • its head bumping into its own body.

This example consists of two parts:

  1. Training the Deep Q-Network (DQN) in Node.js
  2. Live demo in the browser

Training the Deep Q-Network in Node.js

To train the DQN, use command:

yarn
yarn train

If you have a CUDA-enabled GPU installed on your system, along with all the required drivers and libraries, append the --gpu flag to the command above to let use the GPU for training, which will lead to a significant increase in the training speed:

yarn train --gpu

To monitor the training progress using TensorBoard, use the --logDir flag and point it to a log directory, e.g.,

yarn train --logDir /tmp/snake_logs

During the training, you can use TensorBoard to visualize the curves of

  • Cumulative reward values from the games
  • Training speed (game frames per second)
  • Value of the epsilon from the epsilon-greedy algorithm and so forth.

Specifically, open a separate terminal. In the terminal, install tensorboard and launch the backend server of tensorboard:

pip install tensorboard
tensorboard --logdir /tmp/snake_logs

A detailed TensorBoard training log is hosted and viewable at this TensorBoard.dev link.

Once started, the tensorboard backend process will print an http:// URL to the console. Open your browser and navigate to the URL to see the logged curves.

Running the demo in the browser

After the DQN training completes, you can use the following command to launch a demo that shows how the network plays the game in the browser:

yarn watch
You can’t perform that action at this time.