Skip to content
An open-source implementation of the AlphaGoZero algorithm
C++ Python TypeScript Shell Starlark HTML Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cc fix bug in resign_threshold calculation Jan 30, 2020
cluster update docker images to TF 1.15.0 Nov 21, 2019
minigui Adds `//cc:concurrent_selfplay`. Oct 18, 2019
ml_perf sample fixed fraction of examples from fixed number of games Jan 30, 2020
oneoffs Bump handlebars from 4.1.2 to 4.5.3 in /oneoffs/joseki (#953) Jan 29, 2020
ratings "Rate_subdir" utility (#918) Oct 14, 2019
rl_loop clean up Oct 1, 2019
sgf Added comments that games are public domain Feb 1, 2018
testing update docker images to TF 1.15.0 Nov 21, 2019
tests TPU Nov 20, 2019
.bazelrc add generated .bazelrc files to .gitignore Jul 30, 2019
.gitignore Adds a "Joseki Explorer" React App (#875) Sep 10, 2019
.pylintrc use explicit imports for tf.contrib modules Nov 21, 2019
CODE_OF_CONDUCT.md Add code of conduct. Jan 30, 2018
CONTRIBUTING.md Several python cleanups. Feb 1, 2018
LICENSE initial commit Nov 10, 2017
README.md update to TF 1.15.0 Nov 21, 2019
RESULTS.md more results.md comments Jan 16, 2019
WORKSPACE update to TF 1.15.0 Nov 21, 2019
__init__.py Several python cleanups. Feb 1, 2018
batch_exporter.py Slide export window forward, add thinning option Apr 30, 2019
bigtable_input.py use explicit imports for tf.contrib modules Nov 21, 2019
bigtable_output.py [EvalServer] Add ringmaster_evaluator (#919) Oct 25, 2019
bootstrap.py Clean up BOARD_SIZE=19 (#687) Feb 4, 2019
cloud_logging.py Glue all the RL loop bits back together (#315) Jul 16, 2018
coords.py s/MiniGo/Minigo/ (#734) Feb 27, 2019
dual_net.py Add support for NCHW input features Dec 14, 2019
dual_net_edge_tpu.py Added docstrings. Apr 8, 2019
evaluate.py Use a random fake model for bootstrapping Mar 5, 2019
features.py architecture tweaks & checkpoint support Dec 27, 2019
freeze_graph.py fix dual_net_test Aug 12, 2019
go.py adds a script to inspect training examples Oct 21, 2019
gtp.py Fixing earlier bad merge. Apr 8, 2019
gtp_cmd_handlers.py pylint + flake8 (#698) Feb 8, 2019
gtp_engine.py Minigui explore mode Nov 13, 2018
mask_flags.py Disable pass during bootstrap games, tweak features Oct 25, 2019
mcts.py Convert to double quote docstrings per PEP257 (#779) Mar 18, 2019
minigo_model.py Add support for NCHW input features Dec 14, 2019
notes.txt k8s ringmaster job (#676) Feb 7, 2019
player_interface.py fix pylint errors Aug 2, 2019
preprocessing.py Add support for NCHW input features Dec 14, 2019
requirements-analysis.txt Add cbt_ratings.py backed by cbt (#819) Apr 9, 2019
requirements-colab.txt New requirements file for upcoming project (#672) Feb 1, 2019
requirements.txt add choix to requirements.txt Oct 17, 2019
selfplay.py Convert to double quote docstrings per PEP257 (#779) Mar 18, 2019
sgf_wrapper.py Convert to double quote docstrings per PEP257 (#779) Mar 18, 2019
strategies.py Convert to double quote docstrings per PEP257 (#779) Mar 18, 2019
symmetries.py address review comments Dec 14, 2019
test.sh Add test scripts for pull-tf-minigo-cc (#547) Nov 6, 2018
train.py architecture tweaks & checkpoint support Dec 27, 2019
tsconfig.json make 'back to main line' button work Nov 13, 2018
utils.py Use a random fake model for bootstrapping Mar 5, 2019
validate.py mlperf 0.7 tweaks Dec 27, 2019

README.md

Minigo: A minimalist Go engine modeled after AlphaGo Zero, built on MuGo

This is an implementation of a neural-network based Go AI, using TensorFlow. While inspired by DeepMind's AlphaGo algorithm, this project is not a DeepMind project nor is it affiliated with the official AlphaGo project.

This is NOT an official version of AlphaGo

Repeat, this is not the official AlphaGo program by DeepMind. This is an independent effort by Go enthusiasts to replicate the results of the AlphaGo Zero paper ("Mastering the Game of Go without Human Knowledge," Nature), with some resources generously made available by Google.

Minigo is based off of Brian Lee's "MuGo" -- a pure Python implementation of the first AlphaGo paper "Mastering the Game of Go with Deep Neural Networks and Tree Search" published in Nature. This implementation adds features and architecture changes present in the more recent AlphaGo Zero paper, "Mastering the Game of Go without Human Knowledge". More recently, this architecture was extended for Chess and Shogi in "Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm". These papers will often be abridged in Minigo documentation as AG (for AlphaGo), AGZ (for AlphaGo Zero), and AZ (for AlphaZero) respectively.

Goals of the Project

  1. Provide a clear set of learning examples using Tensorflow, Kubernetes, and Google Cloud Platform for establishing Reinforcement Learning pipelines on various hardware accelerators.

  2. Reproduce the methods of the original DeepMind AlphaGo papers as faithfully as possible, through an open-source implementation and open-source pipeline tools.

  3. Provide our data, results, and discoveries in the open to benefit the Go, machine learning, and Kubernetes communities.

An explicit non-goal of the project is to produce a competitive Go program that establishes itself as the top Go AI. Instead, we strive for a readable, understandable implementation that can benefit the community, even if that means our implementation is not as fast or efficient as possible.

While this product might produce such a strong model, we hope to focus on the process. Remember, getting there is half the fun. :)

We hope this project is an accessible way for interested developers to have access to a strong Go model with an easy-to-understand platform of python code available for extension, adaptation, etc.

If you'd like to read about our experiences training models, see RESULTS.md.

To see our guidelines for contributing, see CONTRIBUTING.md.

Getting Started

This project assumes you have the following:

The Hitchhiker's guide to python has a good intro to python development and virtualenv usage. The instructions after this point haven't been tested in environments that are not using virtualenv.

pip3 install virtualenv
pip3 install virtualenvwrapper

Install Bazel

BAZEL_VERSION=0.24.1
wget https://github.com/bazelbuild/bazel/releases/download/${BAZEL_VERSION}/bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh
chmod 755 bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh
sudo ./bazel-${BAZEL_VERSION}-installer-linux-x86_64.sh

Install TensorFlow

First set up and enter your virtualenv and then the shared requirements:

pip3 install -r requirements.txt

Then, you'll need to choose to install the GPU or CPU tensorflow requirements:

  • GPU: pip3 install "tensorflow-gpu==1.15.0".
    • Note: You must install CUDA 10.0. for Tensorflow 1.13.0+.
  • CPU: pip3 install "tensorflow==1.15.0".

Setting up the Environment

You may want to use a cloud project for resources. If so set:

PROJECT=foo-project

Then, running

source cluster/common.sh

will set up other environment variables defaults.

Running unit tests

./test.sh

To run individual modules

BOARD_SIZE=9 python3 tests/run_tests.py test_go
BOARD_SIZE=19 python3 tests/run_tests.py test_mcts

Automated Tests

Test Dashboard

To automatically test PRs, Minigo uses Prow, which is a test framework created by the Kubernetes team for testing changes in a hermetic environment. We use prow for running unit tests, linting our code, and launching our test Minigo Kubernetes clusters.

You can see the status of our automated tests by looking at the Prow and Testgrid UIs:

Basics

All commands are compatible with either Google Cloud Storage as a remote file system, or your local file system. The examples here use GCS, but local file paths will work just as well.

To use GCS, set the BUCKET_NAME variable and authenticate via gcloud login. Otherwise, all commands fetching files from GCS will hang.

For instance, this would set a bucket, authenticate, and then look for the most recent model.

# When you first start we recommend using our minigo-pub bucket.
# Later you can setup your own bucket and store data there.
export BUCKET_NAME=minigo-pub/v9-19x19
gcloud auth application-default login
gsutil ls gs://$BUCKET_NAME/models | tail -4

Which might look like:

gs://$BUCKET_NAME/models/000737-fury.data-00000-of-00001
gs://$BUCKET_NAME/models/000737-fury.index
gs://$BUCKET_NAME/models/000737-fury.meta
gs://$BUCKET_NAME/models/000737-fury.pb

These four files comprise the model. Commands that take a model as an argument usually need the path to the model basename, e.g. gs://$BUCKET_NAME/models/000737-fury

You'll need to copy them to your local disk. This fragment copies the files associated with $MODEL_NAME to the directory specified by MINIGO_MODELS:

MODEL_NAME=000737-fury
MINIGO_MODELS=$HOME/minigo-models
mkdir -p $MINIGO_MODELS/models
gsutil ls gs://$BUCKET_NAME/models/$MODEL_NAME.* | \
       gsutil cp -I $MINIGO_MODELS/models

Selfplay

To watch Minigo play a game, you need to specify a model. Here's an example to play using the latest model in your bucket

python3 selfplay.py \
  --verbose=2 \
  --num_readouts=400 \
  --load_file=$MINIGO_MODELS/models/$MODEL_NAME

where READOUTS is how many searches to make per move. Timing information and statistics will be printed at each move. Setting verbosity to 3 or higher will print a board at each move.

Playing Against Minigo

Minigo uses the GTP Protocol, and you can use any gtp-compliant program with it.

# Latest model should look like: /path/to/models/000123-something
LATEST_MODEL=$(ls -d $MINIGO_MODELS/* | tail -1 | cut -f 1 -d '.')
python3 gtp.py --load_file=$LATEST_MODEL --num_readouts=$READOUTS --verbose=3

After some loading messages, it will display GTP engine ready, at which point it can receive commands. GTP cheatsheet:

genmove [color]             # Asks the engine to generate a move for a side
play [color] [coordinate]   # Tells the engine that a move should be played for `color` at `coordinate`
showboard                   # Asks the engine to print the board.

One way to play via GTP is to use gogui-display (which implements a UI that speaks GTP.) You can download the gogui set of tools at http://gogui.sourceforge.net/. See also documentation on interesting ways to use GTP.

gogui-twogtp -black 'python3 gtp.py --load_file=$LATEST_MODEL' -white 'gogui-display' -size 19 -komi 7.5 -verbose -auto

Another way to play via GTP is to watch it play against GnuGo, while spectating the games:

BLACK="gnugo --mode gtp"
WHITE="python3 gtp.py --load_file=$LATEST_MODEL"
TWOGTP="gogui-twogtp -black \"$BLACK\" -white \"$WHITE\" -games 10 \
  -size 19 -alternate -sgffile gnugo"
gogui -size 19 -program "$TWOGTP" -computer-both -auto

Training Minigo

Overview

The following sequence of commands will allow you to do one iteration of reinforcement learning on 9x9. These are the basic commands used to produce the models and games referenced above.

The commands are

  • bootstrap: initializes a random model
  • selfplay: plays games with the latest model, producing data used for training
  • train: trains a new model with the selfplay results from the most recent N generations.

Training works via tf.Estimator; a working directory manages checkpoints and training logs, and the latest checkpoint is periodically exported to GCS, where it gets picked up by selfplay workers.

Configuration for things like "where do debug SGFs get written", "where does training data get written", "where do the latest models get published" are managed by the helper scripts in the rl_loop directory. Those helper scripts execute the same commands as demonstrated below. Configuration for things like "what size network is being used?" or "how many readouts during selfplay" can be passed in as flags. The mask_flags.py utility helps ensure all parts of the pipeline are using the same network configuration.

All local paths in the examples can be replaced with gs:// GCS paths, and the Kubernetes-orchestrated version of the reinforcement learning loop uses GCS.

Bootstrap

This command initializes your working directory for the trainer and a random model. This random model is also exported to --model-save-path so that selfplay can immediately start playing with this random model.

If these directories don't exist, bootstrap will create them for you.

export MODEL_NAME=000000-bootstrap
python3 bootstrap.py \
  --work_dir=estimator_working_dir \
  --export_path=outputs/models/$MODEL_NAME

Self-play

This command starts self-playing, outputting its raw game data as tf.Examples as well as in SGF form in the directories.

python3 selfplay.py \
  --load_file=outputs/models/$MODEL_NAME \
  --num_readouts 10 \
  --verbose 3 \
  --selfplay_dir=outputs/data/selfplay \
  --holdout_dir=outputs/data/holdout \
  --sgf_dir=outputs/sgf

Training

This command takes a directory of tf.Example files from selfplay and trains a new model, starting from the latest model weights in the estimator_working_dir parameter.

Run the training job:

python3 train.py \
  outputs/data/selfplay/* \
  --work_dir=estimator_working_dir \
  --export_path=outputs/models/000001-first_generation

At the end of training, the latest checkpoint will be exported to. Additionally, you can follow along with the training progress with TensorBoard. If you point TensorBoard at the estimator working directory, it will find the training log files and display them.

tensorboard --logdir=estimator_working_dir

Validation

It can be useful to set aside some games to use as a 'validation set' for tracking the model overfitting. One way to do this is with the validate command.

Validating on holdout data

By default, Minigo will hold out 5% of selfplay games for validation. This can be changed by adjusting the holdout_pct flag on the selfplay command.

With this setup, rl_loop/train_and_validate.py will validate on the same window of games that were used to train, writing TensorBoard logs to the estimator working directory.

Validating on a different set of data

This might be useful if you have some known set of 'good data' to test your network against, e.g., a set of pro games. Assuming you've got a set of .sgfs with the proper komi & boardsizes, you'll want to preprocess them into the .tfrecord files, by running something similar to

import preprocessing
filenames = [generate a list of filenames here]
for f in filenames:
    try:
        preprocessing.make_dataset_from_sgf(f, f.replace(".sgf", ".tfrecord.zz"))
    except:
        print(f)

Once you've collected all the files in a directory, producing validation is as easy as

python3 validate.py \
  validation_files/ \
  --work_dir=estimator_working_dir \
  --validation_name=pro_dataset

The validate.py will glob all the .tfrecord.zz files under the directories given as positional arguments and compute the validation error for the positions from those files.

Retraining a model

The training data for most of Minigo's models up to v13 is publicly available in the minigo-pub Cloud storage bucket, e.g.:

gsutil ls gs://minigo-pub/v13-19x19/data/golden_chunks/

For models v14 and onwards, we started using Cloud BigTable and are still working on making that data public.

Here's how to retrain your own model from this source data using a Cloud TPU:

# I wrote these notes using our existing TPU-enabled project, so they're missing
# a few preliminary steps, like setting up a Cloud account, creating a project,
# etc. New users will also need to enable Cloud TPU on their project using the
# TPUs panel.

###############################################################################

# Note that you will be billed for any storage you use and also while you have
# VMs running. Remember to shut down your VMs when you're not using them!

# To use a Cloud TPU on GCE, you need to create a special TPU-enabled VM using
# the `ctpu` tool. First, set up some environment variables:
#   GCE_PROJECT=<your project name>
#   GCE_VM_NAME=<your VM's name>
#   GCE_ZONE<the zone in which you want to bring uo your VM, e.g. us-central1-f>

# In this example, we will use the following values:
GCE_PROJECT=example-project
GCE_VM_NAME=minigo-etpu-test
GCE_ZONE=us-central1-f

# Create the Cloud TPU enabled VM.
ctpu up \
  --project="${GCE_PROJECT}" \
  --zone="${GCE_ZONE}" \
  --name="${GCE_VM_NAME}" \
  --tf-version=1.13

# This will take a few minutes and you should see output similar to the
# following:
#   ctpu will use the following configuration values:
#         Name:                 minigo-etpu-test
#         Zone:                 us-central1-f
#         GCP Project:          example-project
#         TensorFlow Version:   1.13
#  OK to create your Cloud TPU resources with the above configuration? [Yn]: y
#  2019/04/09 10:50:04 Creating GCE VM minigo-etpu-test (this may take a minute)...
#  2019/04/09 10:50:04 Creating TPU minigo-etpu-test (this may take a few minutes)...
#  2019/04/09 10:50:11 GCE operation still running...
#  2019/04/09 10:50:12 TPU operation still running...

# Once the Cloud TPU is created, `ctpu` will have SSHed you into the machine.

# Remember to set the same environment variables on your VM.
GCE_PROJECT=example-project
GCE_VM_NAME=minigo-etpu-test
GCE_ZONE=us-central1-f

# Clone the Minigo Github repository:
git clone https://github.com/tensorflow/minigo
cd minigo

# Install virtualenv.
pip3 install virtualenv virtualenvwrapper

# Create a virtual environment
virtualenv -p /usr/bin/python3 --system-site-packages "${HOME}/.venvs/minigo"

# Activate the virtual environment.
source "${HOME}/.venvs/minigo/bin/activate"

# Install Minigo dependencies (TensorFlow for Cloud TPU is already installed as
# part of the VM image).
pip install -r requirements.txt

# When training on a Cloud TPU, the training work directory must be on Google Cloud Storage.
# You'll need to choose your own globally unique bucket name.
# The bucket location should be close to your VM.
GCS_BUCKET_NAME=minigo_test_bucket
GCE_BUCKET_LOCATION=us-central1
gsutil mb -p "${GCE_PROJECT}" -l "${GCE_BUCKET_LOCATION}" "gs://${GCS_BUCKET_NAME}"

# Run the training script and note the location of the training work_dir
# it reports, e.g.
#    Writing to gs://minigo_test_bucket/train/2019-04-25-18
./oneoffs/train.sh "${GCS_BUCKET_NAME}"

# Launch tensorboard, pointing it at the work_dir reported by the train.sh script.
tensorboard --logdir=gs://minigo_test_bucket/train/2019-04-25-18

# After a few minutes, TensorBoard should start updating.
# Interesting graphs to look at are value_cost_normalized, policy_cost and policy_entropy.

Running Minigo on a Kubernetes Cluster

See more at cluster/README.md

You can’t perform that action at this time.