update

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+.idea

README.md

@@ -1 +1,78 @@
-Welcome.
+# TLeague Project Page
+This is the project page for the following technical reports:
+
+Lei Han∗, Jiechao Xiong∗, Peng Sun∗, Xinghai Sun, Meng Fang, Qingwei Guo, Qiaobo Chen, Tengfei Shi, Zhengyou Zhang.
+TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game.
+[arXiv preprint arXiv:2011.13729](https://arxiv.org/abs/2011.13729), 2020.
+(* Equal contribution, correspondence to the first three authors)
+
+Peng Sun∗, Jiechao Xiong∗, Lei Han∗, Xinghai Sun, Shuxing Li, Jiawei Xu, Meng Fang, Zhengyou Zhang.
+TLeague: A Framework for Competitive Self-Play based Distributed Multi-Agent Reinforcement Learning.
+[arXiv preprint arXiv:2011.12895](https://arxiv.org/abs/2011.12895), 2020.
+(* Equal contribution, correspondence to the first three authors)
+
+<big>**Impatient reader for the StarCraft II AI TStarBot-X could see the [TStarBot-X project page here](tstarbotx/README.md)**<big>.
+
+## Quick Start
+* For general TLeague usage, see [the section below](#usage)
+* For the resources of TStarBotX, see [the page here](tstarbotx/README.md)
+* For the resources of ViZDoom experiments, see [the pager here](vizdoom/README.md)
+* For the resources of Pommerman experiments, see [the page here](pommerman/README.md)
+
+## Usage
+### Minimal Working Example
+To use the TLeague framework and run a minimal training,
+one needs to install the following basic packages:
+* [TLeague](https://github.com/tencent-ailab/TLeague): the main logic of Competitive SelfPlay MultiAgent Reinforcement Learning.
+* [TPolicies](https://github.com/tencent-ailab/TPolicies): a lib for building Neural Net used in RL and IL.
+* [Arena](https://github.com/tencent-ailab/Arena): a lib of environments and env-agent interfaces.
+See the docs therein for how to install.
+
+Then, try the example of training with the simple game `pong-2p` as a sanity check. 
+See the [link here](https://github.com/tencent-ailab/TLeague/blob/dev-open/docs/EXAMPLE_SM.md#pong-2p).
+
+To run training for other environments, extra binaries and/or packages must be installed, as explained in the following.
+
+### StarCraft II Training
+When installing the `Arena` package, 
+one needs additionally install [TImitate](https://github.com/tencent-ailab/TImitate),
+which is a lib for SC2 observation and action, zstat extraction, replay parsing, etc.
+See also the [link here](https://github.com/tencent-ailab/Arena#dependencies).
+
+[Here are examples](https://github.com/tencent-ailab/TLeague/blob/dev-open/docs/EXAMPLE_SM.md#starcraft-ii) 
+for both Reinforcement Learning (CSP-MARL) and Imitation Learning in a single machine.
+
+TODO: pointer to the Docker Auto Build repo and say it's yet-another guide to installation from scratch.
+
+TODO: texts for how to train with k8s
+
+### ViZDoom Training
+When installing the `Arena` package, 
+one needs additionally install ViZDoom (>=1.1.8), 
+see the [link here](https://github.com/tencent-ailab/Arena#dependencies).
+
+[Here are examples](https://github.com/tencent-ailab/TLeague/blob/dev-open/docs/EXAMPLE_SM.md#vizdoom)
+of how to train ViZDoom in a single machine.
+
+TODO: pointer to the Docker Auto Build repo and say it's yet-another guide to installation from scratch.
+
+TODO: texts for how to train with k8s
+
+### Pommerman Training
+When installing the `Arena` package, 
+one needs additionally install Pommerman, 
+see the [link here](https://github.com/tencent-ailab/Arena#dependencies).
+
+[Here are examples](https://github.com/tencent-ailab/TLeague/blob/dev-open/docs/EXAMPLE_SM.md#pommerman)
+for how to train Pommerman in a single machine. 
+
+TODO: pointer to the Docker Auto Build repo and say it's yet-another guide to installation from scratch.
+
+TODO: texts for how to train with k8s
+
+# Disclaimer
+This is not an officially supported Tencent product.
+The code and data in this repository are for research purpose only. 
+No representation or warranty whatsoever, expressed or implied, is made as to its accuracy, reliability or completeness. 
+We assume no liability and are not responsible for any misuse or damage caused by the code and data. 
+Your use of the code and data are subject to applicable laws and your use of them is at your own risk.

pommerman/README.md

@@ -0,0 +1,12 @@
+# Pommerman Experiments
+This page contains the resources for the experiments of Pommerman as discussed in the TLeague technical report.
+
+TODO: link to the trained models
+
+## Downloads
+### Replay Files
+Here are the 100 replay files of our agent against Navocado as discussed in the TLeague technical report:
+[Google Drive](https://drive.google.com/file/d/1miuqo7EpzgNIGHUNtPqdIswe8rKuoRk0/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/csEpj1R3).
+
+Replays can be displayed by drag-and-drop on the page [Pommerman Playback](https://www.pommerman.com/battle).

tstarbotx/README.md

@@ -0,0 +1,32 @@
+# TStarBot-X Project Page
+This is the project page for the StarCraft II AI TStarBot-X, 
+discussed in the following technical report:
+
+Lei Han∗, Jiechao Xiong∗, Peng Sun∗, Xinghai Sun, Meng Fang, Qingwei Guo, Qiaobo Chen, Tengfei Shi, Zhengyou Zhang.
+TStarBot-X: An Open-Sourced and Comprehensive Study for Efficient League Training in StarCraft II Full Game.
+[arXiv preprint arXiv:2011.13729](https://arxiv.org/abs/2011.13729), 2020.
+(* Equal contribution, correspondence to the first three authors)
+
+
+## Quick Start
+* If you are an SC2 player who wants to experience playing against TStarBot-X,
+see our guidance of [human-machine test](hm_test.md). 
+* If you are a researcher/developer interested in the training, see todo.
+* For downloading the resources (Replays, NN models, zstat ...), see the [section below](#downloads).
+
+## Downloads
+### TStarBot-X Replay Files
+Here are the replay files of the human-machine tests as discussed in the TStarBot-X technical report:
+[Google Drive](https://drive.google.com/file/d/1wIOpB_aPrzCCJ9uZbQJmiWSbDFODl4B4/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/Ju44Zkks)
+
+### Maps
+When running the human-machine test or opening the replays,
+you need the map file for `KairosJunction`.
+See [here](https://github.com/deepmind/pysc2#get-the-maps) and [here](https://github.com/Blizzard/s2client-proto#map-packs) for how to download and where to place the maps.
+For your convenience, we provide the `KairosJunction` file here 
+[Google Drive](https://drive.google.com/file/d/1O_L4E91b3sAUunrDxGV-a_uj7bTuwQ8H/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/de2LCco8)
+
+### Data
+For NN models, zstat files, etc., please see the [link here](hm_test.md#downloads).

tstarbotx/hm_test.md

@@ -0,0 +1,138 @@
+# Human-Machine Test
+We provide guidelines for how a human plays against TStarBot-X.
+Please follow the instructions of this page which is self contained (You don't need care the training code when running the human-machine test).
+**Note, currently it only plays zerg-vs-zerg on the map "KairosJunction". SC2 version 4.10.0 is required.**
+
+## Prerequisites and Terminology
+You need TWO machines for the human-machine test. 
+
+**Machine A**: the machine used by a human player. 
+Windows or MacOS.
+Have the SC2 game installed.
+Version 4.10.0 game core is required,
+which can be ensured by opening an arbitrary 4.10.0 replay file (for example, [here](./README.md#tstarbot-x-replay-files)) that the auto-downloading will be triggered when necessary. 
+We've tested for MacOS where the SC2 game is downloaded from https://starcraft2.com/.
+
+**Machine B**: the machine where TStarBot-X deploys.
+Ubuntu is required. 
+Using a GPU is recommended (otherwise the NN forward-pass can be slow, causing high delay and degraded performance).
+TStarBot-X has been tested on a laptop with GTX 1650 and ubuntu 18.04.
+
+## Installs
+Here are the step-by-step instructions.
+
+* Make sure that machine A can connect to machine B via passwordless ssh.
+  - One can google for how to do the passwordless ssh setup, e.g., 
+  [here](https://www.tecmint.com/ssh-passwordless-login-using-ssh-keygen-in-5-easy-steps/).
+  In summary, one generates the public and private keys on machine A, 
+  then copy (e.g, using `scp` for remote copying) the private key to the folder `~/.ssh` under machine B.
+  Make sure the `sshd` service has started on machine B.
+  - Verify the passwordless setup by connecting to B from A and see whether it succeeds.
+* On machine B, install the two packages `DistLadder3`, `pysc2 (Tencent Extension)`.
+See the [section below](#downloads) for downloading.
+Just cd to the corresponding folder and typing `pip install -e .` to complete the installation.
+The Linux SC2 binary version 4.10.0 is required,
+see the link [here](https://github.com/deepmind/pysc2#linux) and [here](https://github.com/Blizzard/s2client-proto#downloads).
+Then install the main package `SC2AgentsZoo2` (see the [section below](#downloads) for downloading) for TStarBot-X as follows:
+  - To let TStarBot-X use GPU, you need install `cudnn` and `cuda`
+  - Have `virtualenv` installed. You can do `pip install virtualenv`
+  - Have ssh server started. You can do `apt-get install openssh-server`
+  - cd to the folder `SC2AgentsZoo2/agent_TLeagueFormal14`, 
+  run the command `bash install_virtualenv3.sh` to complete the installation.
+* On machine A, install `DistLadder3`, `pysc2 (Tencent Extension)` as aforementioned.
+Install also ther commercial SC2 on this machine.
+
+## Usage
+Here is how to start the human-machine test.
+
+On machine A, run the command:
+```
+python3 -m distladder3.bin.play_vs_remote_agent \
+--human \
+--port 6789 \
+--remote <machine-B-usr-name@machine-B-ip-address> \
+--replay_dir /Users/your-name/Desktop \
+--map KairosJunction \
+--game_version 4.10.0 \
+--replay_name xxx-A-machine.SC2Replay
+```
+which starts a game UI that a human can play SC2 with mouse and keyboard.
+For the `--remote` arg, 
+if machine A and B happen to have the same user name, 
+you can also omit the user name and simply write `--remote <machine-B-ip-address>`.
+Examples: `--remote sc2tester@192.168.0.5` or `--remote 192.168.0.5`.
+
+On machine B, run the command:
+```
+python3 -m distladder3.bin.play_vs_remote_agent \
+--port 6789 \
+--replay_dir /Users/your-name/Desktop \
+--map KairosJunction \
+--game_version 4.10.0 \
+--replay_name xxx-B-machine.SC2Replay \
+--player_name_path_config "TStarBot-X,/your/path/to/SC2AgentsZoo2/agent_TLeagueFormal14/,/your/path/to/the/config/file.ini"
+```
+to start the AI,
+where the arg `--player_name_path_config` determines an agent by its name (TStarBotX), 
+path (`/your/path/to/SC2AgentsZoo2/agent_TLeagueFormal14/`), 
+and config (`/your/path/to/SC2AgentsZoo2/agent_TLeagueFormal14/`) in comma separated value.
+
+The `*.ini` config file specifies more detailed args for the agent, 
+e.g., 
+whether to use GPU, 
+the NN model path, 
+the zstat path (a folder), 
+the zstat category, 
+probability for zeroing the zstat,
+etc., 
+as shown in the snippet below:
+```ini
+[config]
+use_gpu_id=-1
+;-1 for not using GPU, 0 means using GPU #0, 1 for GPU #1, etc.
+...
+chkpoints_root_dir=/Users/usr-name/tstarbotx/data
+model_filename=TStarBot-X-33days.model
+...
+zstat_zeroing_prob=0.0
+zstat_category=Normal174
+...
+tleague_interface_config.zstat_data_src=/Users/usr-name/tstarbotx/data/rp2124-mv8-victory-selected-Misc
+...
+```
+
+We've prepared several agent config `.ini` files, 
+as well as the corresponding NN models and zstat files (in a folder),
+see the [section below](#downloads).  
+
+## Downloads
+### Packages
+* `DistLadder3` 
+[Google Drive](https://drive.google.com/file/d/1ufCtU2JIyoSiSMwN4lqT66oxitmZeArh/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/QFgOzG4n)
+* `pysc2 (Tencent Extension)` 
+[Google Drive](https://drive.google.com/file/d/1rJnmK1aNIFaYuYkXXmkDvTe-JRKrzFhe/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/mCCEZtOX)
+* `SC2AgentsZoo2` 
+[Google Drive](https://drive.google.com/file/d/1neXug1fn3miHnKu9Z8tBpMC-ZKfIzXAP/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/NKiLym42)
+
+### Data Files
+* zstat files (`rp2124-mv8-victory-selected-Misc`, a folder) used by the 8/25/33 days model
+[Google Drive](https://drive.google.com/file/d/1pV8wD_AXbbESQL2L4LticKTTwiaQpLCf/view?usp=sharing) 
+or [Tencent Weiyun](https://share.weiyun.com/ZXeYGjZp)
+* Main Agent 8 days: 
+  - `ini` config [Google Drive](https://drive.google.com/file/d/1Ed80rcYaafVRGlQJ7hCcsge1snCx8oLQ/view?usp=sharing) 
+  or [Tencent Weiyun](https://share.weiyun.com/GJ1Bwfie)
+  - NN model [Google Drive](https://drive.google.com/file/d/1mJ9s3dpScgKbYj3vZusnC0fJ1IPQuMrc/view?usp=sharing) 
+  or [Tencent Weiyun](https://share.weiyun.com/spHoQIg3)
+* Main Agent 25 days:
+  - `ini` config [Google Drive](https://drive.google.com/file/d/1JrfERGRQrVaVPOU8AFjn_B9jhy5eeMl1/view?usp=sharing) 
+  or [Tencent Weiyun](https://share.weiyun.com/bNGLU4Zj)
+  - NN model [Google Drive](https://drive.google.com/file/d/1BcQERcIGZvulCd5M4gCej80lJmdIYcLh/view?usp=sharing) 
+  or [Tencent Weiyun](https://share.weiyun.com/24QfxkMZ)
+* Main Agent 33 days: 
+  - `ini` config [Google Drive](https://drive.google.com/file/d/1AohBDH4C4Y86usNbEDrVhq1g2ZUEJfvp/view?usp=sharing) 
+  or [Tencent Weiyun](https://share.weiyun.com/ue3zQ7RG)
+  - NN model [Google Drive](https://drive.google.com/file/d/1M6m-vGGGYNI-KHETq8t8gBKi_luuPLyD/view?usp=sharing) 
+  or [Tencent Weiyun](https://share.weiyun.com/yuh9ZDSe)

vizdoom/README.md

@@ -0,0 +1,7 @@
+# ViZDoom Experiments
+TODO: link to trained models
+
+TODO: code for evaluation (our minor modification over the https://github.com/mihahauke/VDAIC2017), 
+allowing synchronous mode for the host, F1 and "MyPlayer".
+
+TODO: link to the video clips for the evaluation