AI plays WeChat Jump Game
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
cnn_coarse_to_fine minor Jan 10, 2018
resource minor Jan 2, 2018
.gitignore minor Jan 9, 2018
LICENSE Initial commit Dec 30, 2017
model.py add coarse2fine nn model Jan 4, 2018
model_fine.py modify readme and nn_play Jan 4, 2018
nn_play.py no screenshot error msg Jan 25, 2018
play.py no screenshot error msg Jan 25, 2018
readme.md Revert "Update readme.md" Jan 30, 2018
readme_cn.md Revert Jan 30, 2018

readme.md

自动玩微信小游戏跳一跳

中文说明请点这里

Requirements

  • Python
  • Opencv3
  • Tensorflow

for Android

  • Adb tools
  • Android Phone

for IOS (Refer to this site for installation)

  • iPhone
  • Mac
  • WebDriverAgent
  • facebook-wda
  • imobiledevice

Algorithms for Localization

  • Multiscale search
  • Fast search
  • CNN-based coarse-to-fine model

For algorithm details, please go to https://zhuanlan.zhihu.com/p/32636329.

Notice: CV based fast-search only support Android for now

Run

Before running our code, connect to your phone via USB.

If Android phone, open the USB debugging at developer options enter adb devices to ensure that the list is not empty. If iPhone, please ensure that you have a mac. Then following this link for preparation.

It is recommended to download the pre-trained model following the link below and run the following code

python nn_play.py --phone Android --sensitivity 2.045

You can also try play.py by running the following code

python play.py --phone Android --sensitivity 2.045
  • --phone has two options: Android or IOS.
  • --sensitivity is the constant parameter that controls the pressing time.
  • nn_play.py uses CNN-based coarse-to-fine model, supporting Android and IOS (more robust)
  • play.py uses multiscale search and fast search algorithms, supporting Android and IOS (it may fail sometimes in other phones)

Performance

Our method can correctly detect the positions of the man (green dot) and the destination (red dot).

It is easy to reach the state of art as long as you like. But I choose to go die after 859 jumps for about 1.5 hours.

state_859 state_859 sota

Demo Video

Here is a video demo. Excited!

微信跳一跳

Train Log & Data

CNN train log and train&validation data avaliable at

Training: download and untar data into any directory, and then modify self.data_dir in those files under cnn_coarse_to_fine/data_provider directory.

Inference: download and unzip train log dirs(train_logs_coarse and train_logs_fine) into resource directory.

How to Train CNN models by yourself?

  1. Download and untar data into any directory, and then modify self.data_dir in those files under cnn_coarse_to_fine/data_provider directory.
  2. base.large is model dir for coarse model, base.fine is model dir for fine model, other dirs under cnn_coarse_to_fine/config are models we don't use, but if you have interests, you can try train other models by yourself.
  3. Run python3 train.py -g 0 to train your model, -g to specify GPU to use, if you don't have GPU, training model is not recommended because training speed with CPU is very slow.
  4. After training, move or copy .ckpt file to train log dirs(train_logs_coarse and train_logs_fine) for use.