自动玩微信小游戏跳一跳

Requirements

• Python
• Opencv
• tensorflow (if using nn_play.py)

for Android

• Adb tools
• Android Phone

for IOS (Refer to this site for installation)

• iPhone
• Mac
• WebDriverAgent
• facebook-wda
• imobiledevice

Algorithms for Localization

• multiscale-search
• CV based fast-search
• Convolutional Neural Network based coarse-to-fine model

Notice: CV based fast-search only support Android for now

Run

It is recommended to run the following if have an android phone

python play.py --phone Android --sensitivity 2.045

If you have an iPhone, download the model following the link below, and run the following

python nn_play.py --phone IOS --sensitivity 2.045

• --phone has two options: Android or IOS.
• --sensitivity is the constant parameter that controls the pressing time.
• play.py using algorithm based on CV, support Android and IOS
• nn_play.py using algorithm based on Convolutional Neural Network, support Android and IOS, recommend for IOS

Performance

Our method can correctly detect the positions of the man (green dot) and the destination (red dot).

It is easy to reach the state of art as long as you like. But I choose to go die after 859 jumps for about 1.5 hours.

Demo Video

Here is a video demo. Excited!

Train Log & Data

CNN train log and train&validation data avaliable at

• Baidu Drive
• Google Drive

Training: download and untar data into any directory, and then modify self.data_dir in those files under cnn_coarse_to_fine/data_provider directory.

Inference: download and unzip train log dirs into resource directory.

Algorithm Details

For algorithm details, please go to https://zhuanlan.zhihu.com/p/32636329.