Support Semi-automated UI testing in DroidBot

[yuanchun-li]

We want to extend DroidBot to support semi-automated testing. That is, DroidBot learns from the human about how to interact with apps. For example, a user interacts with an app for the first time (eg. swiping windows, drawing PIN code, etc.), we record the user’s input and send to DroidBot, and based on the user’s input, DroidBot will be able to pass the difficult UI states.

The detailed steps may include:

1. Improve the app model that DroidBot used to test an app (see UtgDynamicFactory and AppModel classes in app_event.py);
2. Implement an Android app that can monitor UI (using UIAutomator, etc), record user interaction (using getevent, etc.), and construct the AppModel based on the UI and user interactions;
3. Extend DroidBot to read the AppModel as input and generate events based on the model.

[never2average]

Can I be assigned to work on this feature? @ylimit