Online Environment for GUI Reflection

This is the online environment used in GUI Reflection. The environment is designed in a host-worker framework, where the model inference is only running on the host machine while the Android Emulators are only on the worker machines.

Installation

1. Install the gui_env from requirements.txt.

2. Add the path to this repo to PYTHONPATH.

3. On the Host machine, make sure internvl is installed as specified in GUI_Reflection

4. On the worker machines, set up the Android Emulator according to env_setup.md

5. Make sure the code on the host and worker machines is synchronized, and add the SSH key of the host machine to all worker machines.

Collection Mode

To collect data, correctly set up the config files in scripts/config/collect and run

python scripts/run.py --config-path config/collect --config-name host

Evaluation Mode

To evaluate models, correctly set up the config files in scripts/config/evaluate and run

python scripts/run.py --config-path config/evaluate --config-name host

Acknowledgement

This code is built upon DigiRL, DistRL, and AndroidWorld.