Research project on attentive UI for Android devices.
by Li Sirui, Department of Computer Science, The University of Hong Kong.
The system is the implementation of Attentive Gestural User Interfaces, which make use of eye input data from eye trackers to augment touch-screen interfaces.
Supermonkey, trackerd are core components of the system. The folder scripts includes tool scripts of the system. MonkeyDemo and FactFinder are applications running on the system. HoverLib is an interface library used by the applications.
The system is developed and tested using Tobii X1 Light Eye Tracker and ASUS Nexus 7 (2012) Android tablet.
Daemon for eye trackers. It works as a service that provides network interfaces for controlling the eye tracker as well as the ability to send eye-tracking data over the network.
- Python 2.6
- Tobii SDK 3.0 RC1 (now known as Tobii Analytics SDK) for Win32
- A Tobii eye tracker that is supported by the above SDK
Listens to port 10800, connects to Monkey via port 1080.
A client for trackerd, which controls the eye tracker on the host PC from Android device. It is a regular Android app that connects to trackerd via TCP sockets and provides an easy-to-use graphical interface for users.
A modified version of UI/Application Exerciser Monkey by Android is bundled with SuperMonkey.
- Rooted Android 4.2
- Android SDK 14+
- Android SDK Platform with internal classes (please refer to this)
- Install SuperMonkey just as a normal app.
- Put file
/data/on Android file system
- Correct its permissions.
scripts/install_monkey.bat for details.
- Connect Android device with USB Debugging.
- Set up port forwarding with ADB:
adb forward tcp:1080 tcp:1080
- Run monkey from Android shell:
sh /data/supermonkey.sh --port 1080 --ignore-crashes
Then you can start trackerd and SuperMonkey app. Please see
scripts/monkey.bat for details.
Monkey listens to port 1080 on the Android system. Supermonkey connects to trackerd via port 10800.
HoverLib, MonkeyDemo, and FactFinder
HoverLib: Library project of Android interface components that supports attentive input.
MonkeyDemo: Demo application for the above interface components.
FactFinder: Renamed as "FoodFinder" in its interface, sample application that makes use of the components.
Note: The interfaces are optimised for 7-inch screen with 1280×800-pixels resolution in landscape mode. Slight changes to layout might be needed on other configurations.
- Android 4.2
- Android SDK 16+
- Android support library v4 and v13
- ActionBarSherlock 4.4.0 with a tiny bit of modification
You can find the modified ActionBarSherlock here.
For more detailed and in-depth information regarding this project, please refer to my thesis Attentive Gestural Interface on Touch Screens (download from The HKU Scholars Hub).
Last updated: 1 Dec 2013.
Copyright 2013 Sirui Li Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.