Obtain the power of touchless interaction! Being able to effortlessly navigate and interact with various screen displays without any physical contact, is what this project brings you.
While touchless technology exists, it often remains out of reach for many due to its limited availability and high costs. Our mission with AR Touch is to bridge this gap and democratize touchless interaction for everyone. By implementing this cutting-edge technology only in a user-friendly Android app, we enable you to interact with any screen, regardless of whether it supports touch input or its dimensions.
- It is independent of the target operating system, that is, there is no need to install a special interface software (I already installed it for you!) and it has been tested on Windows and Android operating systems.
- All the processes are done in the device itself, so there is no need to spend extra money to buy an additional tool.
When we place our fingers on the touchscreen, occurred an interrupt to handle touch, which the operating system must handle.
Since we need to have the touch event first for the OS to handle it, we must first determine the touchable area. For this purpose, I have utilized ArUco Markers to identify the edges of the screen frame.
Additionally, I have used hand gestures to generate events, and for their respective computation, I have employed on-device ML-powered solutions available within the MediaPipe framework.
To deliver the touch event independently of the OS, I have utilized BLE technology and the HID protocol.
- A device which Android 9 was init version
- with mid-range processor e.g.
Snapdragon 730
- with mid-range processor e.g.
- Four ArUco markers, can be downloaded here
- Only one hand can be placed in the frame of the screen, so currently zooming is not available as usual.
- Currently, the touch event is generated in line with the content displayed on the phone screen.
- Kotlin
- C/CPP
- View System
- Jetpack Compose
- CameraX
- NDK and JNI
- OpenCV
- MediaPipe
- Coroutines
- Flow
- Bluetooth Low Energy
- Koin
Uses modularization approach and follows the official architecture guidance.
Testing has been performed in this project without using common frameworks for mocking, relying solely on dummy, stub, shadow, and fake test doubles.
Robolectric, a powerful simulator, has been utilized to take advantage of its benefits in executing UI and Integration tests.
To execute Isolated Instrumentation tests, the Orchestrator tool has been enabled in Gradle.
To enhance the components of the producer module, Performance tests have been considered using the Microbenchmark tool. Additionally, to prevent code leaks in production, a separate module has been designated for them.
You can see the list of tested classes here.
If you want to know about the process of evolution and other aspects of this project, you can read the project report. This project was for my undergraduate degree at the University of Tabriz.
I welcome and appreciate contributions from the community to improve the AR Touch project. If you have any ideas, suggestions, or feedback to enhance the project, feel free to share them in the Discussions section.
You can actively participate by:
- Opening
discussionsto propose new features or improvements. - Creating
new issueto report the bugs you encounter during testing. - Submitting
pull requeststo introduce code enhancements or optimizations.
Thank you for taking the time to read about the AR Touch project! If you find the project fascinating, your financial support would be truly heartwarming.
By clicking on the Sponsor button right now, you can become a valuable partner in the project's evolution. Every donation, no matter the amount, makes a significant impact and inspires us to push the boundaries of what's possible.
Join us in shaping the future of touchless interactions with your generous support. Let's make magic together! 🌟
AR Touch is distributed under the terms of the Apache License (Version 2.0). See the license for more information.