Hand Gesture-based Food Kiosk Mobile Application

Prerequisites

• Android Studio (for Android) (2021.2.1 Patch 1 & 2021.3.1 Patch 1)
• Android SDK (API 28 or above) - use SDK Platforms in Android Studio
• Android SDK Command-line Tools - use SDK Tools in Android Studio
• Flutter SDK (3.3.8) - make sure no issues found on flutter doctor
• Xcode (for iOS) untested

Instruction for Android

1. Clone the project in Android Studio
2. Start an Emulator (SDK 30 and above preferred)
3. Navigate to pubspec.yaml and run Pub get command
4. Wait until the dependancies are installed
5. Navigate to External Libraries/Flutter Plugins/tflite_flutter_helper-0.3.1/android/src.main/kotlin.com.tfliteflutter.tflite_flutter_helper/TfliteFlutterHelperPlugin.kt or copy paste the modified file contents from here - skip to Step 7 if you copy-pasted
6. Go to line 143 and remove all the question marks ? from the onRequestPermissionsResult function parameters
7. Allow to modify the project files if it asks
8. Run the project and accept permissions it asks

Additional Information

• If you want to enable your webcam for the emulator: Android: How to use webcam in emulator?
• In some cases, the camera preview will show as rotated. Try to move your hand around if that's the case because I did not perform image augmentation to consider cases from different angles
• To try out different models available, navigate to the project folder: assets/tflite/ and rename any model to model.tflite to replace the current model. Default is EfficientNet-Lite0
• The gesture detection rate is set to every 3 seconds; to note: only the Lite[0-1] models infer in less than 3 seconds. You may change the timer from DETECTION_CHECK_TIMER inside main.dart

Demo

Zahiriddin Rustamov