A 4‑motor robot is controlled by real‑time hand gestures using Python computer vision and an Arduino‑based motion controller.
The system uses MediaPipe and OpenCV to detect the number of raised fingers (0–5) from a webcam feed and sends this count over serial to an Arduino Uno.
The Arduino drives four 28BYJ‑48 stepper motors (with ULN2003 drivers) to move the robot forward, backward, left, right, or stop based on a simple finger‑count mapping (e.g., 1 = forward, 2 = right, 3 = left, 4 = reverse, 0 = stop).
- Vision (Python): MediaPipe Hands for landmark tracking, custom finger‑counting algorithm, PySerial for communication.
- Robot (Arduino): Stepper library, motor control logic, serial command parsing, gesture→motion switch‑case mapping.
pip install opencv-python mediapipe pyserialRun the gesture script to:
- Open webcam
- Detect hand landmarks
- Count raised fingers
- Send the finger count to Arduino over serial
- Board: Arduino Uno
- Motors: 4 × 28BYJ‑48 + ULN2003 drivers
- Upload the provided Arduino sketch, which:
- Reads an integer 0–5 from
Serial - Executes the corresponding movement pattern for the four steppers
- Reads an integer 0–5 from
| Fingers | Action |
|---|---|
| 0 | Stop |
| 1 | Forward |
| 2 | Turn right |
| 3 | Turn left |
| 4 | Reverse |
- ~90% average gesture recognition accuracy in controlled lighting.
- End‑to‑end latency ≈ 160 ms (vision + serial + motor response), sufficient for smooth teleoperation.