-
Notifications
You must be signed in to change notification settings - Fork 0
Partial sim with gesture control
(6 Feb 2024). Uses mediapipe AI to recognize hand gestures (open, closed right hand) to control the simulated copter flight. In this example holding up an open right hand (in front of usb camera) causes the sim copter to pause the mission (the CC could then take over control). A close right hand (fist) continues the mission.
In script I modified for this example the above screenshot does not show the tracing (actually it does, but not long enough for you to see it). Below are examples from the unmodified script that show the tracing.
For (draft) details see the first chapter in
- Gdrive
- Document B1-3a-1_pymavlink_mediapipe_vxx_24.xxxx.docx first chapter section "24.0206-1 guesture control (works)"
FYI: There are many mediapipe variations to use. These are also interesting.
Drawing with the finger (left) or (right) using full body gestures (this would best for controlling a copter outside, but these mediapipe examples (that I found) do not contain the recognition part, only the tracing)... maybe do this in the near future).
Note that gestures are not exactly a part of my eventual goal. But I thought it would be interesting to be able to control a copter with gestures. Note that mediapipe has a full suite of AI tools.
24.0907 Wiki, Gdrive, Youtube, ZiptieAI.com, ZiptieAI docs. The author is looking for a job!
- The race 24.0818
- Why (you need) ZiptieAI
- Why (I created) ZiptieAI
- Wiki organization (6 epics) (outdated)
- Blog (latest)
- Conferences 24.0801
(0) Reference
EPIC 1 - Build/fly FPV drone
(1) FPV simulators (inav notes 0608)
- 2.0 Technical overviews
- 2.1 MVP (min viable platform)
- 2.2 GPS
- 2.3 Video
- 2.4 Carbon frame
- 2.5 Missions
- 2.6 Tuning (new)
- 2.7 Post-crash rebuild 24.0620
(3) SBeeF405/BF (1b)
(4) SBeeF405/AP (1c)
EPIC 2 - Build/fly Pixhawk drone
(5) Pix6c/PX4 (2a)
(6) Pix6c/AP (2b)
EPIC 3 - Add AI to Pixhawk drone
(7) AI cc + cam
(8) AI CC+FW
- (8a) AI Nano PX4
- (8b) AI Nano AP
- (8c) AI PI PX4
- (8d) AI PI AP
- 9.0 SIH NANO/PX4 24.0703 (WORKS!)
EPIC 4 – Basic Autonomy
(11) FC_Mavlink_API (via CC/GCS APIs) (5.3)
(13c) CC autonomy algorithms
EPIC 5 – Advanced Autonomy
(20) Kalman KK (+Python,vectornav)
(14) Firmware dev (5)
(22) AI
- (21) CV
- 13.1 AI Ubuntu
- 13.5 AI STM 24.0430
- 13.6 AI study
EPIC 6 – Project management 24.0903
- 21.1 Project docs (cms+conf/jira)
- 21.2 Git
- 21.3 APIs (create, swagger, publish
- 21.4 End user (dev) docs
EPIC 7 – Mission platforms / special projects
- 15.1 AI Kamikadze 24.0806
(16) Special projects (5.6,5.5)
EPIC 8 – PITS (Pie in the sky)
(12) ROS (+ROSMAV) (was E4)
(14b) Matlab (was E5/4(HITL))
(10) (skip) SITL (total sim) (1) (was E4)