Blog
Will start travelling tomorrow. Tested my travel setup, seems everything is working :)
Powered up, ran object recog test, and the copter (just the FC) went into land mode. :)
- Red: FC with ETH adapter (Nano commands to FC via ethernet)
- Blue: Nano with camera.
Started QGC (on Nano) to check status. Note "Land" in yellow box.
The stand for the monitor is too heavy, so use the travel fan. :)
Note enough USB ports on Nano, but the extension works. Might need to use USB sticks on travel.
I am really big on organization and concepts. I reorg'd the last epics 4/5 into epics 4-6 (see diagram below). Time to redo (yet again) the Wiki organization (5 epics) and the Youtube playlist. :)
The change was motivated by a better understanding (I think its better) of what HITL, SITL, Matlab, and ROS are (now epic 6). For ZiptieAI drone purposes, basically PITS "Pie in the sky". PITS means
- Too complex. You require an entire dev team to integrate. Especially with the AI routines (AI is a core focus of ZiptieAI). Since I am learning this stuff on my own, a big focus is avoid rabbit holes and PITS.
- Whats the point? Sure if you are building customized proprietary drones with a big dev team, then fine. But that's not the ZiptieAI focus (ziptie'ing components is a core focus on ZiptieAI).
The doc 9.0_SIH_nano_px4_ETH_v07_ shows how how I (somehow, magically) got SIH + object detection + mission interrupt working (simulation test)
- SIH (sw in hw)
- nano-px4 connection via ETH (ethernet on telem3)
- obj recognition interrupts mission and lands copter when nano sees human
This is an important milestone (just a few days before my trip to Lviv/Kyiv). HITL demanded too many resources, and I saw no value from using jmavsim. SIH seems to be a practical way to test the logic of python programs on the CC (Nano) that are to be used to make the copter do (potentially complex) maneuvers in response to the Nano AI detecting something. This logic needs to be simulation tested before trying in the field.
Note: I have not seen any other demo like this one that includes the actual setup details. The details for this demo are still a little bit rough. I wanted to get this working (and posted on the wiki) before july 4th and before my impending summer (working) vacation in Lithuania/Ukraine that starts 6th july. Never thought I would actually get a demo of this all working before 6th july!
PS: New wiki page: 9.0 SIH NANO/PX4.
Person recognized (my fingers) when i opened camera shutter (copter is already flying SIH mission)
Nano switches (pymavlink) copter mission mode to land // copter lands.
test setup (everything running on pixhawk 6c and nano)
9.0_HITL_SIH_px4_nano_ETH_v04_24.0701.docx is very sparse text with many pics description of tests with HITL/SIH.
Need this so that can run HITL (I need to use the USB for HITL, so run NANO<>6c via telem2 or telem3/eth).
I think the org below makes sense:
- part 7: setup the cc (nano or pi5/4), cam. also: qgc/startup scripts
- part 8: setup connect/code, then kitchen/field test. for one of 4 combinations
- NANO <> PX4
- NANO <> AP
- PI <> PX4
- PI <> AP
They packaged together the jetson ORIN nano ($500) .. thats wwhat I am trying to do manually with the jetson nano ($150) :).
https://youtu.be/BLXh2JWJ3kc?t=251
$1500 with Orin all preconfigured.. nice, if it works, if the quality is there. If good reviews, i might get one, stop working on the low level grunt stuff, and just focus on the AI :).
PS: The following video is about the Ukrainian "Lucky strike" drone. Looks quite similiar to the X-650 (from Holybro, discussed earlier in the video above). Lucky Strike uses Ardupilot!
https://youtu.be/nQBN8oW8so0?t=74
24.0626 Created a Youtube playlist for ZiptieAI
This is the first time I have created Youtube videos, so still learning the basics.
Updated shopping list added.
24.0619 evening !!! first successful AI object recognition (person) / CC forces land / OUTDOOR test !!!
7.2.2 Flight build test in outdoors
In this video (note the load motor noise from the struggle to keep all that extra weight in the air) I put the copter into loiter (hold) mode and then walked around in front of the camera... the nano AI recognized me as human, and put the copter into land mode (it overrode the loiter mode). Just a 10 second video, but a big deal for me. :)
My first test flight AI object recog with this Pixhawk PX4 copter, and it worked. I am having great luck with this copter.
Note: I did not use a laptop. I just brought the copter to the field, powered on and tested. This was the process:
- Stand behind the camera.
- Open camera lid.
- Turn on TX-12 RC.
- COnnect Pixhawk 6c FC to 4s battery.
- Connect Nano to its phone charger battery.
- Use RC to take off in stabilize.
- Flick switch to go into loiter mode.
- (where video starts) Walk in front of camera (which was pointing down slightly).
- (THe Nano sends command to copter to go into land mode).
Next test will be with mission flight. But tonight I preferred to fly the copter in a slightly different location, so flying the saved in eeprom mission was not possible (dont need a mission to verify the functinality anyway).
Note: I put the Nano+battery at the bottom because I was afraid that the copter leg design was not strong enough to hold the weight on land. The problem is its not easy to fly with the weight at the bottom. But it worked good enough.
For details
- 7.2 JNano 24.0612
24.0617-0618 endless problems getting py script to recognize port AND get heartbeat (finally got a hack to work)
https://discuss.px4.io/t/pymavlink-script-not-getting-heartbeat-from-pixhawk-6c-sometimes/39261
Custom MavLink Packets with Simulink + Connect Multiple instances of Drones to QGroundControl
what i really need is to know what qgc is doing to always get heartbeat. this video shows how to create a simulated drone and setup the heartbeat.
@13:30 He sets up heartbeat emission
@17:30 he is running the sim against qgc. i think the heartbeat part is in the red box.
what i want is to open his example code, and see what matlab generates for the heartbeat. but need to beg for a trial license extension for matlab (again). :) this is why i added months ago section 3.3 HITL Matlab.. i think much of the code for this stuff in px4 was generated in matlab (seems like much of the px4 software was generated.. it has that "generated" look and feel). need to study this in more detail eventually... this is the key (i think) to creating customized FC firmware (bloggers like to boast about creating their custom firmware, but never get into the details).
Seems like getting a heartbeat error. I posted in px4 forum about this.
figured out how to "coax" this to work after a lot of testing on this. Word file for details (with screenshots; see the two chapters “0615 …” for details of pixhawk 6c <> jnano object recognition tests) E3-7.2.1b_jnano_pix6c_pymavlink_tests_v04_24.0609_0615 @ E3-7 AI basic examples (2c) - Google Drive
setup
i ordered this ... thought about connecting via ethernet.. maybe try later.. been wanting to set this up and try.
If this does not work, them maybe try to start from 0... reinstall PX4. clean.
QGC can receive via holybro telem radio link from jnano. Have not figured out how to send (is it possible) using py script via qgc? That functionality once existed i think, but now gone.
See flatpak_install_v02_24.0615
Just when i thought i had it all completed... PX4 throws me a curve ball.
The link between JNano and 6c were working fine on 24.0609 (see jnano_pix6c_pymavlink_tests_v03_24.0609.docx),
but on 0613 did not work. Tried everything for 2 days... no luck.
Cant get a heartbeat from Jnano on ttl-usb or telem2, telem3. tried different connectors. tried various settings in mavlink (screenshot: win11 qgc via tele radio).
also tried using pi5. same problem.. no heartbeat. used various cables, usb, ftdi adapters. no luck.
@ ~3-4 meters, ~11 fps (good enough for flight test)
win11: mode = stabilize (link via wireless telem)
nano camera (920s) recognizes person (at ~22 frames/sec @ 1080x720 (!) :)
nano changes the FC mode to land (screenshot from win11: mode = land)
setup (could use RC to change modes, start a flight mission (no props, but motors spin), etc)
Now need to
- 3 Figure out how to power Jnano (own battery or split from drone battery?)
- 4 start up procedure
then plan to test (very simple test)
- 1 copter flies simple mission.
- 2 I walk in front of camera, jnano recognizes me, and commands copter to land
trying hitl with jnano object recog + pymavlink changing the flight mode to land. so many problems...
- 1 i was trying to run mavproxy while running the pymavlink. i think thats a no no. did not realize it until late.
- 2 thats caused because i was using both usb ports (top and side)... and internally apparently these use same hardware.
- 3 hitl requires the usb, and the usb (ttl) is the only reliable connection i have so far.
- 4 so i need to find an other way to connect. (telem2, etc).
- 5 or just go to SITL (which is probably good enough).
person detected at 22 fps :)
Switch between RTL and land works.
Now need to
- 1 add 920s camera to jnano (did this before).. or picam? (920s is usb connection.. simpler, good quality, but at distance?)
- 2 Add obj recog to Jnano (did this months ago)
- 3 Figure out how to power Jnano (own battery or split from drone battery?)
- 4 start up procedure
then plan to test (very simple test)
- 0 HITL test? (jmavsim... done this before)
- 1 copter flies simple mission.
- 2 I walk in front of camera, jnano recognizes me, and commands copter to land
mavlink links
do with mavsdk in future?
Strange feeling after all the problems with the FPV drone :) .
Video of mission plan flight (I flicked a switched and the copter did the rest by itself
Making stupid mistakes with cheap FPV and then training on Liftoff simulator paid off. :)
Video of loiter/hold + land in wind.
this works:
- motor test spin
- take outside and get a gps lock
- rc inputs
See
seems like everything need for test flight. but..
- but need to add legs.
rain today.. first attempt flight maybe tomorrow.
tested this morning.. one motor messed up, flipped over and hit (wet) dirt. This will take some time to fix. time to switch to Pixhawk 6c drone.
1a.5 Flight 5 (missions / flight modes)
Click Simulation video to see how I recommend to any beginner to train on Liftoff simulator. For more info see FPV simulators.
24.0603 drone flies good (phone video of last part of RTH): Failsafe, Angle and RTH modes work
Training with the simulator + working angle mode made it much more relaxing flying (under control).
MODES tested:
- 1 Beep OK
- 2 Failsafe (land) OK
- 3 Nav poshold.. seemed to start rocketing into space again.. FAILED?
- 4 NAV RTH (return to home) OK
- 5 Angle OK (Great)
- 6 Nav WP (waypoints, mission) TOMORROW.. not enough battery and gps blinking strange... will try mission tomorrow.
VTX tx wire came loose.. cheap wires (SBEE wires)
1a.3b-Smart-audio#setting-levels-band--works-240602-using-tx12-lua-and-switch
switch
lua
sticks
24.0602 Added a new part to this wiki: X Special projects 24.0602
Right now only contains 2 sections:
- X.6 Stealth (wired comms). The idea is to have a drone that makes no wireless comm emissions.
- X.2 ESP32 Radar. Not sure about this.. Just discovered today. Seems to be a way to discover those (uavs) around you if they have the same equipment? Or is it like real radar?
24.0601 Getting ready to test INAV mission flight (not this weekend... bad weather)
Liftoff was $20 very well spent. See FPV-simulators.
Why INAV climbs rapidly in Altitude Hold mode?. This video explains why in one of my first flights when I took off the copter shot up into the sky, straight up?? There is an error in the flight algorigthms in INAV?
(details)
i am starting to appreciate how flying an FPV is like learning to drive a car. when i first learned to drive, i could not keep the car going straight, was constantly correctly the steering wheel. braking was a challenge. i hated it... moving a ton of steel down the road at 100 km/h was not fun. but after a while you subconciously learn to make inputs that smooth out movements.
same with flying a drone. except
- you are in air (more complex)
- crashing wont kill you (if the copter does not crash into you)
crashed several times.. battery thrown off, 2-3 zipties broken that held it in place. but the copter still works. impressive. today was the first day i could hold it in the air for a few minutes and move around.. just now starting to get used to the sticks. just like when i first started drive a car. i have now learned respect for the skills of good copter pilots. amazing that these aspects for first pilots are never discussed by the experts.
(I added a piece of plywood down the middle to mount the GPS and RX on (so that its not so crowded). For flight 2 (outside).
24.0524 First fairly useful version of the shopping list posted.
24.0521 Added my first tech overview (7a Tech overviews).
These tech overviews will explain tech concepts behind major copter subsystems that the experts never explain (or problems/issues they conveniently ignore when telling you how easy things are).
24.0515 Created page Part 7 Config files.
Describes types of config files that will be stored on the gdrive. These files are the critical result of configurations, but for some reason the experts rarely talk about them. File types include .hex, .bin, .yml, .etx, .txt.
24.0514 Updated The AI part
with a unique definition of AI for the the ZiptieAI project.
(goal is, among other things, "stealth" drone (all comms over wire, no wireless xmissions... see 5.6 Stealth (wired comms))):
- 24.0511 Built edgtx (rc firmware). Goal is to understand how RC comms with FC (via Rx). See 5.5 PA5 RxTx dev.
- 24.0505 Ran my first px4 mod. Tested on Pixhawk 6c. See 5.2 FC FW dev.
- 24.05 Built PX4 (firmware). Tested on Pixhawk 6c. See 4b.4 PB4 Firmware build.
-
24.0507 Installed INAV on SpeedyBee F405 and completed "kitchen table flight test". See 4a.5 SpeedyBee F405v3 config.
-
24.0415 Installed Betaflight (BF) on SpeedyBee F405 and completed "kitchen table flight test" (my very first build). See 4a.5 SpeedyBee F405v3 config.
- 24.0422 Installed PX4 on Pixhawk 6c and completed "kitchen table flight test". See 4b.5 Pixhawk 6c config.
download from vehicle
{ "fileType": "Plan", "geoFence": { "circles": [ ], "polygons": [ ], "version": 2 }, "groundStation": "QGroundControl", "mission": { "cruiseSpeed": 15, "firmwareType": 12, "globalPlanAltitudeMode": 1, "hoverSpeed": 5, "items": [ { "autoContinue": true, "command": 178, "doJumpId": 1, "frame": 2, "params": [ 1, 1.34112, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 3.07848, "AltitudeMode": 1, "autoContinue": true, "command": 22, "doJumpId": 2, "frame": 3, "params": [ 0, 0, 0, null, 39.2906189, -85.21254549999999, 3.07848 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 3.1089599999999997, "AltitudeMode": 1, "autoContinue": true, "command": 16, "doJumpId": 3, "frame": 3, "params": [ 5, 0, 0, null, 39.290677441725286, -85.21242845493745, 3.1089599999999997 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 4, "frame": 2, "params": [ 1, 1.430528, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 3.1394400000000005, "AltitudeMode": 1, "autoContinue": true, "command": 16, "doJumpId": 5, "frame": 3, "params": [ 5, 0, 0, null, 39.29051529181098, -85.2124325814585, 3.1394400000000005 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 6, "frame": 2, "params": [ 1, 1.4752319999999999, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 0, "AltitudeMode": 1, "autoContinue": true, "command": 21, "doJumpId": 7, "frame": 3, "params": [ 0, 0, 0, null, 39.290568, -85.2125471, 0 ], "type": "SimpleItem" } ], "plannedHomePosition": [ 39.29059337396191, -85.21254665630224, 297 ], "vehicleType": 2, "version": 2 }, "rallyPoints": { "points": [ ], "version": 2 }, "version": 1 }
mission start
takeoff
waypoint (3)
waypoint (5)
land
{ "fileType": "Plan", "geoFence": { "circles": [ ], "polygons": [ ], "version": 2 }, "groundStation": "QGroundControl", "mission": { "cruiseSpeed": 15, "firmwareType": 12, "globalPlanAltitudeMode": 1, "hoverSpeed": 5, "items": [ { "autoContinue": true, "command": 178, "doJumpId": 1, "frame": 2, "params": [ 1, 2.2352, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 2.16408, "AltitudeMode": 1, "autoContinue": true, "command": 22, "doJumpId": 2, "frame": 3, "params": [ 0, 0, 0, null, 39.290618958778154, -85.21254559958125, 2.16408 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 2.49936, "AltitudeMode": 1, "autoContinue": true, "command": 16, "doJumpId": 3, "frame": 3, "params": [ 2, 0, 0, null, 39.290644090845596, -85.21245626221605, 2.49936 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 4, "frame": 2, "params": [ 1, 2.3246080000000005, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 2.5298400000000005, "AltitudeMode": 1, "autoContinue": true, "command": 16, "doJumpId": 5, "frame": 3, "params": [ 3, 0, 0, null, 39.29053454640151, -85.21245748652615, 2.5298400000000005 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 6, "frame": 2, "params": [ 1, 2.369312, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 0, "AltitudeMode": 1, "autoContinue": true, "command": 21, "doJumpId": 7, "frame": 3, "params": [ 0, 0, 0, null, 39.29056800942455, -85.21254710190264, 0 ], "type": "SimpleItem" } ], "plannedHomePosition": [ 39.290618958778154, -85.21254559958125, null ], "vehicleType": 2, "version": 2 }, "rallyPoints": { "points": [ ], "version": 2 }, "version": 1 }
uploaded
{ "fileType": "Plan", "geoFence": { "circles": [ ], "polygons": [ ], "version": 2 }, "groundStation": "QGroundControl", "mission": { "cruiseSpeed": 15, "firmwareType": 12, "globalPlanAltitudeMode": 1, "hoverSpeed": 5, "items": [ { "autoContinue": true, "command": 530, "doJumpId": 1, "frame": 2, "params": [ 0, 2, null, null, null, null, null ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 2, "frame": 2, "params": [ 1, 5, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 3.0510480403900146, "AltitudeMode": 1, "autoContinue": true, "command": 22, "doJumpId": 3, "frame": 3, "params": [ 0, 0, 0, null, 39.2906012, -85.212526, 3.0510480403900146 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 3.060192108154297, "AltitudeMode": 1, "autoContinue": true, "command": 16, "doJumpId": 4, "frame": 3, "params": [ 4, 0, 0, null, 39.290656299999995, -85.2124032, 3.060192108154297 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 5, "frame": 2, "params": [ 1, 2.414016008377075, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 3.0662879943847656, "AltitudeMode": 1, "autoContinue": true, "command": 16, "doJumpId": 6, "frame": 3, "params": [ 6, 0, 0, null, 39.2904809, -85.2124006, 3.0662879943847656 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 178, "doJumpId": 7, "frame": 2, "params": [ 1, 2.5034239292144775, -1, 0, 0, 0, 0 ], "type": "SimpleItem" }, { "AMSLAltAboveTerrain": null, "Altitude": 0, "AltitudeMode": 1, "autoContinue": true, "command": 21, "doJumpId": 8, "frame": 3, "params": [ 0, 0, 0, null, 39.2905595, -85.21252319999999, 0 ], "type": "SimpleItem" }, { "autoContinue": true, "command": 20, "doJumpId": 9, "frame": 2, "params": [ 0, 0, 0, 0, 0, 0, 0 ], "type": "SimpleItem" } ], "plannedHomePosition": [ 39.29087099617688, -85.212526, 297 ], "vehicleType": 2, "version": 2 }, "rallyPoints": { "points": [ ], "version": 2 }, "version": 1 }