How to send custom data between AI deck and crazyflie while using lighthouse deck?

[pladosz]

Hi,

We are trying to use lighthouse deck and AI deck together, while being able to send custom data between crazyflie and lighthouse deck. We understand that currently this is a bit problematic as they both use UART1 to communicate with crazyflie. I am aware that one solution is to use https://github.com/neoamos/Lighthouse-and-AI-Deck-Compatibility (in essence routing communication via Nina). While this works great, it introduces some issues when trying to find a way to communicate custom data from AI deck to crazyflie. I suppose my question can be split into two

1. Does there exist/ is anyone aware of a way of combining lighthouse deck and AI deck without https://github.com/neoamos/Lighthouse-and-AI-Deck-Compatibility workaround?

2. Is there a tutorial/instruction which shows how to get communication working with AI deck, crazyflie while using Nina workaround? I could find some information here https://github.com/GreenWaves-Technologies/gap_sdk/issues/267 and in the https://github.com/neoamos/Lighthouse-and-AI-Deck-Compatibility but I struggle to compile AI-deck side code.

Thank you for all the help!

[evoggy]

Hi,

Yes, when using the AI deck and Lighthouse deck together the UART1 will be the problem. But the UART1 is only a problem for the GAP8<>STM32 link, the UART2 which is used for ESP32<>STM32 can still be used. We've made an implementation of a protocol we've named CPX which is able to send data back and forth between the GAP8 and STM32 by going though the ESP32. This will add latency, but will make it possible to communicate. Here's the doc

1. Yes, you can use CPX for sending data GAP8<>STM32 via the ESP32.
2. There's no example covering exactly this, but here's a few links to look at:

wifi-img-streamer.c - Using CPX to send and receive data to/from the PC, but this could be redirected to the STM32 by just changing the target

cpx.h - Using CPX on the STM32. This still needs some improvements to it's possible to route the data to an app instead of using it directly from inside the firmware. But hooking in here will allow you to receive the messages.

[pladosz]

Thank you for pointing us in the direction of CPX. It seems exactly like what we need.
Just to clarify your point about CPX on STM32, do you mean CPX cannot currently be used to communicate with the app layer on the STM32 'out of the box'?

[pladosz]

Just to add to my previous comment. If we want to control velocity from AI-deck, would we just send CRTP message over CPX? (based on following PR https://github.com/bitcraze/crazyflie-firmware/pull/1068), Or would we need to have Crazyflie side app to send data to the autopilot?

[evoggy]

You could do either or, but it's probably easiest sending CRTP packets via CPX to the STM32. Note that you will have to send a packet to "switch" CRTP from the radio link (via the NRF) to the CPX link. But after that you should be able to send commands, have a look here to see how to format the package. Note that you have to send these regularly (at least at 10Hz I think) otherwise the Crazyflie will shut off the motors (it will think that it's lost the connection)

[pladosz]

Awesome thanks! I will open another discussion should we have more questions

[noahzn]

@evoggy Hi, if I send the message to the app layer first, and then control the drone, do I still need to meet the requirement of sending commands at 10 Hz? How can I make the drone keep moving forward before it receives a command?

My processing code can only run at 1 frame per second, so I don't know how to keep sending commands when the processing is still running. @pladosz has implemented sending a command directly from the GAP8 to the STM32, and we have a discussion in the link below. Thank you in advance.
pladosz/AI_deck_vel_send_cpx#1