Sim-to-Real

[sesem738]

Hello,

I have trained my Crazyflie in simulation and I am looking forward to implementing it on the real Crazyflie. I used this simulator to train my neural network to land the Crazyflie on a moving platform. The network inputs are position, linear velocity, orientation in quaternion, angular speed, and distance error to the moving platform. The network output is attitude rates. My Crazyflie is the AI bundle and I have access to an Optitrack Motion capture system. I need some advice on how to successfully execute the Sim-to-real transfer.

1. What is the best way to get the network inputs given what I have?
2. Should I run the neural network on the AI deck instead of running it on the computer and sending the output?
3. Any github repos to help?

I am hoping to create a tutorial of the entire process once I am done.

[knmcguire]

Hi! That seems to be possible but it is a question if the network fits on the AIdeck or not.

There has been an autonomous drone challenge last year (see this blogpost), where we saw quite some people use the Webots simulator to train on and implemented it on the AIdeck.

@LorenzoLamberti94 was actually in the team that won that time, perhaps he can give some pointers

[LorenzoLamberti94]

hi!
for onboard deployment, you can refer to this work
https://ieeexplore.ieee.org/document/9606685

https://github.com/pulp-platform/quantlab
https://github.com/pulp-platform/dory
https://github.com/pulp-platform/pulp-dronet

for sim to real, you have to wait a few months while we publish our approach :)

[sesem738]

How do I get the angular velocity from the kalman filter with the Python API?

[knmcguire]

You mean the attitude rate? I don't think the kalman gives that as output. The PID controller actaully makes use of the filtered gyroscope values. These are the logging values for that https://www.bitcraze.io/documentation/repository/crazyflie-firmware/master/api/logs/#gyro

This is the documentation for the crazyflie python api: https://www.bitcraze.io/documentation/repository/crazyflie-lib-python/master/

Just to let you know, for support we generally prefer one question or support issue per thread. I you have any questions that is not related to the aideck, then you should start a new one.