The simulator used is AirSim from Microsoft, window identification was implmented on the Unreal version of AirSim and precision landing wa implemented in the Unity version.
This is achieved by a proportional visual controller that identifies the center of the marker and moves the drone in a parallel plane to the marker until it is centered enough, then it descends and this process is repeated until the dron is at a safa landing distance.
landig.mp4
The idea was to implment a neural network able to identify windows and then fly trough them, however due to time constraints only the identification of windows was implemented.
Data was collected using AirSim´s API, as shown bellow. The data and images provided by the simulaton were written to a directory in pascal_voc format.
window.Detect.speed.up.mp4
Thanks to the simulation, a lot of information was gathered with different climatic and illumination conditions. Using Google Colab, a version of EfficientDet was trained.
Simulated environment:
Data collected:
Network test with dust storm:
Network test in simulation:
