Rover is getting stuck repeatedly or the quadcopter flips

[PrasadNR]

Describe the Bug
I have been working on Webots to get my object tracking drone up and running. Link: https://github.com/PrasadNR/Webots-Quadcopter-Python-SITL . However, when the simulation time step in WorldInfo is small, the rover gets stuck. If it is large, the quadcopter flips. If I simulate quadcopter and rover separately, they both work properly in independent simulations. However, I am not able to get both of them running.

Steps to Reproduce

1. Clone the repo.
2. Just double click on (or run) Webots-Quadcopter-Python-SITL/blob/master/worlds/empty.wbt
3. Change WorldInfo -> basicTimeStep to 8
4. See error

Expected behavior
Rover should not get stuck and the quadcopter should not flip. The behaviour has to be graceful.

System

• Operating System: Windows 8.1
• Graphics Card: NVIDIA GeForce GTX 860M

Additional context
I have attached the video.
roverStuck.zip

P.S.: I have discussed this with @omichel .

[PrasadNR]

In the meanwhile, are there any quick patches or workarounds for this?

EDIT: Also, can anybody let me know the root cause of it? If my intuition is right, I can fix it. I remember trying something in ODE in Windows to replace linux specific ROS Gazebo before.

[omichel]

I am having trouble installing the Python dependencies on Windows 10. Which version of Python/pip do your use?

[omichel]

Because of missing Python dependencies, the drone doesn't fly and remains on the top of the Pioneer robot. However, when I change the WorldInfo.basicTimeStep to 8, the Pioneer robot is still able to follow the track, moving normally. So it seems I cannot reproduce the problem... Could you try to replace the controller of the drone by a void controller and see whether the Pioneer robot works (with a WorldInfo.basicTimeStep of 8)? In my case it works.

Note that I will be off next week, so my colleagues will follow-up on this issue.

[PrasadNR]

@omichel I am using the following:

Python 3.7.5 (tags/v3.7.5:5c02a39a0b, Oct 15 2019, 00:11:34) [MSC v.1916 64 bit (AMD64)] on win32

pip 19.3.1 from c:\users\prasad\appdata\local\programs\python\python37\lib\site-
packages\pip (python 3.7)

[PrasadNR]

Also, regarding the rover, I am not sure what the issue is. If I set TIME_STEP to 64, it follows the line (even for like 8 hours. I tested all these using Fast Mode. Thanks to Webots). But, when I set to something like TIME_STEP = 4 or so, it suddenly veers off the track after a while. I am not sure if quadcopter physics is interfering.

Plus, I keep getting the following warning:

WARNING: The current physics step could not be computed correctly. Your world may be too complex. If this problem persists, try simplifying your bounding object(s), reducing the number of joints, or reducing WorldInfo.basicTimeStep.

[PrasadNR]

Just a note. I am developing this repository being here in Bangalore. After 3 weeks or so, I may be moving to a different country and it may not be possible for me to devote time to Webots. If anybody could give me hints as to what could be done (at least temporarily), I would be happy to fix this bug myself and share my repository for review (so as to include in official examples).

[PrasadNR]

So it seems I cannot reproduce the problem...

@omichel I am able to replicate the issue (just by following the steps I listed in my bug report). Thanks for looking into the issue though.

Please see the video (this time I figured out that I can record video within Webots. That is an awesome feature! 🙂 ).
Webots.zip

[PrasadNR]

I believe one of these is happening. Please feel free to correct me if I am wrong.

1. In Webots, Gazebo is being run with some additional GUI. At least, they are more or less similar internally.

2. The wheel slip plugin is not working correctly (especially when the time step is small) as there may be some sort of synchronisation issue.

3. The internal code within Webots is running GUI asynchronously with mutex/semaphore locks or something similar and that is causing issues.

But, how can I start digging into the Webots source code?

[fabienrohrer]

I allow myself to follow up this topic, as @omichel is off this week.

I looked quickly at your issue, I can run your simulation in my environment.

• I can indeed reproduce the issue about the Mavic2 which changes its behavior depending on the WorldInfo.basicTimeStep field. However, I think this is a quite normal behavior: this simulation is very sensitive on inputs, constants have been empirically found and optimized for the mavic_2_pro.wbt simulation only. If you use other physics parameters (including WorldInfo.basicTimeStep), these constants should be adjusted. Moreover, the example is more a proof of concept than an example to follow. If you would like to go further in drone control, I would recommend you to use an existing drone control library, and wrap it with our simulation.
• About the Pioneer3at behavior which depends on the WorldInfo.basicTimeStep field, this is more surprising. I cannot reproduce this issue. Are you sure it does not comes from your controller which seems to stop the robot in case of some OpenCV event, and try to display some OpenCV window (which may block the controller)?

Let me also answer your misc questions:

In Webots, Gazebo is being run with some additional GUI. At least, they are more or less similar internally.

Webots and Gazebo are not linked at all in any way :-)
Webots uses Qt for this GUI, ODE for its physics engine, and WREN (home made rendering engine9 for its rendering.

The wheel slip plugin is not working correctly (especially when the time step is small) as there may be some sort of synchronisation issue.

The wheel slip is managed by ODE.

The internal code within Webots is running GUI asynchronously with mutex/semaphore locks or something similar and that is causing issues.

Maybe. But I would suspect first your Pioneer controller. Could you simplify it? (removing the cv2 code for example)

But, how can I start digging into the Webots source code?

You can build Webots from sources using these instructions: https://github.com/cyberbotics/webots/wiki#installation-of-the-webots-development-environment

---

I would recommend you to open more issues, but to reduce their focus and simplify them at best first ;-)

[fabienrohrer]

Moreover, to have a similar setup than mavic_2_pro.wbt, you really need to copy the WorldInfo fields (basicTimeStep, and damping) from there:
https://github.com/cyberbotics/webots/blob/revision/projects/robots/dji/mavic/worlds/mavic_2_pro.wbt

I just observed it's not the case on your repo@master..

[PrasadNR]

@fabienrohrer Thanks for looking into this. I was testing teleoperation long ago. That is why I have one variable teleoperation in my rover controller. I always set it to false (at least while testing the quadcopter code).

About the Pioneer3at behavior which depends on the WorldInfo.basicTimeStep field, this is more surprising. I cannot reproduce this issue. Are you sure it does not comes from your controller which seems to stop the robot in case of some OpenCV event, and try to display some OpenCV window (which may block the controller)?

My controller just follows the line. I would have been happy if the robot were to have stopped. However, as seen in the video, the wheels rotate (which indicates that my controller works) and the robot does not move (which means that ODE integration is failing somewhere)

My Pioneer controller code just sets the velocities of the motors. I can create another repo to test it. However, I would request you to let me know all those steps that you need me to test. When I test quadcopter alone or rover alone, they both work in independent simulations. However, when I try to test both, there will be some physics problem.

Moreover, to have a similar setup than mavic_2_pro.wbt, you really need to copy the WorldInfo fields (basicTimeStep, and damping) from there:
https://github.com/cyberbotics/webots/blob/revision/projects/robots/dji/mavic/worlds/mavic_2_pro.wbt

I just observed it's not the case on your repo@master..

Thanks for looking into my code @fabienrohrer . However, I am trying to simulate realistic physics without any additional external parameters (at least for now).

[PrasadNR]

Also, when debugging, I am seeing that the camera is going gray (exactly at the place where the rover is getting stuck and the wheels are rotating) for no reason in certain locations in image. I am not sure if my computer Lenovo Y50-70 is not able to run Webots so fast. But, I was able to run a full fledged ROS Gazebo simulation that I built (as mentioned in research).

EDIT: In that case, I would be happier if the GUI is not realtime. But, the fidelity of simulation has to be preserved. I am not sure what is causing this.

[PrasadNR]

As per discussion with @fabienrohrer , I will be trying a workaround for now. I will likely push a workaround by day after tomorrow. However, the workaround solution may not directly resolve this bug.

To all future readers, please note: This is the main reason why I like Webots. It makes development so much easier. Sure, there may be bugs. When I wrote my custom ROS Gazebo simulator, I did not face any bugs like this. That was awesome too. But, I had to install OS about 15 times during crashes for ROS and Gazebo! Webots is cross platform + there is active development.

[fabienrohrer]

Thank you for the feedback.

I am seeing that the camera is going gray (exactly at the place where the rover is getting stuck and the wheels are rotating) for no reason in certain locations in image.

For the log too, this bug is because the rover camera is defined inside the robot shape. Adding a translation offset solves the issue (typically by setting translation 0 0 -0.25 and near 0.01 for Pioneer3at.extensionSlot.Camera).

[PrasadNR]

I just archived my repository as I could not work with this bug. I failed to find any production-ready workaround. Link: https://github.com/PrasadNR/Webots-Quadcopter-Python-SITL

From README

Note: This repository fundamentally uses Webots to simulate robots. I found a critical bug in Webots. Camera flickers and rover gets stuck repeatedly. I tried to find a workaround. Link: #1081 However, in Webots, the coordinates are inherently flipped (y-axis along the floor). Given the problems of camera not working, rover getting stuck and flipped coordinates, I could understand that the controller cannot be made production-ready. So, I have archived this repository. However, the flight controller is built on sound math. So, it should work in any SITL or Python-based hardware.

[fabienrohrer]

Thank you for the feedback, and sorry to read this. I think that these issues could be fixed as we have a working C program doing this released in production.

I hope you've not done with Webots ;-)