VTOL Offboard attitude setpoints getting reset/overwritten

[RicardoM17]

So I'm currently trying to run a VTOL Offboard simulation in gazebo but I'm having some issues.

The model used is the standard gazebo tailsitter. Using yesterdays master though I don't think it's too relevant.

I'm publishing a message on /mavros/setpoint_raw/attitude with the attitude quaternion and a thrust value. The type_mask used is 7.

The published message is pretty much always the same and without any major issues. However as you can see by this log the setpoints keep getting "reset" or overwritten which disrupts the performance of the drone.

https://logs.px4.io/plot_app?log=83530229-b60d-422a-83e0-5aeb800bbc27

This is a log running the same code in another machine in which we don't have this issue:

https://logs.px4.io/plot_app?log=69b60928-77ee-4fab-9908-0944b598d715

However even in this scenario it's only because it hit the "sweet spot" because depending on the performance of the offboard "node" it can also have the same issue.

If I look at the the topic /mavros/setpoint_raw/target_attitude I see instead that it jumps a lot. For example in the thrust it's always jumping between 0 and 1.0.

Am I wrong in assuming that #13141 should support this type of control or perhaps it's just that I am doing something wrong.

Also I understand that the publishing rate is quite important but I am wrong in assuming that if there is a a small delay in the publishing the message won't be overwritten? (which is the case I'm finding where a message is overwritten several times a second).

Apologies if this is not the right place to ask.

[dagar]

@Jaeyoung-Lim any thoughts here?

[Jaeyoung-Lim]

I tried reproducing it through from the current master

I cannot reproduce the problem with the setpoints jumping. The log can be seen as below.
Log: https://review.px4.io/plot_app?log=668bd418-f042-462c-8c31-035dd07de365

Also attitude setpoints in MC mode looked reasonable:
Log: https://review.px4.io/plot_app?log=bf06d53b-e094-40c9-b0f1-ccb9b1813548

I sent attitude setpoints which is supposed to track a circular trajectory for VTOL MC mode and send a transition service through mavros to push it to vtol FW mode. The attitude setpoints are being streamed at a 100Hz. The type mask is also defined as 7.

@RicardoM17 I am not sure what is going on with your setup. Could you explain more on what the performance of the node means?

[RicardoM17]

To add some more information. Here is the recorded rosbag from the respective topics.

/mavros/setpoint_raw/attitude . In this case I'm plotting the thrust value because it's simpler to visualize.

As you can see the value is "normal" with only small jumps that come from my controllers. However compared to the same plot in /mavros/setpoint_raw/target_attitude which then gets "propagated" to the drone:

As you can see the graphics somewhat match but there is tons of dropouts where the value is 0.0. Like I asked on my post am I wrong in assuming that these should should match?

In this test /mavros/setpoint_raw/attitude is being published at around 50Hz while /mavros/setpoint_raw/target_attitude is being published much slower around 5-10Hz.

To clarify the "performance of the node statement". Currently the node that is sending the /mavros/setpoint_raw/attitude messages is a simulink model. The way it is setup it will run at different speeds depending on the computer. We also slow down the gazebo simulation to "match" them more closely. However depending on the exact speed that we run gazebo we can have clean messages with no dropouts and consequently a perfect trajectory like the one I linked in the original post.

I'll test with your node next and get back here with the results.

[RicardoM17]

Another update:

It seems that this is not directly related to my node (but it could be something with my system. Any debug tips or possible solutions are very much appreciated).

I started a gazebo simulation with a tailstter, took off and transitioned via QGC and then published a message at 100 Hz using:

rostopic pub -r 100 /mavros/setpoint_raw/attitude mavros_msgs/AttitudeTarget "header: seq: 0 stamp: {secs: 0, nsecs: 0} frame_id: '' type_mask: 7 orientation: {x: -0.05, y: -0.3, z: 0.84, w: 0.43} body_rate: {x: 0.0, y: 0.0, z: 0.0} thrust: 1.0"
and I still get the same issue with the mixed target_attitude topic messages as can be seen in this log:

https://logs.px4.io/plot_app?log=14f6e81b-9c2b-42d1-96bd-93e349043f63

I also managed to replicate this in another machine:

https://logs.px4.io/plot_app?log=cc945981-3ece-443a-90c0-24771b922460

Edit: I understand that this particular quaternion doesn't have the correct norm. However results are the same with a 0,0,0,1 no rotation quaternion.

[RicardoM17]

During my research I found something interesting. In #13209 @ThomasRigi shares a log in which some similar behaviour is observed. The difference between that it's not being observed at all times and also not in thrust but only in Pitch and Roll. attached link for convenience:

https://logs.px4.io/plot_app?log=1fae08b3-82f3-4a45-b5d9-ff997bb9e7a3

@ThomasRigi is this something that you have experienced in your tests/development? I understand that you are using the MAVSDK but perhaps there is some overlap here.

[ThomasRigi]

@RicardoM17 I don't think that these issues are very related. I had a look at your attitude setpoint in the .csv file generated with pyulog. The faulty zero thrust setpoints are new setpoints that the drone receives from somewhere (the timestamp is increasing on every setpoint). In my case the issue is (HIL) that there is an old attitude setpoint that gets copied ever and ever again, suppressing/overwriting the new ones. This problem has been completely solved in #13141 for SITL, but somehow not for HIL (why HIL doesn't work is still completely unclear to me). On a sidenote: thrust is also affected, it just doesn't show up on the plot. It should be 0.6 during the first offboard phase.

I have never touched ROS, but I think your issue lies between /mavros/setpoint_raw/attitude and /mavros/setpoint_raw/target_attitude (what is the difference / why are there two of them? :S )

I hope you find the bug soon!

[RicardoM17]

I'm digging in and thanks for the feedback :)

Yes I agree with you @ThomasRigi those faulty setpoints are coming from someplace else (I suspect from the PX4 side as from ROS nodes I've gone to using rostopic pubs so I'm pretty sure it's not from there. also rqt_graph finds no culprits.

I just mentioned you because I saw in your log stuff like this:

Which to me looks similar. Is this the old setpoint that you mentioned that gets copied?

From the mavlink docs it says about the ATTITUDE_TARGET msg

Reports the current commanded attitude of the vehicle as specified by the autopilot. This should match the commands sent in a SET_ATTITUDE_TARGET message if the vehicle is being controlled this way.

https://mavlink.io/en/messages/common.html#ATTITUDE_TARGET

Which is not happening.

I'll have to check where this message is sent in PX4 and investigate further but obviously it would be good to have some tips from the more experienced devs ;)

[ThomasRigi]

I see this part of the graph for the first time now 😬 This happens during the offboard body control phase, using offboard->set_velocity_body() function from the MAVSDK. It is no longer an attitude setpoint that is sent with offboard control, but something more high level (a velocity setpoint judging from the name). During this phase the drone should fly a circle. I never worked with this function, it's just there in the offboard example. I don't know what's going on during that part, but I guess it's somehow linked to the actuator saturation that happens at the same time. In any case the attitude setpoints come from within PX4.

The problem with the old setpoint is only when using offboard->set_attitude(), i.e. from 0:39-0:48

[Jaeyoung-Lim]

@RicardoM17 A rosbag would be helpful to find out what is going on. I suspect that the zero setpoints are actually being sent by something and not being set by the firmware

[RicardoM17]

@Jaeyoung-Lim here is the rosbag:

https://we.tl/t-TLYzN7Rtsr

(Github doesn't support this format so I had to use WeTransfer but if you wish let me know and I can send it to you directly via Slack/e-mail).

Here is the corresponding log:

https://logs.px4.io/plot_app?log=81650a2b-abbc-4f6c-82e4-80b9d544a68f

(there is quite some idle time at the beginning while I was setting everything up but the rosbag doesn't have this issue.)

For the record I had a rostopic publishing into /mavros/setpoint_raw/attitude and one echoing /mavros/setpoint_raw/target_attitude

Also here is the exact same procedure but without transitioning into VTOL FW

https://logs.px4.io/plot_app?log=4a2a1af2-2cb3-49dc-a533-179483f55ecd

Obviously the trajectory followed is junk as I didn't want to change the published message but as you can see this issue is not present so I fear it's something on the Firmware side, more precisely either on the FW attitude controller or the VTOL controller.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. Thank you for your contributions.

[Jaeyoung-Lim]

@likehengqq Please do not hijack the topic of this issue. You can post a new issue or post your question in
the public forum(discuss.px4.io)