Could we have some kind of sub-protocol to handle multi-GCS configurations in a graceful manner?

[Davidsastresas]

I have seen this situation recursively. Multi GCS configurations where the "request control" functionality is needed.

Right now, if the autopilot supports it by a parameter or similar, like Ardupilot does, we can do workarounds playing with GCS-GCS custom mavlink messaging. But we don't have any mavlink standard for this, and I think we could have, it is a nice feature.

I am thinking about something like:

• A new message, broadcasted by the UAV, indicating which sysid is in control of the UAV. This message could be broadcasted at a very small rate, as long as the autopilot sends a new one as soon as control changes. Besides for the protocol itself, this would be good for displaying on a GCS if it is currently in control or not of the vehicle it is connected to.
• A GCS wanting to get control, would get the above message, extract the system id in control, and send a command requesting control.
• The GCS currently in control would receive the message from the other GCS, and could display a popup for the user informing of the other GCS wanting control, and it could approve or decline it.
• In case the GCS in control accepts the control request, it sends a new MAV_CMD to the autopilot, to command the change.
• Autopilot would broadcast immediately the message in point 1, so both GCS would be updated of the change, and could indicate it to the user.

All the above is inspired by how we work with it on gimbal protocol v2 and GIMBAL_MANAGER_STATUS.

Would something like this be valuable? I would like to hear your thoughts @hamishwillee @julianoes

[hamishwillee]

Thanks @Davidsastresas

This has come up before, both for multi GCS control of components (e.g. separate flight and gimbal control) and also for multi GCS observers.

There is a (closed) proposal here #1954 (discussion rambled off-topic but the core bits may be useful). I can't remember the detail, but I think it was conceptually similar. It was closed by the author and I don't remember why.

I think it is a reasonable use case - if you need it and want to provide an implementation then very happy to discuss/take this further.

[Davidsastresas]

Thank you @hamishwillee, very interesting. I read that proposal. There were some very interesting topics in that discussion. I think I miss the point at the ending, as you guys reference something you discussed in the dev call, and I have no details about it.

From what I understood from that thread, the way they presented it on its totality would mean a bunch of other changes in the mavlink protocol in general. This is not necessarily bad of course, some very valid points were mentioned there, and they seemed to have spent a bunch of time studying the situation.

But maybe it is overkill to discuss about such substantial changes it if we can have something working with the current version of mavlink, without messing with system id adquisition protocol and some other topics raised on that thread.

I wasn't aware of the current CHANGE_OPERATOR_CONTROL messages. It is cool, but I would add a broadcast message, like:

MSG CONTROL_STATUS: Message broadcasted by the vehicle, by the "owner" of a system ID corresponding to a vehicle ( A payload should not broadcast it ). It could be a very low rate as long as the system sends it right away after changed. I can think of the following field:

• Sys id in control

I like the fact that CHANGE_OPERATOR_CONTROL and CHANGE_OPERATOR_CONTROL_ACK are standalone messages and not MAV_CMD based. Because this would allow an interesting scenario:

• The GCS wanting to get control sends CHANGE_OPERATOR_CONTROL, and it doesn't only arrives to the autopilot but also to the current GCS in control.
• This would allow an implementation where the autopilot is not responsible for accepting/rejecting control, but the current GCS in control is.
• This way, if a GCS is in control, there isn't any way for another GCS to take over unless the GCS in control explicitely allows it, or if it looses connection and the vehicle is "free of control". That is why MSG_CONTROL_STATUS would be an important addition, it would help understand all the nodes in the system who is in control.

So, with the adition of that CONTROL_STATUS message I would be happy. However, I think it is worth discussing some "guidelines" or actual specifications of a first iteration of a "operator control microservice". For instance, these are some topics that come to my mind would be worth discussing, and trying to reach a consensus:

• Regarding the example above where an autopilot isn't responsible of accepting/rejecting control, but the GCS in control is. Do we want always this behaviour? It is certainly interesting in a lot of situations, but maybe there are other situations where the autopilot being in control of that makes more sense. Then it would be up to the autopilot to establish priorities about which GCS should override other GCS control. This could be programmed in an autopilot by parameters, or in case of Ardupilot a Lua script, some kind of small database of the GCS system ids that are most likely to ask for control, and set priorities for them. Or maybe this can be done already with the "passkey" field of CHANGE_OPERATOR_CONTROL. Maybe the autopilot could have 2 or 3 levels of passkey priorities, so depending on the passkey provided by a GCS the autopilot will grant current control or not.
  As the CHANGE_OPERATOR_CONTROL_ACK has different options for the reason control was rejected, this could work I think.

• Should we consider gimbal/camera control? I think nothing should be changed at the level protocol about this, because I think currently it would be possible for a GCS to have main control, and another one just gimbal control. The GCS not in control will receive gimbal/camera telemetry anyway, and they can always attempt to use gimbal manager v2 protocol to control them, even if they are not on control of the vehicle. I don't think at the mavlink level there is any major issue with it right now. I think the issue is just actually a consensus about how this looks like, and document it so all the parties can implement this particularity for gimbal/camera control.

Now, not strictly related with the scope of this discussion, but kind of related too:

The issue mentioned in #1954 about several GCS having the same sysid is a problem indeed. Of course it is not a mavlink protocol issue, as if the architecture of the system has the system id on all the nodes correctly set this would not even be a problem. But I think we can all agree this is a very likely situation to happen eventually.

So, I would like to also establish some guidelines about this, something for GCS to implement to try to prevent or fix ASAP when they detect there is another GCS in the system with same sys id. I think the straightforward approach would be to evaluate this when any mavlink traffic is received.

If a GCS connects to a vehicle, and through the network of that vehicle it sees another GCS heartbeat with same sys id, it changes its sysid. Maybe it can prompt the user with a dialog explaining the issue, and auto suggest a new system id, and maybe stop sending momentarily heartbeat to that vehicle, until the situation is resolved.

This would be possible with the current protocol right? Of course it would not be extremely robust and "idiotproof", but as discussed in #1954 if we keep adding layers and layers of robustness we will end up re-inventing the wheel.

But for the moment, I think it is worth at least establishing some guidelines about this and document them. I think we don't need much or any change in the protocol for this particular matter, and I think it would be good to have something very basic about the matter, so new projects facing this need could at least have a base to work over, or we could all have a base from where we can possibly start discussing building a more sophisticated new mavlink version in the future, possibly considering this discussion on the improvements.

Thanks!

[hamishwillee]

@Davidsastresas Grep of PX4 and ArduPilot are not showing an implementation of CHANGE_OPERATOR_CONTROL. I think we

• need some more signs of interest in the autopilots before we invest too heavily in this.
• would from working out what use cases we want to address - control over everything, or control over a gimbal or whatever.
• In terms of owns the decision about what GCS owns the system. My own "2 bits" is that whoever owns the decision it is the autopilot that should mediate the handover. I would actually prefer this be done with a command, and yes, it should be possible to query as to the ownership, in particular if you are interested in taking it over.

The problem of duplicate ID allocation is hard to detect - it is possible for commands to loop back through different channels in a network, so to know that the message you just got that has your ID was not sent by you, you'd have to remember that you didn't sent it. Lots of work that shouldn't be needed. I'd prefer a setup where all system ids were allocated from a server.

FYI the RAS-A variant of mavlink, which only lives on IP networks, uses the IP address instead of a system ID for addressing. This can be allocated like any other IP address. Which is all very cool provided you only want to run on IP networks.

I've added to the MAVlink call agenda to see what others think.

[julianoes]

One use case for a clear MAVLink operator notion to me is the MANUAL_CONTROL message. With this one, I can see conflicts between ground stations both sending it.

[hamishwillee]

So the summary of mav call is that:

• we can see clear use cases for this.
• Whether it is the GCS or the autopilot that decides who has control, the autopilot is what has to mediate the discussion
• There also has to be a way for a GCS to seize control in the event that the owning GCS does not allow it.
• We're not interested in this as a security feature so much as a collaborative sharing feature. If you wanted this secure you'd use it over a secure network.
• Needs commitment to implement.
• @julianoes is interested and will continue this discussion.

[Davidsastresas]

Sorry I missed the call, I read your message too late. About your points, it looks great to me, I think it is a great summary of the scope of this, and I would be totally happy with such changes.

I will try to catch the attention of Ardupilot for them to join. Thanks!

[hamishwillee]

Thanks. Would appreciate if you and @julianoes could keep me posted.

[Davidsastresas]

I will keep you posted. I recently talked with @rmackay9 for AP dev team to be aware of this. Thanks!

[rmackay9]

Hi, yes, it seems like a good idea and I agree that the message or command could be much like the MAV_CMD_DO_GIMBAL_MANAGER_CONFIGURE command. I think I also prefer a command over a new message although I don't feel really strongly about it.

I also agree with the idea that it's not meant to be a super strict security feature.

In AP we already check that the MYGCS_SYSID parameter matches the message/command trying to control the vehicle via RC_OVERRIDE, MANUAL_OVERRIDE (I think) and mavlink commands so I suspect extending this would not be too hard (but who knows for sure).

Anyway, we've had requests from users over the years about multi-gcs control so there is interest out there and AP would be happy to implement this.

[rmackay9]

Ah, I forgot to mention that I think we should separate out the issue of multiple GCSs using the same sysid. That's a misconfiguration so in the short-term at least we could "handle it through documentation" as we say.

[hamishwillee]

Ah, I forgot to mention that I think we should separate out the issue of multiple GCSs using the same sysid. That's a misconfiguration so in the short-term at least we could "handle it through documentation" as we say.

Absolutely.

Anyway, we've had requests from users over the years about multi-gcs control so there is interest out there and AP would be happy to implement this.

Great. Let's see what the final design looks like.

[julianoes]

"handle it through documentation"

And try to add some sort of warnings to make the user aware of potential conflicts, if possible.

[Davidsastresas]

Great, so we are all on the same page for preparing a draft PR based on this discussion? The summary would be:

- MSG CONTROL_STATUS: Message broadcasted by the vehicle, by the "owner" of a system ID corresponding to a vehicle ( A payload should not broadcast it ). It could be a very low rate as long as the system sends it right away after changed. It would have the following fields:

• Sys id in control

- CHANGE_OPERATOR_CONTROL/ACK: Do we preserve these? or we use CMD instead? What about the passkey, we preserve it right? I personally like it, could be a parameter living in the autopilot or similar.

Thanks!

[julianoes]

That sounds reasonable. Do we need sysid+compid in control or sysid only? 🤔

I would probably suggest to use a proper command instead of having this bespoke message+ack combo.

[hamishwillee]

It depends on what the final proposal looks like - I've lost track.

I think the proposal is something like:

• a vehicle that is owned broadcasts the system id of its owner.
• A GCS can send the vehicle a command/message to say "I'd like to be the owner" (or possibly "I'd like this system ID to be the owner") and gets back an ack from the vehicle if ownership is granted.
• It isn't clear if the current owner can refuse to hand over or similar, if there is a fallback, if there is mediation between current owner, and what happens if the current owner disappears from the network, and for how long.
• It isn't clear what the semantics are for "no owner", or how we deal with the case where a companion is the overall owner.
• It isn't clear what a component that is not an owner should do if it gets the command? It needs to know that it should ignore commands from other items if it sees the ack. Or perhaps it should just monitor the emitted owner message to determine who the current owner of its system is, and whether it should accept commands.

If something can be owned, then you know its owner from the message, but you also know its source system and component ID from the message by default. So you can target to the message to that sys/comp id combo, and you should.

However I'm not sure what you do if you allow multiple components to be owned? Sure, two components with owners can emit the message, and you can change their owners, but it won't be clear to other components of the system how they determine their owner.
Upshot, this is much easier if you have one owner for a whole system.

But without the flow and use cases, not able to comment helpfully.

EDIT - I think this actually a case which might benefit from using the RFC process template to ensure that the options are explored.
Note also, FYI, that there are ArduPilot parameters that allow the owning SYSID to be stored, and another that sets whether the vehicle will ignore commands and command-like messages from any other GCS. Just something to keep in mind.

[Davidsastresas]

@hamishwillee, your points make sense to me, but as you say, some of the points are not clear. My take on them is as follows, adding in bold my thoughts:

• A vehicle that is owned broadcasts the system id of its owner. Lets call this one CONTROL_STATUS for the sake of the example
• A GCS can send the vehicle a command/message to say "I'd like to be the owner" (or possibly "I'd like this system ID to be the owner") and gets back an ack from the vehicle if ownership is granted. About this one, what would be the use case for a system requesting control on behalf of a system ID that isn't its own? I think it adds complexity and I am not sure if there is a strong use case to justify it
• It isn't clear if the current owner can refuse to hand over or similar, if there is a fallback, if there is mediation between current owner, and what happens if the current owner disappears from the network, and for how long. I think there is value on "asking" first the current owner. It would allow for nicer integration. The GCS in control could show a popup when another GCS requested control, to confirm or refuse. In the case of a companion computer being in control, it can be up to the implementation of the companion computer to accept automatically or not.
  
  In case nobody is in control ( autopilot stops receiving hearbeats from the current system on control ) we could indicate this somehow in CONTROL_STATUS, for instance using 0 as sysid in control, could indicate nobody in control.
  
  Then, the remaining GCS on the network could pick that nobody is in control, and request control automatically, or just notify the user the vehicle lost current owner. That could be something left configurable in GCS, up to the user
• It isn't clear what the semantics are for "no owner", or how we deal with the case where a companion is the overall owner.
  In that case, all the GCS would receive telemetry from vehicle, but would not be able to control, unless they request control first.
  
  For the case when just a companion computer is the owner, we could detect it without using compid. If CONTROL_STATUS comes from the same sysid it reports as being the owner, we know control is coming from within the system. Of course we could also extend "ownership" to sysid-compid pair, but I am not sure it is needed at this stage.
  
• It isn't clear what a component that is not an owner should do if it gets the command? It needs to know that it should ignore commands from other items if it sees the ack. Or perhaps it should just monitor the emitted owner message to determine who the current owner of its system is, and whether it should accept commands..
  I think the key is in monitoring CONTROL_STATUS to understand who is in control. If it is itself, it can be indicated on the UI. That UI indicator in GCS could only be shown if the system is seeing other heartbeats from items that could potentially control the vehicle ( basically GCS and companion computers, compids 190-196 )

I am happy to start preparing some diagrams and charts to keep the discussion, I am learning how they have been done for mavlink docs in mermaid ( very cool by the way! ) but I wanted to iterate just a couple of messages more to understand better if we are all on the same page on the base of it.

Overall, my thought is to keep it as simple as possible. If this works nicely it will certainly evolve. I think the key is to have something that works and covers the basic needs of multi GCS operations, so people start using it. If people start using it on real scenarios we could have real feedback on what else is needed, and extend the implementation. Although I am pretty new to engineering new mavlink features, so maybe my approach is not good and it is better to think more in the future and spend more time if needed in cover extra scenarios, be more "futureproof". I am of course open to your opinions on this front!

About Ardupilot, indeed, they have SYSID_MYGCS parameter, and it serves the purpose just fine, with that parameter and mav_cmd_user sent between GCS on the system we could really do this. In fact that is how this all started, planning multi GCS support for Ardupilot based on that parameter and custom commands. I brought the matter here because it would be cool if this isn't something that depends on an Ardupilot specific parameter and it is rather a protocol standard. I really think it is a feature worth having, multi GCS systems are not that strange nowadays.

[hamishwillee]

@Davidsastresas Thanks very much. All of that sounds good, so happy for you to progress with diagrams, messages etc. Here are some "thoughts" from reading this:

1. There is just controlled "brain" in the vehicle - we should not allow separate control of autopilot and companion - it doesn't make sense that one GCS might command a vehicle to do something and another should control the flight stack to do another.
2. The cases we might want separate control are of peripherals where you might need control via another person. I know that the gimbal has inbuilt mechanisms for this, but presumably a winch or similar might also want independent control?
   • In any case, I'd rather we had a simple API that allowed control by one GCS at a time than a flexible but complicated API that allows control of anything.
   • I suspect we can do this by having the CONTROL_STATUS emitted by a winch IFF it allows independent control. Otherwise components are assumed to be owned by the autopilot or companion - whichever emits CONTROL_STATUS.
3. Possibly simplest model is that:
   • Autopilot emits CONTROL_STATUS.
     • All other components in the system only accept commands from the system id (of GCS) specified in that message.
   • Any GCS can request transfer of ownership
     • this is always granted.
     • Other GCS that had ownership can display "ownership now changed."
   • If autopilot loses the GCS that has control status then it publishes no owner, Ownership can be requested an is always granted.
4. If we want to support multiple ownership of components is similar except for this point:
   • Autopilot emits CONTROL_STATUS.
     • All other components in the system only accept commands from the system id (of GCS) specified in that message.
     • Except an explicitly independently controlled component an also emit CONTROL_STATUS
       • You shouldn't have both a companion and an autopilot emitting CONTROL_STATUS
       • As above, no other components should look at CONTROL_STATUS except from the autopilot
5. If we want to ask for transfer of ownership not so sure. Maybe.
   • Request transfer request sent to autopilot (or other controlled component)
   • If autopilot has no owner: it just accepts the command and sets owner. The ACK means "command acknowledged". The GCS only assumes ownership once it sees CONTROL_STATUS with its ID sent by autopilot.
   • If autopilot has owner
     • it responds with ACK to indicate "command acknowledged".
     • it sends "request transfer" to owning GCS
       • if GCS does not acknowledge or responds with ACCEPTED then the autopilot changes the owner and emits new owner in CONTROL_STATUS. If the
       • Otherwise (NACK) ownership is not transferred.

Not entirely sure about last option because a requestor has to wait for a pretty open ended time to know whether they are going to get ownership - there is no refusal notification or "i'll get back to you". But on the plus side, at least the GCS has right of refusal. I'm not sure how important that is as a use case though - you might argue that in a trusted network the parties should be talking before taking control. So I lean towards "4".

• This could be done with a long running command to the autopilot but that wouldn't work on ArduPilot.
• Open to other ideas.

Anyway, looking forward to seeing what suggestions you have/compromises you think we should make. FYI @julianoes

[Davidsastresas]

Thanks @hamishwillee.

I agree with your points, and I agree we should forget for the moment on more "complex" ownership like companion computers. In the end of the day they can control the autopilot using mavlink commands ( guided mode in Ardupilot, I am not sure in Px4 but I think the concept works similarly ) so I also think we can consider autopilot-companion a single logic block.

I just have one question. Why do you say that we could not do that for Ardupilot?

In any case, I think we have discussed enough to start preparing those diagrams, and start shaping this. I will work on it on the next few days.

Thank you!

[hamishwillee]

@Davidsastresas AFAIK Ardupilot does not currently support the long running commands. I'm not 100% convinced of them myself, though I think the use case is valid.