RemoteID v2 and adding additional remote ID definitions #1865
Conversation
|
New messages in common.xml should be marked as WIP until there is a working/prototyped implementation. At that point we can bring it to the MAVLink call to discuss whether we're happy with the stability. With respect to the ACK message, "generally" if there is a requirement for a command/response we use the command protocol. Can you show me the message sequence you're expecting where you might need to send this ACK? What is the transponder / sender supposed to DO in this case? @friissoren I'm hoping you can confirm that you understand the use cases here and are happy with the implementation. |
|
Marked the messages as WIP.
Message sequence example 1:
Nothing, the auto pilot system can keep sending OPEN_DRONE_ID_MESSAGE_PACK packages. Message sequence example 2:
The auto pilot knows that it should use other OpenDrone ID packets types. In this example it would try the individual messages instead like OPEN_DRONE_ID_BASIC_ID, OPEN_DRONE_ID_LOCATION. Compatibility issues In general -in my opinion- the auto pilot system needs to know if the transponder has accepted Remote ID messages. Remote ID will become a mandatory technology, so if messages are not supported or received faulty by the transponder, Remote ID signals will be broadcast incompletely or not at all. If the command protocol is a better solution to this, we should use that. I will do some more reading on this. If you need additional examples, please let me know. |
|
Another example would be for the OPEN_DRONE_ID_CMD message. Not all transponders support all transmission modes. Most support only BLE ones as can be seen on the page for stand-alone devices. For instance, if the auto pilot sends message OPEN_DRONE_ID_CMD with MAV_ODID_TRANSMISSION_MODE_WLAN_BEACON as transmission mode, the ACK message (MAV_ODID_ACK_ERR_NOT_SUPPORTED) can be used by the transponder to indicate that it doesn't support WiFi transmissions as it only has BLE hardware. The auto pilot should then try other transmission modes. Mandatory transmission modes differ a bit between the regions. Reading a bit into MAVLink commands, I think OPEN_DRONE_ID_CMD should be a command and not defined as a message. |
|
I definitely see a need to be able to control the transmission type/frequency etc. That is a clear missing item in the current set of messages. Whether that should be via a message or command is not clear to me. I am not familiar enough with general MAVLink conventions to comment on that. I can see some value in the acknowledgement message as well. I think we just need to document further when/where it would be expected that this message is used and in what types of typical contexts (just like @BluemarkInnovations already started describing earlier in this thread). |
|
@lukasbrchl: does your team possibly have any comments/input on this? |
message_definitions/v1.0/common.xml
Outdated
| <field type="uint8_t" name="command" enum="MAV_ODID_COMMAND">Controls the transmission of the OpenDroneID device.</field> | ||
| <field type="uint8_t" name="transmission_mode" enum="MAV_ODID_TRANSMISSION_MODE">Configure the OpenDroneID transmission mode.</field> | ||
| <field type="uint16_t" name="transmission_period_ms" units="ms">Configures the OpenDroneID transmission period. Valid values are between 100 and 3000 ms.</field> | ||
| </message> |
There was a problem hiding this comment.
Could we add the following note at the end?
<!-- The message ids 12918 - 12919 are reserved for OpenDroneID. -->
At some point some control messages related to network remote id will need to be added and it would make sense to have things in the same place.
There was a problem hiding this comment.
I will add it in the next PR and see what happens. I will also 'claim' ID 12906 to 12914.
|
@BluemarkInnovations @friissoren commands encode up to seven parameter values as floats in a COMMAND_LONG or 5 floats and 2 ints in a COMMAND_INT. They are intended for sending commands - i.e. "do something", with this configuration information. That is not what you are doing, and your information is too large to fit in a command, so my suggestion is incorrect. However I still might not use the ACK approach for this because it would be wasteful on the link. Consider your example #1865 (comment) - you'll be sent that ACK with every response when you knew after the first time it was supported. Some other options:
Thoughts? I don't want to tell you what particularly to do. It's just the the ACK changes the semantics from "we just sent out what we like" to "we have to understand what you support". That is a very different problem. |
|
Thanks all for the feedback, very useful. I will go back to the drawing board and propose new message definitions. This will likely be early next week. |
Thanks. Note you don't have to propose new messages if you end up working out these are the best compromise. I just want to make sure we do think about this stuff. |
|
With this new PR, I have updated the definitions. I think it is now a better proposal that makes better use of the existing MAVLink commands and reduces MAVLink communication. I dropped the ACK message definition and added the WLAN channel in the CMD message. I see two use cases/modes of the remote ID messages:
A transponder should support one or both modes. legacy/dumb modeThis modes uses the existing remote ID message definitions: OPEN_DRONE_ID_BASIC_ID, OPEN_DRONE_ID_LOCATION, OPEN_DRONE_ID_SELF_ID,OPEN_DRONE_ID_SYSTEM and OPEN_DRONE_ID_OPERATOR_ID. It is only one way communication from auto pilot system to the transponder. The workflow is as follows: each second: every 10 seconds (or other interval): Broadcast settings are defined by the firmware of the transponder. In current PR for ArduPilot this legacy/dumb mode is implemented. In this mode also the newly OPEN_DRONE_ID_CMD message can be used, but the auto pilot system has no knowledge if this command is fully accepted by the transponder. enhanced/intelligent modeIn this mode, the transponder has an active role and will ask the auto pilot system for information, by using the MAV_CMD_REQUEST_MESSAGE / MAV_CMD_SET_MESSAGE_INTERVAL commands. The workflow is as follows: startup:
Note 1: In case the transponder restarts, temporarily power loss etc, it will request this information again (start at point 1). |
|
In the above explanation comment, I changed transmit to broadcast. Also I redefined point 6: original it was Req Param 4 was the maximum allowed broadcast (transmission) protocol. In real life this is limited already by the standard to maximum of 3000 ms. So it is not useful. It is replaced by sending information about the WLAN radio capabilities (together with Req Param 5) of the transponder. This was needed as I allowed a primary and secondary WLAN channel for broadcasting. |
|
@BluemarkInnovations As I said before there are many ways to do this. I think this direction is pretty good, but others might want to consider others. Here are my comments: To paraphrase the enhanced proposal (please confirm, to test my understanding), the transponder drives the interactions rather than the autopilot just emitting what it supports:
Some questions / notes.
Now to your questions.
The TLDR; answer is "for all cases it is highly recommended" Essentially we use enums to maintain compatibility forever. If there is any possibility of the values on the ends of those links changing then you really have to have enums. It is also more clear for users since the enums are cross-linked. The cases where I'd say people don't need to is when there are only one or 2 possible values.
This is the case where you would use param 1 for the basic ID that you want. "index ID" here can mean "instance id". I think we meant instance when we wrote this. If it were me I would specify in your OPEN_DRONE_ID_BASIC_ID that an instance/index value of 0 means "send all", and use 1, 2, 3 for specific cases.
That's cool. If the authentication messages have instances you can use the same approach as in my comment above - 0 for all, or id for specific instance. This is quite flexible because it means you can choose to re-request an intermediate instance if you note one is missing.
This is not needed. You will get an ACK back with I'm not sure what you mean by use the legacy/dumb option. The legacy option is managed by the autopilot - it just sends everything at whatever rate it wants. As in one of the notes above, we need to decide what we do in that case.
We should refer to this as OpenID protocol v2 and define what happens if the different protocol versions for transponder/autopilot are used with each other. For example, current ArduPilot version will presumably always be emitting the legacy messages because it only supports v1 (though maybe you have some param to tell it that a transponder is connected?). Anyway, if this autopilot version is connected to a v1 transponder nothing changes. If it is connected to a v2 transponder we might say "the transponder should set the rates of the autopilot appropriately usiing the request and interval messages". Sorry, if I am belaboring the point - trying to be exhaustive responding to your points. |
|
@hamishwillee I had yesterday a meeting with @friissoren to discuss this proposal and made some changes. The reasons were that some scenarios were missing, but also that it will likely not be supported by ArduPilot and other systems that use MAVLink. (Because you cannot include parameters to the MAV_CMD_REQUEST_MESSAGE except for the message ID. ) So, the new proposal (57f3b90) is both supported by Søren and myself (Roel). I will make a new description below this comment.
Yes.
In an updated PR this message was changed to OPEN_DRONE_ID_BROADCAST_CONTROL.
Thanks, I will include that in the new description!
Good point. To my knowledge there are no implementations yet of the legacy approach. (Except for the BlueMark MAVLink transponder.) In the end you want to migrate to the new approach to unlock all features. However, the legacy approach may be sufficient in some scenarios. Let me describe first in the new comment how the newest approach looks like. Then we could decide how to handle it. For now, myy suggestion would be to keep both approaches supported.
Yes, the transponder is a dumb device and does not store data.
Your idea about param1 is wrong. I looked it up in the ArduPilot source code, param1 is the message ID. And other parameters (like param2) are not supported. This is one of the reasons for a new approach. The MAV_CMD_REQUEST_MESSAGE implementation in AruPilot can be found here. The function handle_command_request_message is called when such a packet arrives. In this function only the message id is decoded (param1).
Good point.
In the updated PR there are ENUMs.
This approach won't work as param1 is the message ID (see above).
This approach won't work as param1 is the message ID (see above).
Okay, good point.
Hopefully in the new description I cover this sufficiently.
Okay.
Thanks for you long response, it will get the PR better! |
|
The most important changes in the newest PR (57f3b90) are:
|
|
@BluemarkInnovations : In your most recent commit, you have somehow managed to modify every single line in common.xml. When I source compare the two versions, I don't see any differences in most of the lines though. I think what might have happened is that you have added/removed some CR/LF characters at the end of each line. Could you please try to fix this so only the lines that actually needs to be changed/added are included in the commit? |
|
There are two cases/modes how remote ID messages can be exchanged between MAVLink devices, such as an auto pilot system and a remote ID transmitter. Note, in case of a remote ID transmitter, it is assumed that it a dumb device. It does not store any data, but receives everything from the auto pilot system. (There is one exception, the transmitter may store a unique serial number and send a separate Basic ID message with this number.) These cases are:
The reason for a capabilities message is that a remote ID device can:
If for instance the auto pilot system wants to control the radio transmission, it needs to have the above information. Each remote ID system (auto pilot, transmitter, receiver) should support at least remote ID v1. There, are however enough incentives to support (part of) remote ID v2 too. remote ID v1transmitter The workflow is as follows: each second: every 10 seconds (or other interval): Optionally a transmitter can use the Transmitter settings (mode, interval etc) are defined by the firmware of the transmitter. In current ArduPilot PR this remote ID v1 mode is implemented. In this remote ID v1 mode, also the newly
receiver The workflow is as follows: Every time the receiver receives a remote ID signal, it will send a MAVLink message to the auto pilot. If the remote ID signal includes multiple packets, it will send for each information block (operator ID, system, self ID, location, authentication) a separate MAVLink message. Receiver -> Auto pilot: remote ID MAVLink message. In this remote ID v1 mode, also the newly
transceiver The remote ID v2 description will be in a new comment. |
Should be fixed in 57f3b90. |
remote ID v2In this v2 approach, the OpenDroneID device will send If the auto pilot system does not receive transmitter The workflow is as follows:
receiver So remote ID v2 means for a receiver that the auto pilot system and receiver support the transceiver |
|
Hello, Our current approach So our current approach to support MAVLink Controllers is to interface with the controller and read the Telemetry data available from the controller actively(Subscribing to interval MAVLink messages such as
This is by no means an ideal approach to this problem, however, it is for now the easiest to implement(And probably not even the correct given the nature of some messages used by us). I wanted to add this approach to the mix as it was not mentioned here. And for devices, which have the option to work in the stand-alone mode with separate applications and other features this might be the preferred way of doing things. There might be an option to replace the messages for the ODID ones, which would lead to a prettier implementation. Future development We are looking into implementing the "remote ID v1 or the next one" solution. We are looking from the point of mandatory ODID transmission and that all of the ODID information are provided by the Flight controller itself. I have a few remarks on this from the developer if tranciever point of view. The messages should be sent only when a change in the data occurs, which can be driven by the interval or by the data itself. The transmitter is trying to send always the newest data possible. The only live message in the ODID protocol is the
The easiest way to change the implementation is by the transmitter to retrieve on startup/MAVLink connection or parse then on change. For proper integration of the receiver into the system should report using the HEARTBEAT message and prevent the system from operating until it has all necessary information(either they are polled or received by interval sending) for correct ODID operation. To add something to progress the implementation discussion indeed there should be some Backward compatibility would be assured by receiving the TLDR: Current implementation of ODID messages should be extended for the
As for the receiver we don't have any major remarks on that. It should poll the |
|
@BluemarkInnovations Thanks for clarification about MAV_CMD_REQUEST_MESSAGE support in ArduPilot. I haven't read the rest of this yet, but you might want to talk to @peterbarker about getting the support updated to match the current spec so you could choose to use this. I'll look at the new proposal now. |
message_definitions/v1.0/common.xml
Outdated
| @@ -4595,7 +4695,7 @@ | |||
| </description> | |||
| </entry> | |||
| <entry value="2" name="NAV_VTOL_LAND_OPTIONS_HOVER_DESCENT"> | |||
| <description>Land in multicopter mode on reaching the landing co-ordinates (the whole landing is by "hover descent").</description> | |||
| <description>Land in multicopter mode on reaching the landing coordinates (the whole landing is by "hover descent").</description> | |||
There was a problem hiding this comment.
FYI These typo fixes are all great, but should have gone in a separate PR. OK here now.
There was a problem hiding this comment.
I'm not so familiar with git. So I forgot to rebase the code prior to pushing this PR. Hence,changes from other PRs got included as well. Hopefully, it is solved now with the latest commits. If not, please let me know.
|
@BluemarkInnovations Thanks very much. I quite like the symmetry of the design. FYI only V2 description might be easier to parse if it was written as though v1 did not exist. I think it looks like this (you can click the image to be taken to an editor for this). So essentially most of this is pretty simple in v2:
My concerns:
|
|
@optical-o I'm going to leave @BluemarkInnovations to comment broadly on your response. A few minor points.
Yes, we get that. The version 1 is an earlier implementation that did not understand this approach, so it streams the static data at low rate to ensure that it is received if, say, a transponder is connected late. I want v2 to replace v1 - not co-exist forever. The concern in V2 is that we have things like the OPEN_DRONE_ID_RADIO_CONTROL which might change on the vehicle. If we send them once then we don't know for sure they were received unless (ish) we stream them. Though this could be a command too :-)
It is more than just the radio control/capabilities. It is a switch from streamed static data to requested data. From a MAVLink point of view we need to identify the different workflows for implementers. If this was all agreed we'd have two separate protocols, with v2 superseding v1 Getting the data needed from other messages is certainly a valid approach, albeit requires much more knowledge of MAVLink and systems than most implementers are likely to want. |
|
I think I'm caught up up and the preceding sounds reasonable enough for "the next steps". Sorry to diverge a bit from the work above. However, we do not yet have a message that goes from the DroneID module back to the autopilot to authorize arming or say "I'm not ready". I do not see how a device that only receives messages can meet the FAA''s "Standard RID" regulatory requirements. How do we know the transmitter is sending data correctly, receiving data correctly, passed preflight tests, etc.? Say The discussion for that message should likely be separate as I see that message as a required item within this month. |
|
@optical-o, thanks for the feedback and sharing your current MAVLink implementation. It is good see more manufacturers that want to support the MAVLink remote ID messages. And also share their opinion about this PR. @hamishwillee already explained why it is called v2 and v1. Basically you describe using the remote ID v1 message definitions outside the intended usage.
The
I think you mean transmitter instead of receiver. (So the transponder that receives data from the auto pilot system.) I think a message type like mentioned here by @hendjoshsr71 would be useful.
There are different regulations and it is up to the auto pilot system. I can imagine that when the motors are off, you also don't send any remote ID signals. Basically also the auto pilot system needs to implement the remote ID standard correctly. This is a bit out of the scope of this MAVLink message definitions.
Indeed, it could well be that most transmission parameters are set statically in the system. However, the capabilities message also returns which packets the transmitter (transponder) supports. This is useful, unless of course there is a strict definition of remote ID in MAVLink that does not allow room for different implementations. |
|
@hamishwillee Here a quick reply. Next week more extensive one.
The auto pilot system could send a
It should be the first message. W.r.t. to the
Could be both. Depends on the auto pilot implementation. |
…o control message
F3411 does not define add-on modules at all. ASD-STAN specifically specify an Add-on as: "A standalone Direct Remote ID broadcast device integrating a GNSS function and a communication function, being able to provide position, height, speed over ground, track clockwise with true north, of the UA, and the remote pilot position or it’s take-off position." Since the proposed MAVLink messages here are clearly targeted at communication between the flight controller/GCS and the transmitter module, I would say that when these are used, the transmitter device is considered integrated into the UAS and thus need to follow the rule-set adhering to Standard RID/Direct Remote ID. I.e. the serial number must be the serial number of the UA, the operator location must be dynamic (updated continuously), any faults of the transmitter device must be detected/communicated via the flight controller/GCS etc. |
BluemarkInnovations
force-pushed
the
remoteID_additions
branch
from
d419260
to
d1e7e9c
Compare
this is required for the pre-arm check that the transmitter is ready and passing tests message ID chosen so as not to conflict with PR mavlink#1865
|
For PX4, we started with an implementation of what would be described here as the "legacy" protocol. PX4/PX4-Autopilot#20036 |
|
@BluemarkInnovations I believe you have addressed my concerns iabout v1/v2 control in #1865 (comment) - The radio control bit won't quite work: This bit is good. The autopilot sets RADIO_CONTROL at any time in a command, and gets back and ACK. It knows it has been set, and it does after the capabilities are set But you cant request a command in the request message. It just won't work - you can only request a message. This is a little odd because normally an autopilot acts as a server answering queries from a GCS. So if we ever need to have a GCS interaction to set this radio info, things might get complicated. |
|
@hamishwillee, thanks for your feedback. The last message flow would work from my perspective, copied at the bottom of this comment.
Not sure what you mean with this, perhaps it is lost in translation. Is the message flow below awkward to implement for PX4/ArduPilot? Given the latest developments at ArduPilot and PX4, I'm less likely to implement remote ID v2 myself, or at least follow what these projects will do. I started this PR as in my opinion remote ID v1 has shortcomings and there wasn't much activity in this area when I opened this PR. @tridge, @ThomasDebrunner could you please give your opinion about this PR; does the proposed v2 work for you to implement? If not, what you want to have changed? |
|
@BluemarkInnovations Yes, that protocol looks fine. For now, we just send out the LOCATION and DRONE_ID_BASIC_ID messages automatically, without having to configure them. Also, there needs to be a mechanism that the Autopilot knows that the transponder is still working. E.g. this message. #1873 |
|
@BluemarkInnovations Yes, that message is in. Though it is slightly redundant as the transmitter emits a heartbeat. @friissoren We should do a last review of this. An push it in for prototyping. Also document as a v2 protocol and reframe the original doc as a v1 protocol. |
| <field type="uint8_t" name="basic_id_max">Describes how many different basic IDs the device supports. The valid range is 1 to 4.</field> | ||
| <field type="uint8_t" name="authentication_pages_max">Describes how many Authentication pages the device supports. Maximum value is 16.</field> | ||
| </message> | ||
| <!-- The message ids 12918 - 12919 are reserved for OpenDroneID. --> |
There was a problem hiding this comment.
This should be changed to only 12919 when rebasing this on top of the latest master. 12918 was already merged in.
| <field type="uint8_t" name="radio" enum="MAV_ODID_RADIO_TYPES" display="bitmask">Describes which radio types the device has (Bluetooth, WiFi).</field> | ||
| <field type="uint8_t" name="radio_wifi_frequency_bands" enum="MAV_ODID_RADIO_WIFI_SUPPORTED_FREQUENCY_BANDS" display="bitmask">Describes supported frequency bands of the WiFi radio(s).</field> | ||
| <field type="uint8_t" name="radio_wifi_count">Describes how many (independent) WiFi radio(s) the device has.</field> | ||
| <field type="uint8_t" name="commands_supported" enum="MAV_ODID_MESSAGES_SUPPORTED" display="bitmask">Describes which OpenDroneID messages the device supports.</field> |
There was a problem hiding this comment.
Should we make this uint16_t instead? Right now the enum exactly fits in the 8 bits allocated.
However, there is a theoretical possibility that the standards will be amended with additional message and it then won't be possible to add additional bitmask enum values to indicate those.
There was a problem hiding this comment.
yes, it would make it future proof.
Well the heartbeat has a bit of functionality as the transmitter is a critical component in the system. If it would fail there are no heartbeats anymore. However, we could modify the v2 protocol a bit and replace the heartbeat message by the OPEN_DRONE_ID_ARM_STATUS message. That one is also streamed. |
@BluemarkInnovations Sorry All I am saying is that I don't understand how the settings that the autopilot provides to the transponder will be populated in the autopilot. Normally you would write settings from a GCS. It may be that these are hard coded by an integrator on a per-device level. Upshot, I don't know if there is a "GCS intergration" required. What is it that concerns you about PX4/ArduPilot work that concerns you. I hope you'll implement this - right now I see you as the key stakeholder :-0. |
|
@ThomasDebrunner @tridge Could you please review the messages and workflow described in #1865 (comment)
|
When I started this PR, I was focused on Europe. Here, it is mandatory after Jan 1st 2024. So, enough time to start PRs like this one and slowly walking towards a solution and implementation. This has changed since a few weeks, as the "world" seems to get awakened; ArduPilot and PX4 rush to get remote ID integrated due to the September deadline in USA. This changes the whole process a bit and yes, of course, I will try to get it implemented. Due to this strict deadline, I find it important to get consensus with PX4/ArduPilot and avoid discussions afterwards which could result in the need for v3 or end up with different implementations that may cause incompatibility issues with the remote ID hardware. In my opinion v1, has shortcomings that need to be tackled and addressed here in this PR. You are right, that I should not (over)complicate things, and just move on with my effort getting this done. |
|
see my comments on #1880 for why I think we should not move fwd with the v2 proposal at this time |
hamishwillee
changed the title
|
@BluemarkInnovations Thanks very much. Given all the churn this week I think we should pause this discussion for a little while and wait until the dust settles. Could we perhaps re-engage on this early next year? |
Okay for me. |
At the moment I'm working both on integrating Remote ID MAVLink support in ArduPilot and releasing a Remote ID MAVLink capable transponder.
An outcome of this process is that -in my opinion - the current set of MAVLink message definitions are incomplete. Next, I have created an issue at the OpenDrone ID project to discuss new message definitions with the maintainers. That is now complete and in this PR I'm submitting two additional message definitions: