Merge pull request #47 from UAVCAN/v1-standard-data-types

[WIP] Initial draft proposal of the new standard data type set for UAVCAN v1.0

README.md

@@ -2,7 +2,6 @@ Standard DSDL definitions
 =========================
 
 [![Build Status](https://travis-ci.org/UAVCAN/dsdl.svg?branch=master)](https://travis-ci.org/UAVCAN/dsdl)
-[![Gitter](https://img.shields.io/badge/gitter-join%20chat-green.svg)](https://gitter.im/UAVCAN/general)
 
 This repository contains the DSDL definitions of the standard UAVCAN messages and services.
 It is intended for inclusion as a submodule into implementations of the UAVCAN protocol stack.
@@ -11,4 +10,140 @@ UAVCAN is an open lightweight protocol designed for reliable communication in ae
 robust vehicle bus networks.
 
 * [**UAVCAN website**](http://uavcan.org)
-* [**UAVCAN mailing list**](https://groups.google.com/forum/#!forum/uavcan)
+* [**UAVCAN forum**](https://forum.uavcan.org)
+
+## Identifier ranges
+
+### Subjects
+
+For message subjects, the following range mapping is adopted (limits inclusive).
+Unused ranges are reserved for future expansion of adjacent ranges.
+
+From    | To        | Purpose
+--------|-----------|------------------------------------------------
+0       | 32767     | Application-specific unregulated identifiers (freely chosen by the integrator)
+57344   | 59391     | Vendor-specific fixed identifiers (stored in a public repository)
+62804   | 65535     | Standard fixed identifiers
+
+### Services
+
+For services, the following range mapping is adopted (limits inclusive).
+Unused ranges are reserved for future expansion of adjacent ranges.
+
+From    | To        | Purpose
+--------|-----------|------------------------------------------------
+0       | 127       | Application-specific unregulated identifiers (freely chosen by the integrator)
+256     | 319       | Vendor-specific fixed identifiers (stored in a public repository)
+384     | 511       | Standard fixed identifiers
+
+## Standard static identifier allocation
+
+### Subjects
+
+Ordered by priority from high to low.
+
+Namespace                   | Lower bound (inclusive)
+----------------------------|-------------------------
+`uavcan.time`               | 62804
+`uavcan.node`               | 62805
+`uavcan.pnp`                | 62810/65533
+`uavcan.internet`           | 65510
+`uavcan.diagnostic`         | 65520
+
+### Services
+
+Ordered by priority from high to low.
+
+Namespace                   | Lower bound (inclusive)
+----------------------------|-------------------------
+`uavcan.register`           | 384
+`uavcan.pnp`                | 390
+`uavcan.file`               | 400
+`uavcan.node`               | 430
+`uavcan.internet`           | 500
+
+## Generic data type definitions
+
+### SI
+
+The namespace `uavcan.si` contains a collection of generic data types describing commonly used
+physical quantities.
+
+Some of the messages are time stamped, in which case the time stamp is always situated at the end in order
+to facilitate binary compatibility with non-timestamped messages (both time-stamped and non-time-stamped
+messages have identical headers, so if the time stamp is not required they can be used interchangeably).
+Names of the time stamped messages have the suffux `TS`.
+
+All units follow the [International System of Units](https://en.wikipedia.org/wiki/International_System_of_Units).
+All units are unscaled basic units of measure (e.g., meters rather than kilometers, kilograms rather than milligrams),
+unless a different multiplier is explicitly specified in the definition (e.g., nanosecond).
+
+All coordinate systems are right-handed.
+In relation to body, the preferred standard is as follows: **X** -- forward, **Y** -- right, **Z** -- down.
+In case of cameras, the following convention should be preferred: **Z** -- forward, **X** -- right, **Y** -- down.
+For world frames, the North-East-Down (NED) notation should be preferred.
+
+Rotation and angular velocities are represented in fixed-axis roll (about X), pitch (about Y), and yaw (about Z).
+Quaternions and other redundant representations are intentionally avoided due to bandwidth and latency concerns;
+should they ever be used, the following element ordering should be adopted: W, X, Y, Z.
+
+### Primitives
+
+A collection of primitive data types is intended as a very generic solution for odd use cases
+and prototyping. They permit the user to broadcast a completely arbitrary value via the bus
+while not having to deal with custom data type design and distribution.
+
+Since these types lack any semantic information, their usage in production environments is discouraged.
+
+Another important application of these types is in the schemaless register protocol defined
+in the namespace `uavcan.register`.
+
+### Registers
+
+The register protocol provides a highly generic interface to vendor-specific functionality
+and configuration parameters via named registers.
+
+## Guidelines for data type authors
+
+In order to maximize compatibility with resource-constrained nodes,
+standard messages should not be larger than 366 bytes when serialized.
+The number is dictated by the size of the largest data structure, which is the response part of the service
+`uavcan.node.GetInfo`.
+
+Follow the naming conventions defined in the specification.
+
+Every data type definition must have a header comment, where the first and the last lines are empty;
+every field must be preceded by a comment, unless it is absolutely certain that it is completely
+self-explanatory.
+An exception is made for highly generic definitions which often do not require any additional comments.
+
+Attributes must be separated by exactly one blank line, excepting tightly related attributes and
+void fields used for pre-alignment (e.g., before dynamic arrays), in which case blank lines are not necessary.
+More than one blank line is never allowed.
+There must be exactly one blank line at the end of the file.
+
+The lines of text must not be longer than 120 characters.
+
+Here is an example:
+
+    #
+    # This is a header comment.
+    # It explains what this data type definition is for and why do we need it.
+    # Mind the empty lines before and after the header comment.
+    #
+
+    # This space is reserved for future use.
+    void42
+
+    # This is an enumeration.
+    # We don't need blank lines because the items are tightly related.
+    uint8 VALUE_A = 1       # A comment describing the constant.
+    uint8 VALUE_B = 2       # Another one.
+    uint8 value
+
+    # This is a new field, mind the blank line above.
+    void1
+    float32[<100] aligned_array
+
+Remember, the set of standard data types is an important part of the protocol specification,
+so the quality of the documentation is very important.

uavcan/internet/README.md

@@ -0,0 +1,16 @@
+Internet/LAN forwarding protocol
+================================
+
+Data types defined in this namespace can be used to establish direct connectivity between arbitrary
+local UAVCAN nodes and hosts on the Internet or a local area network (LAN) by means of so called
+*modem nodes*.
+Normally, a modem node would be implemented on an on-board cellular, RF, or satellite communication
+hardware, such as an on-board telemetry modem.
+
+## Application examples
+
+* Direct telemetry transmission from UAVCAN nodes via simple modem nodes.
+* Creation of web API for on board equipment. For example, a camera vendor can provide a web API
+for their products that directly interacts with the cameras on board, bypassing all third party logic.
+* Reception of real time correction data streams (e.g. RTCM RC-104) for precise positioning applications.
+* Automatic firmware upgrades directly from the equipment vendor's website.

uavcan/internet/udp/500.HandleIncomingPacket.0.1.uavcan

@@ -0,0 +1,48 @@
+#
+# This message carries UDP packets sent from a remote host on the Internet or a LAN to a node on the local UAVCAN bus.
+# Please refer to the definition of the message type OutgoingPacket for a general overview of the packet forwarding
+# logic.
+#
+# This data type has been made a service type rather than a message type in order to make its transfers addressable,
+# allowing nodes to employ hardware acceptance filters for filtering out forwarded datagrams that are not addressed
+# to them. Additionally, requiring the destination nodes to always respond upon reception of the forwarded datagram
+# opens interesting opportunities for future extensions of the forwarding protocol.
+#
+# It should be noted that this data type definition intentionally leaves out the source address. This is done in
+# order to simplify the implementation, reduce the bus traffic overhead, and because the nature of the
+# communication patterns proposed by this set of messages does not provide a valid way to implement server hosts
+# on the local UAVCAN bus. It is assumed that local nodes can be only clients, and therefore, they will be able to
+# determine the address of the sender simply by mapping the field session_id to their internally maintained states.
+# Furthermore, it is uncertain what is the optimal way of representing the source address for
+# client nodes is: it is assumed that the local nodes will mostly use DNS names rather than IP addresses, so if there
+# was a source address field, modem nodes would have to perform reverse mapping from the IP address they received
+# the datagram from to the corresponding DNS name that was used by the local node with the outgoing message. This
+# approach creates a number of troubling corner cases and adds a fair amount of hidden complexities to the
+# implementation of modem nodes.
+#
+# It is recommended to perform service invocations at the same transfer priority level as was used for broadcasting
+# the latest matching message of type OutgoingPacket. However, meeting this recommendation would require the modem
+# node to implement additional logic, which may be undesirable. Therefore, implementers are free to deviate from
+# this recommendation and resort to a fixed priority level instead. In the case of a fixed priority level, it is
+# advised to use the lowest transfer priority level.
+#
+
+# This field must contain the same value that was used by the local node when sending the corresponding outgoing
+# packet using the message type OutgoingPacket. This value will be used by the local node to match the response
+# with its local context.
+uint16 session_id
+
+# Effective payload. This data will be forwarded from the remote host verbatim.
+# UDP packets that contain more than 508 bytes of payload may be dropped by some types of
+# communication equipment. Refer to RFC 791 and 2460 for an in-depth review.
+# UAVCAN further limits the maximum packet size to reduce the memory and traffic burden on the nodes.
+# Datagrams that exceed the capacity of this field should be discarded by the modem node.
+void7
+uint8[<=320] payload
+
+---
+
+# If the service invocation times out, the modem node is permitted to remove the corresponding entry from
+# the NAT table immediately, not waiting for its TTL to expire.
+
+void56      # Reserved for future use.

uavcan/internet/udp/65510.OutgoingPacket.0.1.uavcan

@@ -0,0 +1,133 @@
+#
+# This message carries UDP packets from a node on the local bus to a remote host on the Internet or a LAN.
+#
+# Any node can broadcast a message of this type.
+#
+# All nodes that are capable of communication with the Internet or a LAN should subscribe to messages
+# of this type and forward the payload to the indicated host and port using exactly one UDP datagram
+# per message (i.e. additional fragmentation is to be avoided). Such nodes will be referred to as
+# "modem nodes".
+#
+# It is expected that some systems will have more than one modem node available.
+# Each modem node is supposed to forward every message it sees, which will naturally create
+# some degree of modular redundancy and fault tolerance. The remote host should therefore be able to
+# properly handle possibly duplicated messages from different source addresses, in addition to
+# possible duplications introduced by the UDP/IP protocol itself. There are at least two obvious
+# strategies that can be employed by the remote host:
+#
+#   - Accept only the first message, ignore duplicates. This approach requires that the UDP stream
+#     should contain some metadata necessary for the remote host to determine the source and ordering
+#     of each received datum. This approach works best for periodic data, such as telemetry, where
+#     the sender does not expect any responses.
+#
+#   - Process all messages, including duplicates. This approach assumes that the remote host acts
+#     as a server, processing all received requests and providing responses to each. This arrangement
+#     implies that the client may receive duplicated responses. It is therefore the client's
+#     responsibility to resolve the possible ambiguity. An obvious solution is to accept the first
+#     arrived response and ignore the later ones.
+#
+# Applications are free to choose whatever redundancy management strategy works best for them.
+#
+# If the source node expects that the remote host will send some data back, it must explicitly notify
+# the modem nodes about this, so that they could prepare to perform reverse forwarding when the
+# expected data arrives from the remote host. The technique of reverse forwarding is known in
+# networking as IP Masquerading, or (in general) Network Address Translation (NAT). The notification
+# is performed by means of setting one of the corresponding flags defined below.
+#
+# In order to be able to match datagrams received from remote hosts and the local nodes they should
+# be forwarded to, modem nodes are required to keep certain metadata about outgoing datagrams. Such
+# metadata is stored in a data structure referred to as "NAT table", where every entry would normally
+# contain at least the following fields:
+#   - The local UDP port number that was used to send the outgoing datagram from.
+#     Per RFC 4787, the port number is chosen by the modem node automatically.
+#   - The node ID of the local node that has sent the outgoing datagram.
+#   - Value of the field session_id defined below.
+#   - Possibly some other data, depending on the implementation.
+#
+# The modem nodes are required to keep each NAT table entry for at least NAT_ENTRY_MIN_TTL seconds
+# since the last reverse forwarding action was performed. Should the memory resources of the modem node
+# be exhausted, it is allowed to remove old NAT entries earlier, following the policy of least recent use.
+#
+# Having received a UDP packet from a remote host, the modem node would check the NAT table in order
+# to determine where on the UAVCAN bus the received data should be forwarded to. If the NAT table
+# contains no matches, the received data should be silently dropped. If a match is found, the
+# modem node will forward the data to the recipient node using the service HandleIncomingPacket.
+# If the service invocation times out, the modem node is permitted to remove the corresponding entry from
+# the NAT table immediately (but it is not required). This will ensure that the modem nodes will not be
+# tasked with translations for client nodes that are no longer online or are unreachable.
+# Additionally, client nodes will be able to hint the modem nodes to remove translation entries they no
+# longer need by simply refusing to respond to the corresponding service invocation. Please refer to
+# the definition of that service data type for a more in-depth review of the reverse forwarding process.
+#
+# Modem nodes can also perform traffic shaping, if needed, by means of delaying or dropping UDP
+# datagrams that exceed the quota.
+#
+# To summarize, a typical data exchange occurrence should amount to the following actions:
+#
+#   - A local UAVCAN node broadcasts a message of type OutgoingPacket with the payload it needs
+#     to forward. If the node expects the remote host to send any data back, it sets the masquerading flag.
+#
+#   - Every modem node on the bus receives the message and performs the following actions:
+#
+#       - The domain name is resolved, unless the destination address provided in the message
+#         is already an IP address, in which case this step should be skipped.
+#
+#       - The domain name to IP address mapping is added to the local DNS cache, although this
+#         part is entirely implementation defined and is not required.
+#
+#       - The masquerading flag is checked. If it is set, a new entry is added to the NAT table.
+#         If such entry already existed, its expiration timeout is reset. If no such entry existed
+#         and a new one cannot be added because of memory limitations, the least recently used
+#         (i.e. oldest) entry of the NAT table is replaced with the new one.
+#
+#       - The payload is forwarded to the determined IP address.
+#
+#   - At this point, direct forwarding is complete. Should any of the modem nodes receive an incoming
+#     packet, they would attempt to perform a reverse forwarding according to the above provided algorithm.
+#
+# It is recommended to use the lowest transport priority level when broadcasting messages of this type,
+# in order to avoid interference with a real-time traffic on the bus. Usage of higher priority levels is
+# unlikely to be practical because the latency and throughput limitations introduced by the on-board radio
+# communication equipment are likely to vastly exceed those of the local CAN bus.
+#
+
+# Modem nodes are required to keep the NAT table entries alive for at least this amount of time, unless the
+# table is overflowed, in which case they are allowed to remove least recently used entries in favor of
+# newer ones. Modem nodes are required to be able to accommodate at least 100 entries in the NAT table.
+uint32 NAT_ENTRY_MIN_TTL = 86400        # [second]
+
+# This field is set to an arbitrary value by the transmitting node in order to be able to match the response
+# with the locally kept context. The function of this field is virtually identical to that of UDP/IP port
+# numbers. This value can be set to zero safely if the sending node does not have multiple contexts to
+# distinguish between.
+uint16 session_id
+
+# UDP destination port number.
+uint16 destination_port
+
+# Option flags.
+bool use_masquerading   # Expect data back (i.e. instruct the modem to use the NAT table)
+void7
+
+# Domain name or IP address where the payload should be forwarded to.
+# Note that broadcast addresses are allowed here, for example, 255.255.255.255.
+# Broadcasting with masquerading enabled works the same way as unicasting with masquerading enabled: the modem
+# node should take care to channel all traffic arriving at the opened port from any source to the node that
+# requested masquerading.
+# The full domain name length may not exceed 253 octets, according to the DNS specification.
+# UAVCAN imposes a stricter length limit in order to reduce the memory and traffic burden on the bus: 45 characters.
+# 45 characters is the amount of space that is required to represent the longest possible form of an IPv6 address
+# (an IPv4-mapped IPv6 address). Examples:
+#   "forum.uavcan.org"                              - domain name
+#   "192.168.1.1"                                   - IPv4 address
+#   "2001:0db8:85a3:0000:0000:8a2e:0370:7334"       - IPv6 address, full form
+#   "2001:db8:85a3::8a2e:370:7334"                  - IPv6 address, same as above, short form (preferred)
+#   "ABCD:ABCD:ABCD:ABCD:ABCD:ABCD:192.168.158.190" - IPv4-mapped IPv6, full form (length limit, 45 characters)
+void2
+uint8[<=45] destination_address
+
+# Effective payload. This data will be forwarded to the remote host verbatim.
+# UDP packets that contain more than 508 bytes of payload may be dropped by some types of
+# communication equipment. Refer to RFC 791 and 2460 for an in-depth review.
+# UAVCAN further limits the maximum packet size to reduce the memory and traffic burden on the nodes.
+uint8[<256] payload