Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Moving Cyphal/UDP to multicast #253

Merged
merged 170 commits into from Mar 12, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
170 commits
Select commit Hold shift + click to select a range
0c23a17
Added source_node_id to UDPFrame
maksimdrachov Oct 9, 2022
fd79260
small changes
maksimdrachov Oct 15, 2022
8ba6d30
Added source_node_id to UDPOutputSession
maksimdrachov Oct 15, 2022
f4ff256
Updated unit tests with source_node_id
maksimdrachov Oct 15, 2022
5fd67ba
Black style format
maksimdrachov Oct 15, 2022
c2ce7ca
Updated Message format
maksimdrachov Oct 16, 2022
d7b2865
assert error fix
maksimdrachov Oct 22, 2022
8ed75f0
Fix unit test v4
maksimdrachov Oct 22, 2022
e7e1e58
fix _unittest_socket_factory
maksimdrachov Oct 29, 2022
3267178
Updated Service (to multicast)
maksimdrachov Oct 30, 2022
298e34e
pycyphal/transport/udp/_ip ported to multicast
maksimdrachov Nov 6, 2022
a28ee19
pycyphal/transport/udp/_session using new datagram
maksimdrachov Nov 6, 2022
049c460
pycyphal/transport/udp port to multicast
maksimdrachov Nov 6, 2022
42526d4
pycyphal/transport/udp to multiport
maksimdrachov Nov 8, 2022
53e832f
unit test _unittest_socket_factory working
maksimdrachov Nov 18, 2022
d31eae1
doctests fixes
maksimdrachov Nov 19, 2022
e7349e6
_unittest_output_session fixes
maksimdrachov Nov 19, 2022
0ecfc85
small fix
maksimdrachov Nov 19, 2022
1a0245a
_socket_reader ported
maksimdrachov Nov 20, 2022
81078d2
DOMAIN_ID_MASK replaced with NODE_ID_MASK
maksimdrachov Nov 20, 2022
1a22adc
Black style format
maksimdrachov Nov 20, 2022
828ed45
small fix _socket_reader.py
maksimdrachov Nov 20, 2022
db1b4ec
fix _unittest_socket_reader
maksimdrachov Nov 20, 2022
962dbd1
_socket_reader port
maksimdrachov Nov 23, 2022
a79b727
added local_node_id to _socket_reader
maksimdrachov Nov 23, 2022
814bf61
source_node_id is None for anonymous
maksimdrachov Nov 23, 2022
d1dbecb
update _socket_reader unit tests
maksimdrachov Nov 23, 2022
1b6091b
anonymous frame processing
maksimdrachov Nov 25, 2022
ce194d2
service multicast issue
maksimdrachov Nov 27, 2022
5566313
domain -> subnet
maksimdrachov Nov 28, 2022
6bffe39
tests: domain -> subnet
maksimdrachov Nov 28, 2022
12dad59
small changes
maksimdrachov Nov 28, 2022
6463d3a
scott's frame, todo: testing
maksimdrachov Dec 7, 2022
0ac5695
_frame: testing done
maksimdrachov Dec 10, 2022
14e6c2b
service_multicast_specifier_to_multicast_group
maksimdrachov Dec 10, 2022
be62796
service_multicast_group_to_node_id
maksimdrachov Dec 10, 2022
e9789a4
_endpoint_mapping done
maksimdrachov Dec 10, 2022
7b6b08e
udp/_ip done
maksimdrachov Dec 10, 2022
cd028df
udp/_ip fixes
maksimdrachov Dec 11, 2022
c9b6027
_input: rough draft
maksimdrachov Dec 11, 2022
86eedbf
_unittest_socket_factory update
maksimdrachov Dec 13, 2022
6cf4224
_output_session unit test port
maksimdrachov Dec 13, 2022
e95edd1
changed CRC library
maksimdrachov Dec 14, 2022
47f692e
improved API _frame
maksimdrachov Dec 14, 2022
1ed9bb5
CRC call hanging
maksimdrachov Dec 14, 2022
391bf1c
fix CRC header
maksimdrachov Dec 19, 2022
599288c
_frame.py: parse() updated
maksimdrachov Dec 19, 2022
e0e6768
_unittest_udp_output_session working
maksimdrachov Dec 19, 2022
b29d665
removed unneccesary comment
maksimdrachov Dec 19, 2022
fd3a1df
_frame small fixes
maksimdrachov Dec 22, 2022
32852df
_output_session: small fix crc
maksimdrachov Dec 22, 2022
439b5fa
_input session wip
maksimdrachov Dec 22, 2022
d4db8b8
_input_session working
maksimdrachov Dec 26, 2022
36eabbb
Messy solution to None frames
maksimdrachov Dec 26, 2022
c115847
INVALID frame test
maksimdrachov Dec 26, 2022
862e3bb
Invalid HEADER_CRC
maksimdrachov Dec 26, 2022
64f6ab7
cleanup _input.py
maksimdrachov Dec 26, 2022
308766c
messy is not messy anymore
maksimdrachov Dec 26, 2022
ccb4f30
_transfer_reassembler: added crc for single-frame
maksimdrachov Dec 26, 2022
f498a08
added check crc for single-frame
maksimdrachov Dec 26, 2022
73c107a
_unittest_transfer_reassembler_anonymous
maksimdrachov Dec 29, 2022
ce2c053
TransferReassembler: single frame with CRC
maksimdrachov Dec 29, 2022
f138e2d
input_session almost working
maksimdrachov Jan 15, 2023
9f7dd87
DESTINATION_PORT -> CYPHAL_PORT
maksimdrachov Jan 16, 2023
bc8e02b
debugging process_frame
maksimdrachov Jan 16, 2023
b0e9097
working _input_session unit tests
maksimdrachov Jan 16, 2023
c87f13a
looping receive
maksimdrachov Jan 16, 2023
d9ab260
sniffer not sniffing
maksimdrachov Jan 17, 2023
da7aa20
sniffer working, without system_ns assert
maksimdrachov Jan 17, 2023
4b77a7e
small fix
maksimdrachov Jan 17, 2023
5d96ad0
fixing tracer
maksimdrachov Jan 18, 2023
00391dc
run-unit-tests.sh is working
maksimdrachov Jan 18, 2023
c664b75
MVP
maksimdrachov Jan 18, 2023
a4a7bf2
INTEGRITY_ERROR
maksimdrachov Jan 26, 2023
7f6ef17
_transfer_reassembler unit test fix
maksimdrachov Jan 27, 2023
619af69
fix udp doctest
maksimdrachov Jan 27, 2023
9788b1b
fix serialize_transfer
maksimdrachov Jan 27, 2023
29190ce
_unittest_serial_tracer fix
maksimdrachov Jan 27, 2023
e4d309e
remove payload_crc from _unittest_frame_parse
maksimdrachov Jan 27, 2023
fd0a5a6
serial _frame payload crc check removed
maksimdrachov Jan 27, 2023
7acbada
pycyphal/transport/serial tests working
maksimdrachov Jan 27, 2023
c50a196
tests/transport/serial working
maksimdrachov Jan 27, 2023
fe38b7f
pytest pycyphal/application passing
maksimdrachov Jan 29, 2023
5a870fb
pytest tests/ -k "not redundant" passing
maksimdrachov Jan 29, 2023
bd8652f
redundant issue
maksimdrachov Jan 30, 2023
2016030
Pavel's fix
maksimdrachov Feb 1, 2023
bc2a636
redundant debugging
maksimdrachov Feb 1, 2023
c62a8d3
cleanup udp/_session/_input
maksimdrachov Feb 1, 2023
ae94164
stopping shooting at foot
maksimdrachov Feb 1, 2023
fcf2f47
turning on UDP anymous (conftest.py)
maksimdrachov Feb 1, 2023
f028a9e
not creating loop in _input
maksimdrachov Feb 2, 2023
7a1347c
_reader_thread fixed
maksimdrachov Feb 18, 2023
a54164e
_unittest_slow_node fix
maksimdrachov Feb 19, 2023
8772895
removed debug prints
maksimdrachov Feb 24, 2023
feef9e0
_udp: anonymous node tested
maksimdrachov Feb 24, 2023
0fbccbe
improved debug message
maksimdrachov Mar 3, 2023
6d8a20c
removed run-unit-tests.sh
maksimdrachov Mar 3, 2023
8c511da
fix _unittest_slow_node
maksimdrachov Mar 3, 2023
fd98631
updated monkey fix asyncio_allow_event... doctest
maksimdrachov Mar 3, 2023
0a29df1
fix some, break some (local_node_id)
maksimdrachov Mar 3, 2023
7414778
new documentation
maksimdrachov Mar 4, 2023
4d5b310
fix annotations
maksimdrachov Mar 4, 2023
79f3382
fix _udp error handling
maksimdrachov Mar 4, 2023
c6b221f
added error case in get_output_session
maksimdrachov Mar 4, 2023
bead8bd
fixes sphinx errors
maksimdrachov Mar 4, 2023
ab4b03a
fix unit test udp
maksimdrachov Mar 4, 2023
9942c2b
fix timing issue demo app
maksimdrachov Mar 4, 2023
a45c92e
update ubuntu image
maksimdrachov Mar 4, 2023
1243f32
an experimental fix to the monkey patch
pavel-kirienko Mar 4, 2023
2b0c2be
Revert "update ubuntu image"
pavel-kirienko Mar 4, 2023
4de8a3d
An experimental change
pavel-kirienko Mar 4, 2023
d780537
Revert "An experimental change"
pavel-kirienko Mar 4, 2023
7b91183
Revert "an experimental fix to the monkey patch"
pavel-kirienko Mar 4, 2023
e3a56b7
Try this
pavel-kirienko Mar 4, 2023
e234a3b
Undo an incorrect fix
pavel-kirienko Mar 4, 2023
dabf3ae
Reduce the extent of the monkey patch
pavel-kirienko Mar 4, 2023
5b95b08
Handle session closure correctly in the ClientImpl
pavel-kirienko Mar 4, 2023
d4ddb7f
documentation fixes
maksimdrachov Mar 4, 2023
3b1d7f5
Remove unused functions
pavel-kirienko Mar 4, 2023
0075182
fix mypy errors (_frame)
maksimdrachov Mar 5, 2023
c4ba62d
fix mypy errors (_endpoint_mapping)
maksimdrachov Mar 5, 2023
4e44cf3
fix mypy errors (_input)
maksimdrachov Mar 5, 2023
df00579
fix mypy errors (_frame)
maksimdrachov Mar 5, 2023
7e73b37
fix mypy errors (_socket_factory)
maksimdrachov Mar 5, 2023
6a8bcbd
fix mypy errors (_v4)
maksimdrachov Mar 5, 2023
c0ce9d1
fix mypy errors (_output)
maksimdrachov Mar 5, 2023
dd1ad66
fix mypy errors (_udp)
maksimdrachov Mar 5, 2023
b94be47
fix mypy errors (tests)
maksimdrachov Mar 5, 2023
2efa4db
added UDPTransportStatistics
maksimdrachov Mar 5, 2023
da3082d
black code style format
maksimdrachov Mar 5, 2023
e547d39
added cy_addr_version
maksimdrachov Mar 5, 2023
ecb3e17
update documentation
maksimdrachov Mar 5, 2023
fcd32fd
fix doc test
maksimdrachov Mar 5, 2023
283ccc7
fixing happy accidents
maksimdrachov Mar 9, 2023
7b5cf42
changes to payload_crc (single frame)
maksimdrachov Mar 9, 2023
21bfe26
polymorphism joined the chat
maksimdrachov Mar 9, 2023
9b58d5a
black code style
maksimdrachov Mar 9, 2023
facc240
pylint stuff
maksimdrachov Mar 9, 2023
1106f04
fix pylint
maksimdrachov Mar 9, 2023
189d12f
fix failing serial test
maksimdrachov Mar 9, 2023
a2f3e9f
black format
maksimdrachov Mar 9, 2023
b829744
fix mypy
maksimdrachov Mar 10, 2023
43faa73
cleaning
maksimdrachov Mar 10, 2023
91ad4bb
mypy fix
maksimdrachov Mar 10, 2023
00d180a
removed sink
maksim-estonia Mar 11, 2023
8506573
_unittest_udp_socket_factory_v4 windows issue
maksim-estonia Mar 11, 2023
3e33eff
tests/_input_session windows issue
maksim-estonia Mar 11, 2023
6620fd4
import sys
maksim-estonia Mar 11, 2023
66c4ca3
Revert "removed sink"
maksim-estonia Mar 11, 2023
20562e0
unittest_sniff removed sink rx
maksim-estonia Mar 11, 2023
22bfe61
pylint
maksim-estonia Mar 11, 2023
48646b9
Update tests/transport/redundant/_redundant.py
maksimdrachov Mar 12, 2023
43b2a51
Update pycyphal/transport/udp/_session/_output.py
maksimdrachov Mar 12, 2023
d00105c
Update pycyphal/transport/serial/_frame.py
maksimdrachov Mar 12, 2023
f3acc29
Update pycyphal/transport/serial/_frame.py
maksimdrachov Mar 12, 2023
c7e9d78
Update pycyphal/transport/udp/_frame.py
maksimdrachov Mar 12, 2023
c69c20e
Update pycyphal/transport/udp/_frame.py
maksimdrachov Mar 12, 2023
38dbccb
Update pycyphal/transport/udp/_frame.py
maksimdrachov Mar 12, 2023
c0d50f2
Update pycyphal/transport/udp/_session/_input.py
maksimdrachov Mar 12, 2023
61bf876
Update pycyphal/transport/udp/_session/_output.py
maksimdrachov Mar 12, 2023
28bf1c1
Update pycyphal/transport/udp/_session/_output.py
maksimdrachov Mar 12, 2023
c8d1c78
IPAddress fixes
maksimdrachov Mar 12, 2023
6097440
removing UDPStatistics (some # for future ref)
maksimdrachov Mar 12, 2023
4a943d1
removing #
maksimdrachov Mar 12, 2023
7dd7b86
mypy fixes
maksimdrachov Mar 12, 2023
170df26
changelog and version update
maksimdrachov Mar 12, 2023
5fd6bc3
some future shit
maksimdrachov Mar 12, 2023
8832525
Update the demo docs to use the new UDP transport
pavel-kirienko Mar 12, 2023
88e27c5
Update _udp.py
pavel-kirienko Mar 12, 2023
20c18a1
Update _udp.py
pavel-kirienko Mar 12, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.rst
Expand Up @@ -3,6 +3,12 @@
Changelog
=========

v1.13
-----

- Cyphal/UDP: Service transfer are also multicast now. Make sure to take into account the updated IP mapping specifications.
See (`post <https://forum.opencyphal.org/t/cyphal-udp-architectural-issues-caused-by-the-dependency-between-the-nodes-ip-address-and-its-identity/1765>`_).

v1.12
-----

Expand Down
25 changes: 7 additions & 18 deletions docs/pages/demo.rst
Expand Up @@ -101,7 +101,7 @@ We can resolve this by passing the correct register values via environment varia
.. code-block:: sh

export UAVCAN__NODE__ID=42 # Set the local node-ID 42 (anonymous by default)
export UAVCAN__UDP__IFACE=127.9.0.0 # Use Cyphal/UDP transport via 127.9.0.42 (sic!)
export UAVCAN__UDP__IFACE=127.0.0.1 # Use Cyphal/UDP transport via localhost
export UAVCAN__SUB__TEMPERATURE_SETPOINT__ID=2345 # Subject "temperature_setpoint" on ID 2345
export UAVCAN__SUB__TEMPERATURE_MEASUREMENT__ID=2346 # Subject "temperature_measurement" on ID 2346
export UAVCAN__PUB__HEATER_VOLTAGE__ID=2347 # Subject "heater_voltage" on ID 2347
Expand All @@ -114,17 +114,6 @@ The snippet is valid for sh/bash/zsh; if you are using PowerShell on Windows, re
and take values into double quotes.
Further snippets will not include this remark.

.. tip:: macOS loopback addresses

macOS does not properly adhere to RFC3330 such that you will need to manually create aliases for each
iface + node-ID used.
In our example, with iface being ``127.9.0.0`` and node-ID being ``42`` this would be
``sudo ifconfig lo0 alias 127.9.0.42 up``
(see `this <https://superuser.com/questions/458875>`_ superuser article for more details).

If you come across ``InvalidMediaConfigurationError`` that usually means you haven't set the right alias,
you can verify this using ``ifconfig``.

An environment variable ``UAVCAN__SUB__TEMPERATURE_SETPOINT__ID`` sets register ``uavcan.sub.temperature_setpoint.id``,
and so on.

Expand Down Expand Up @@ -183,35 +172,35 @@ If you decided to change the working directory or move the compilation outputs,
make sure to export the ``YAKUT_PATH`` environment variable pointing to the correct location.

The commands shown later need to operate on the same network as the demo.
Earlier we configured the demo to use Cyphal/UDP via 127.9.0.42.
Earlier we configured the demo to use Cyphal/UDP via the localhost interface.
So, for Yakut, we can export this configuration to let it run on the same network anonymously:

.. code-block:: sh

export UAVCAN__UDP__IFACE=127.9.0.0 # We don't export the node-ID, so it will remain anonymous.
export UAVCAN__UDP__IFACE=127.0.0.1 # We don't export the node-ID, so it will remain anonymous.

To listen to the demo's heartbeat and diagnostics,
launch the following in a new terminal and leave it running (``y`` is a convenience shortcut for ``yakut``):

.. code-block:: sh

export UAVCAN__UDP__IFACE=127.9.0.0
export UAVCAN__UDP__IFACE=127.0.0.1
y sub --with-metadata uavcan.node.heartbeat uavcan.diagnostic.record # You should see heartbeats

Now let's see how the simple thermostat node is operating.
Launch another subscriber to see the published voltage command (it is not going to print anything yet):

.. code-block:: sh

export UAVCAN__UDP__IFACE=127.9.0.0
export UAVCAN__UDP__IFACE=127.0.0.1
y sub 2347:uavcan.si.unit.voltage.scalar --redraw # Prints nothing.

And publish the setpoint along with the measurement (process variable):

.. code-block:: sh

export UAVCAN__UDP__IFACE=127.9.0.0
export UAVCAN__NODE__ID=111 # We need a node-ID to publish messages
export UAVCAN__UDP__IFACE=127.0.0.1
export UAVCAN__NODE__ID=111 # We need a node-ID to publish messages properly
y pub --count=10 2345:uavcan.si.unit.temperature.scalar 250 \
2346:uavcan.si.sample.temperature.scalar 'kelvin: 240'

Expand Down
2 changes: 1 addition & 1 deletion pycyphal/_version.py
@@ -1 +1 @@
__version__ = "1.12.1"
__version__ = "1.13.0"
6 changes: 3 additions & 3 deletions pycyphal/application/__init__.py
Expand Up @@ -196,7 +196,7 @@
>>> os.environ["UAVCAN__NODE__ID"] = "42"
>>> os.environ["UAVCAN__PUB__MEASURED_VOLTAGE__ID"] = "6543"
>>> os.environ["UAVCAN__SUB__OPTIONAL_PORT__ID"] = "65535"
>>> os.environ["UAVCAN__UDP__IFACE"] = "127.63.0.0"
>>> os.environ["UAVCAN__UDP__IFACE"] = "127.0.0.1"
>>> os.environ["UAVCAN__SERIAL__IFACE"] = "socket://127.0.0.1:50905"
>>> os.environ["UAVCAN__DIAGNOSTIC__SEVERITY"] = "3.1"
>>> os.environ["M__MOTOR__INDUCTANCE_DQ"] = "0.12 0.13"
Expand All @@ -208,15 +208,15 @@
UAVCAN__NODE__ID 42
UAVCAN__PUB__MEASURED_VOLTAGE__ID 6543
UAVCAN__SUB__OPTIONAL_PORT__ID 65535
UAVCAN__UDP__IFACE 127.63.0.0
UAVCAN__UDP__IFACE 127.0.0.1
UAVCAN__SERIAL__IFACE socket://127.0.0.1:50905
UAVCAN__DIAGNOSTIC__SEVERITY 3.1
M__MOTOR__INDUCTANCE_DQ 0.12 0.13
>>> node = pycyphal.application.make_node(node_info, "registers.db") # The file will be created if doesn't exist.
>>> node.id
42
>>> node.presentation.transport # Heterogeneously redundant transport: UDP+Serial, as specified in env vars.
RedundantTransport(UDPTransport('127.63.0.42', ...), SerialTransport('socket://127.0.0.1:50905', ...))
RedundantTransport(UDPTransport('127.0.0.1', local_node_id=42, ...), SerialTransport('socket://127.0.0.1:50905', ...))
>>> pub_voltage = node.make_publisher(uavcan.si.unit.voltage.Scalar_1, "measured_voltage")
>>> pub_voltage.port_id
6543
Expand Down
6 changes: 3 additions & 3 deletions pycyphal/application/_transport_factory.py
Expand Up @@ -154,16 +154,16 @@ def make_transport(

>>> from pycyphal.application.register import ValueProxy, Natural16, Natural32
>>> reg = {
... "uavcan.udp.iface": ValueProxy("127.99.0.0"),
... "uavcan.udp.iface": ValueProxy("127.0.0.1"),
... "uavcan.node.id": ValueProxy(Natural16([257])),
... }
>>> tr = make_transport(reg)
>>> tr
UDPTransport('127.99.1.1', local_node_id=257, ...)
UDPTransport('127.0.0.1', local_node_id=257, ...)
>>> tr.close()
>>> tr = make_transport(reg, reconfigurable=True) # Same but reconfigurable.
>>> tr # Wrapped into RedundantTransport.
RedundantTransport(UDPTransport('127.99.1.1', local_node_id=257, ...))
RedundantTransport(UDPTransport('127.0.0.1', local_node_id=257, ...))
>>> tr.close()

>>> int(reg["uavcan.udp.mtu"]) # Defaults created automatically to expose all configurables.
Expand Down
2 changes: 2 additions & 0 deletions pycyphal/presentation/_port/_client.py
Expand Up @@ -356,6 +356,8 @@ async def _task_function(self) -> None:
fut.set_result((response, transfer))
except asyncio.CancelledError:
_logger.debug("Cancelling the task of %s", self)
except pycyphal.transport.ResourceClosedError as ex:
_logger.debug("Cancelling the task of %s because the underlying resource is closed: %s", self, ex)
except Exception as ex:
exception = ex
# Do not use f-string because it can throw, unlike the built-in formatting facility of the logger
Expand Down
2 changes: 1 addition & 1 deletion pycyphal/transport/_data_specifier.py
Expand Up @@ -23,7 +23,7 @@ class MessageDataSpecifier(DataSpecifier):

def __post_init__(self) -> None:
if not (0 <= self.subject_id <= self.SUBJECT_ID_MASK):
raise ValueError(f"Invalid subject ID: {self.subject_id}")
raise ValueError(f"Invalid subject-ID: {self.subject_id}")


@dataclasses.dataclass(frozen=True)
Expand Down