Route back (#544)

* Add a bind ip and port to work on containers. * Add ENV read for config. And update docs. * Add log.debug to see config overwritte by env vars. * Rollback version in file. * Remove details from changelog.md * Add details to changelog.md * Remove local_port for Routing mode. * xknx.yaml: Add config examples for connections. * Connect: Check net_bind address * Update docs/configuration.md Sure, ok for this update :) Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update xknx/io/connectionstate.py Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update xknx/io/request_response.py Ok Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update xknx/io/disconnect.py Better code design :) Co-authored-by: Matthias Alphart <farmio@alphart.net> * Remove bind_ and add route_back. * Remove bind_ and add route_back. * Remove local_port from config. * Update tests. * Update examples. * minor updates to allow tox success. * Minor config update and add tests. * Update docs/configuration.md Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update docs/xknx.md Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update docs/xknx.md Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update docs/xknx.md Co-authored-by: Matthias Alphart <farmio@alphart.net> * Update after revue. * Update after revue (AUTO on tests + default config file). * Fix /issues/570 in my own branch See [issue here](#570) * fix broken tests from merging master * default parameters and changelog * Tunnel Interface changed - gateway_ip, gateway_port before local_ip, local_port added with default `0` Co-authored-by: Mielune <gitlab@breuillard.eu> Co-authored-by: Matthias Alphart <farmio@alphart.net>
XKNX · Feb 17, 2021 · 4f83cfe · 4f83cfe
1 parent 4f8ae49
commit 4f83cfe
Show file tree

Hide file tree

Showing 17 changed files with 443 additions and 29 deletions.
diff --git a/changelog.md b/changelog.md
@@ -2,6 +2,11 @@
 
 ## Unreleased changes
 
+### New Features
+
+- Add new optional config `route_back` for connections to be able to work behind NAT.
+- Read env vars after reading config file to allow dynamic config.
+
 ### HA integration
 
 - knx_event: fire also for outgoing telegrams
@@ -19,6 +24,7 @@
 - return the payload (or None) in RemoteValue.payload_valid(payload) instead of bool
 - Light colors are represented as `Tuple[Tuple[int,int,int], int]` instead of `Tuple[List[int], int]` now
 - DPT 3 payloads/values are not invertable anymore.
+- Tunnel: Interface changed - gateway_ip, gateway_port before local_ip, local_port added with default `0`.
 
 ## 0.16.3 Fan contributions 2021-02-06
 

diff --git a/docs/configuration.md b/docs/configuration.md
@@ -24,6 +24,7 @@ The configuration file can contain three sections.
     - `gateway_ip` (required) sets the ip address of the KNX tunneling interface
     - `gateway_port` (optional) sets the port the KNX tunneling interface is listening on
     - `local_ip` (optional) sets the ip address that is used by xknx
+    - `route_back` (optional) When True the KNXnet/IP Server shall use the IP address and the port number from the IP package received as the target IP address or port number for the response to the KNXnet/IP Client (for NAT / Docker). Defalut: `False`
   - `routing` for a UDP multicast connection
     - `local_ip` (optional) sets the ip address that is used by xknx
 - Within the `groups` sections all devices are defined. For each type of device more then one section might be specified. You need to append numbers or strings to differentiate the entries, as in the example below. The appended number or string must be unique.

diff --git a/docs/xknx.md b/docs/xknx.md
@@ -146,5 +146,27 @@ async def main():
 asyncio.run(main())
 ```
 
-
-
+# [](#header-2)Dockerised xknx's app
+
+To run xknx from a container, set 'route_back=true' or use host network mode.
+
+Available env variables are:
+- XKNX_GENERAL_OWN_ADDRESS
+- XKNX_GENERAL_RATE_LIMIT
+- XKNX_GENERAL_MULTICAST_GROUP
+- XKNX_GENERAL_MULTICAST_PORT
+- XKNX_CONNECTION_GATEWAY_IP: Your KNX Gateway IP address
+- XKNX_CONNECTION_GATEWAY_PORT: Your KNX Gateway UDP port
+- XKNX_CONNECTION_LOCAL_IP
+- XKNX_CONNECTION_ROUTE_BACK: Set 'true' to be able to work in a container
+
+Example of a `docker run` with an xknx based app:
+
+```bash
+docker run --name myapp -d \
+  -e XKNX_CONNECTION_GATEWAY_IP='192.168.0.123' \
+  -e XKNX_CONNECTION_LOCAL_PORT=12399 \
+  -e XKNX_CONNECTION_ROUTE_BACK=true \
+  -p 12300:12399/udp \
+  myapp:latest
+```
diff --git a/examples/example_tunnel.py b/examples/example_tunnel.py
@@ -30,9 +30,11 @@ async def main():
 
     tunnel = Tunnel(
         xknx,
-        local_ip=gateway.local_ip,
         gateway_ip=gateway.ip_addr,
         gateway_port=gateway.port,
+        local_ip=gateway.local_ip,
+        local_port=0,
+        route_back=False,
     )
 
     await tunnel.connect()

diff --git a/test/config_tests/config_v1_test.py b/test/config_tests/config_v1_test.py
@@ -1,5 +1,6 @@
 """Unit test for Configuration logic."""
 import asyncio
+import os
 import unittest
 from unittest.mock import patch
 
@@ -608,3 +609,113 @@ def test_config_file_error(self):
             mock_parse.side_effect = XKNXException()
             XKNX(config="xknx.yaml")
             self.assertEqual(mock_err.call_count, 1)
+
+    XKNX_GENERAL_OWN_ADDRESS = "1.2.3"
+    XKNX_GENERAL_RATE_LIMIT = "20"
+    XKNX_GENERAL_MULTICAST_GROUP = "1.2.3.4"
+    XKNX_GENERAL_MULTICAST_PORT = "1111"
+
+    def test_config_general_from_env(self):
+        os.environ["XKNX_GENERAL_OWN_ADDRESS"] = self.XKNX_GENERAL_OWN_ADDRESS
+        os.environ["XKNX_GENERAL_RATE_LIMIT"] = self.XKNX_GENERAL_RATE_LIMIT
+        os.environ["XKNX_GENERAL_MULTICAST_GROUP"] = self.XKNX_GENERAL_MULTICAST_GROUP
+        os.environ["XKNX_GENERAL_MULTICAST_PORT"] = self.XKNX_GENERAL_MULTICAST_PORT
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_GENERAL_OWN_ADDRESS"]
+        del os.environ["XKNX_GENERAL_RATE_LIMIT"]
+        del os.environ["XKNX_GENERAL_MULTICAST_GROUP"]
+        del os.environ["XKNX_GENERAL_MULTICAST_PORT"]
+        self.assertEqual(str(self.xknx.own_address), self.XKNX_GENERAL_OWN_ADDRESS)
+        self.assertEqual(self.xknx.rate_limit, int(self.XKNX_GENERAL_RATE_LIMIT))
+        self.assertEqual(self.xknx.multicast_group, self.XKNX_GENERAL_MULTICAST_GROUP)
+        self.assertEqual(
+            self.xknx.multicast_port, int(self.XKNX_GENERAL_MULTICAST_PORT)
+        )
+
+    XKNX_CONNECTION_GATEWAY_IP = "192.168.12.34"
+    XKNX_CONNECTION_GATEWAY_PORT = "1234"
+    XKNX_CONNECTION_LOCAL_IP = "192.168.11.11"
+    XKNX_CONNECTION_ROUTE_BACK = "true"
+
+    def test_config_cnx_from_env(self):
+        os.environ["XKNX_CONNECTION_GATEWAY_IP"] = self.XKNX_CONNECTION_GATEWAY_IP
+        os.environ["XKNX_CONNECTION_GATEWAY_PORT"] = self.XKNX_CONNECTION_GATEWAY_PORT
+        os.environ["XKNX_CONNECTION_LOCAL_IP"] = self.XKNX_CONNECTION_LOCAL_IP
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = self.XKNX_CONNECTION_ROUTE_BACK
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_GATEWAY_IP"]
+        del os.environ["XKNX_CONNECTION_GATEWAY_PORT"]
+        del os.environ["XKNX_CONNECTION_LOCAL_IP"]
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.gateway_ip, self.XKNX_CONNECTION_GATEWAY_IP
+        )
+        self.assertEqual(
+            self.xknx.connection_config.gateway_port,
+            int(self.XKNX_CONNECTION_GATEWAY_PORT),
+        )
+        self.assertEqual(
+            self.xknx.connection_config.local_ip, self.XKNX_CONNECTION_LOCAL_IP
+        )
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            bool(self.XKNX_CONNECTION_ROUTE_BACK),
+        )
+
+    def test_config_cnx_route_back(self):
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = "true"
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            True,
+        )
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = "yes"
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            True,
+        )
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = "1"
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            True,
+        )
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = "on"
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            True,
+        )
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = "y"
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            True,
+        )
+        if "XKNX_CONNECTION_ROUTE_BACK" in os.environ:
+            del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.xknx = XKNX(config="xknx.yaml")
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            False,
+        )
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = "another_string"
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            False,
+        )
+        os.environ["XKNX_CONNECTION_ROUTE_BACK"] = ""
+        self.xknx = XKNX(config="xknx.yaml")
+        del os.environ["XKNX_CONNECTION_ROUTE_BACK"]
+        self.assertEqual(
+            self.xknx.connection_config.route_back,
+            False,
+        )
diff --git a/test/io_tests/connect_test.py b/test/io_tests/connect_test.py
@@ -32,7 +32,7 @@ def test_connect(self):
         """Test connecting from KNX bus."""
         xknx = XKNX()
         udp_client = UDPClient(xknx, ("192.168.1.1", 0), ("192.168.1.2", 1234))
-        connect = Connect(xknx, udp_client)
+        connect = Connect(xknx, udp_client, route_back=False)
         connect.timeout_in_seconds = 0
 
         self.assertEqual(connect.awaited_response_class, ConnectResponse)
@@ -83,3 +83,53 @@ def test_connect(self):
         self.assertTrue(connect.success)
         self.assertEqual(connect.communication_channel, 23)
         self.assertEqual(connect.identifier, 7)
+
+    def test_connect_route_back_true(self):
+        """Test connecting from KNX bus."""
+        xknx = XKNX()
+        udp_client = UDPClient(xknx, ("192.168.1.1", 0), ("192.168.1.2", 1234))
+        connect = Connect(xknx, udp_client, route_back=True)
+        connect.timeout_in_seconds = 0
+
+        self.assertEqual(connect.awaited_response_class, ConnectResponse)
+
+        # Expected KNX/IP-Frame:
+        exp_knxipframe = KNXIPFrame.init_from_body(
+            ConnectRequest(xknx, request_type=ConnectRequestType.TUNNEL_CONNECTION)
+        )
+        with patch("xknx.io.UDPClient.send") as mock_udp_send, patch(
+            "xknx.io.UDPClient.getsockname"
+        ) as mock_udp_getsockname:
+            mock_udp_getsockname.return_value = ("192.168.1.3", 4321)
+            self.loop.run_until_complete(connect.start())
+            mock_udp_send.assert_called_with(exp_knxipframe)
+
+        # Response KNX/IP-Frame with wrong type
+        wrong_knxipframe = KNXIPFrame(xknx)
+        wrong_knxipframe.init(KNXIPServiceType.CONNECTIONSTATE_REQUEST)
+        with patch("logging.Logger.warning") as mock_warning:
+            connect.response_rec_callback(wrong_knxipframe, None)
+            mock_warning.assert_called_with("Could not understand knxipframe")
+
+        # Response KNX/IP-Frame with error:
+        err_knxipframe = KNXIPFrame(xknx)
+        err_knxipframe.init(KNXIPServiceType.CONNECT_RESPONSE)
+        err_knxipframe.body.status_code = ErrorCode.E_CONNECTION_ID
+        with patch("logging.Logger.debug") as mock_warning:
+            connect.response_rec_callback(err_knxipframe, None)
+            mock_warning.assert_called_with(
+                "Error: KNX bus responded to request of type '%s' with error in '%s': %s",
+                type(connect).__name__,
+                type(err_knxipframe.body).__name__,
+                ErrorCode.E_CONNECTION_ID,
+            )
+
+        # Correct Response KNX/IP-Frame:
+        res_knxipframe = KNXIPFrame(xknx)
+        res_knxipframe.init(KNXIPServiceType.CONNECT_RESPONSE)
+        res_knxipframe.body.communication_channel = 23
+        res_knxipframe.body.identifier = 7
+        connect.response_rec_callback(res_knxipframe, None)
+        self.assertTrue(connect.success)
+        self.assertEqual(connect.communication_channel, 23)
+        self.assertEqual(connect.identifier, 7)
diff --git a/test/io_tests/connectionstate_test.py b/test/io_tests/connectionstate_test.py
@@ -32,7 +32,9 @@ def test_connectionstate(self):
         xknx = XKNX()
         communication_channel_id = 23
         udp_client = UDPClient(xknx, ("192.168.1.1", 0), ("192.168.1.2", 1234))
-        connectionstate = ConnectionState(xknx, udp_client, communication_channel_id)
+        connectionstate = ConnectionState(
+            xknx, udp_client, communication_channel_id, route_back=False
+        )
         connectionstate.timeout_in_seconds = 0
 
         self.assertEqual(
@@ -82,3 +84,60 @@ def test_connectionstate(self):
         res_knxipframe.init(KNXIPServiceType.CONNECTIONSTATE_RESPONSE)
         connectionstate.response_rec_callback(res_knxipframe, None)
         self.assertTrue(connectionstate.success)
+
+    def test_connectionstate_route_back_true(self):
+        """Test connectionstateing from KNX bus."""
+        xknx = XKNX()
+        communication_channel_id = 23
+        udp_client = UDPClient(xknx, ("192.168.1.1", 0), ("192.168.1.2", 1234))
+        connectionstate = ConnectionState(
+            xknx, udp_client, communication_channel_id, route_back=True
+        )
+        connectionstate.timeout_in_seconds = 0
+
+        self.assertEqual(
+            connectionstate.awaited_response_class, ConnectionStateResponse
+        )
+        self.assertEqual(
+            connectionstate.communication_channel_id, communication_channel_id
+        )
+
+        # Expected KNX/IP-Frame:
+        exp_knxipframe = KNXIPFrame.init_from_body(
+            ConnectionStateRequest(
+                xknx,
+                communication_channel_id=communication_channel_id,
+            )
+        )
+        with patch("xknx.io.UDPClient.send") as mock_udp_send, patch(
+            "xknx.io.UDPClient.getsockname"
+        ) as mock_udp_getsockname:
+            mock_udp_getsockname.return_value = ("192.168.1.3", 4321)
+            self.loop.run_until_complete(connectionstate.start())
+            mock_udp_send.assert_called_with(exp_knxipframe)
+
+        # Response KNX/IP-Frame with wrong type
+        wrong_knxipframe = KNXIPFrame(xknx)
+        wrong_knxipframe.init(KNXIPServiceType.CONNECTIONSTATE_REQUEST)
+        with patch("logging.Logger.warning") as mock_warning:
+            connectionstate.response_rec_callback(wrong_knxipframe, None)
+            mock_warning.assert_called_with("Could not understand knxipframe")
+
+        # Response KNX/IP-Frame with error:
+        err_knxipframe = KNXIPFrame(xknx)
+        err_knxipframe.init(KNXIPServiceType.CONNECTIONSTATE_RESPONSE)
+        err_knxipframe.body.status_code = ErrorCode.E_CONNECTION_ID
+        with patch("logging.Logger.debug") as mock_warning:
+            connectionstate.response_rec_callback(err_knxipframe, None)
+            mock_warning.assert_called_with(
+                "Error: KNX bus responded to request of type '%s' with error in '%s': %s",
+                type(connectionstate).__name__,
+                type(err_knxipframe.body).__name__,
+                ErrorCode.E_CONNECTION_ID,
+            )
+
+        # Correct Response KNX/IP-Frame:
+        res_knxipframe = KNXIPFrame(xknx)
+        res_knxipframe.init(KNXIPServiceType.CONNECTIONSTATE_RESPONSE)
+        connectionstate.response_rec_callback(res_knxipframe, None)
+        self.assertTrue(connectionstate.success)
diff --git a/test/io_tests/disconnect_test.py b/test/io_tests/disconnect_test.py
@@ -32,7 +32,9 @@ def test_disconnect(self):
         xknx = XKNX()
         communication_channel_id = 23
         udp_client = UDPClient(xknx, ("192.168.1.1", 0), ("192.168.1.2", 1234))
-        disconnect = Disconnect(xknx, udp_client, communication_channel_id)
+        disconnect = Disconnect(
+            xknx, udp_client, communication_channel_id, route_back=False
+        )
         disconnect.timeout_in_seconds = 0
 
         self.assertEqual(disconnect.awaited_response_class, DisconnectResponse)
@@ -78,3 +80,56 @@ def test_disconnect(self):
         res_knxipframe.init(KNXIPServiceType.DISCONNECT_RESPONSE)
         disconnect.response_rec_callback(res_knxipframe, None)
         self.assertTrue(disconnect.success)
+
+    def test_disconnect_route_back_true(self):
+        """Test disconnecting from KNX bus."""
+        xknx = XKNX()
+        communication_channel_id = 23
+        udp_client = UDPClient(xknx, ("192.168.1.1", 0), ("192.168.1.2", 1234))
+        disconnect = Disconnect(
+            xknx, udp_client, communication_channel_id, route_back=True
+        )
+        disconnect.timeout_in_seconds = 0
+
+        self.assertEqual(disconnect.awaited_response_class, DisconnectResponse)
+        self.assertEqual(disconnect.communication_channel_id, communication_channel_id)
+
+        # Expected KNX/IP-Frame:
+        exp_knxipframe = KNXIPFrame.init_from_body(
+            DisconnectRequest(
+                xknx,
+                communication_channel_id=communication_channel_id,
+            )
+        )
+        with patch("xknx.io.UDPClient.send") as mock_udp_send, patch(
+            "xknx.io.UDPClient.getsockname"
+        ) as mock_udp_getsockname:
+            mock_udp_getsockname.return_value = ("192.168.1.3", 4321)
+            self.loop.run_until_complete(disconnect.start())
+            mock_udp_send.assert_called_with(exp_knxipframe)
+
+        # Response KNX/IP-Frame with wrong type
+        wrong_knxipframe = KNXIPFrame(xknx)
+        wrong_knxipframe.init(KNXIPServiceType.DISCONNECT_REQUEST)
+        with patch("logging.Logger.warning") as mock_warning:
+            disconnect.response_rec_callback(wrong_knxipframe, None)
+            mock_warning.assert_called_with("Could not understand knxipframe")
+
+        # Response KNX/IP-Frame with error:
+        err_knxipframe = KNXIPFrame(xknx)
+        err_knxipframe.init(KNXIPServiceType.DISCONNECT_RESPONSE)
+        err_knxipframe.body.status_code = ErrorCode.E_CONNECTION_ID
+        with patch("logging.Logger.debug") as mock_warning:
+            disconnect.response_rec_callback(err_knxipframe, None)
+            mock_warning.assert_called_with(
+                "Error: KNX bus responded to request of type '%s' with error in '%s': %s",
+                type(disconnect).__name__,
+                type(err_knxipframe.body).__name__,
+                ErrorCode.E_CONNECTION_ID,
+            )
+
+        # Correct Response KNX/IP-Frame:
+        res_knxipframe = KNXIPFrame(xknx)
+        res_knxipframe.init(KNXIPServiceType.DISCONNECT_RESPONSE)
+        disconnect.response_rec_callback(res_knxipframe, None)
+        self.assertTrue(disconnect.success)