Support for proxy mode to support docker containers and reverse proxies

[ruimarinho]

Hi,

I'm running Home Assistant inside docker using isolated VLAN networks in bridge mode with CoAP unicast integration. Requests and replies using CoAP work fine because the request is initiated by the docker container, therefore the reply is correctly identified with the right Shelly device.

However, periodic and local push messages are initiated by Shelly devices and consequently the CoAP message has the IP of the Home Assistant container and not the IP of the Shelly device (e.g. a docker internal container IP of 172.21.0.4 instead of 192.168.100.195, assuming an IoT VLAN of 192.168.100.0/24).

One of the ideas I've been playing around is to use a CoAP Forwarder/Relayer running on a macvlan on the main subnet which listens to UDP datagrams and adds a tag (prxy) with the original IP address (like an X-Forwarded-For HTTP header). See the original idea here.

The problem with this approach is that aioshelly does not understand that tag, resulting in an error like Invalid Message from unknown host 172.21.0.8: Wrong type, 114. I'm unsure if this is part of the CoAP standard or some custom modification, but the issue is real and hard to solve without support for a similar setting.

Here's the output with (first line) and without the tag (second line) appended:

coap                | b'P\x1e\x96V\xb3cit\x01s\xed\x0b\xec\x00SHEM#B9EA81#2\xd2C\x96\x00\x82\x19\x00\xff{"G":[[0,9103,0],[0,1101,0],[0,4105,552.74],[0,4106,4067801.3],[0,4107,570811.0],[0,4108,241.90],[0,4205,4.49],[0,4206,1680482.2],[0,4207,0.0],[0,4208,241.90],[0,6102,0]]}'
coap                | bytearray(b'prxy\xc0\xa8d\xc3P\x1e\x96V\xb3cit\x01s\xed\x0b\xec\x00SHEM#B9EA81#2\xd2C\x96\x00\x82\x19\x00\xff{"G":[[0,9103,0],[0,1101,0],[0,4105,552.74],[0,4106,4067801.3],[0,4107,570811.0],[0,4108,241.90],[0,4205,4.49],[0,4206,1680482.2],[0,4207,0.0],[0,4208,241.90],[0,6102,0]]}')

Looking forward to hear your thoughts.

[thecode]

The implementation you linked is a custom implementation which create an invalid CoAP packet which is later removed by receiver. We have considered supporting proxying messages, but for a different use case, to allow using unicast while still working with multiple server/containers, however I can't commit to timeline when and if it will happen.

The fact that you use an isolated VLAN or docker container should not impact the periodic updates. I know there are some users who succeeded to work in such setup, I suggest you use the forums to get help. You can try our Community Forum or join our Discord chat server.

[ruimarinho]

Understood! I'll keep my MQTT integration for now and later on reconsider shifting to the built-in integration.

The fact that you use an isolated VLAN or docker container should not impact the periodic updates.

It's not possible in bridge network mode due to way a bridge works. Other networking modes could make it viable (e.g. host), but that sort of defeats the isolation principle of containers. The only solution is including additional metadata of the original request.

[thecode]

@ruimarinho the support for subscribing to device updates by MAC should work under any network configuration as long as the packets can reach aioshelly, without the need to modify the packet payload. this is not yet implemented in Home Assistant but you are welcome to test it using the example in the linked PR

[ruimarinho]

This is pretty awesome @thecode. Great work -- I can finally remove the host mode workaround on my Home Assistant container!

It seems like the integration in Home Assistant automatically uses the device's MAC address for listening to packets? If so, I'll give 2023.2.0b2 a test!

[thecode]

Yes, Home Assistant beta is already using latest aioshelly, if you encounter issues please create an issue in the core repository. Thanks 👍