Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Trådfri gateway security improvements using DTLS #90

Closed
hanpal opened this issue Oct 31, 2017 · 38 comments

Comments

Projects
None yet
@hanpal
Copy link

commented Oct 31, 2017

Just got this from The Ikea of Sweden Trådfri Team.
I hope you're already aware of this, I don't understand the technical details but I assume that this is a breaking change for pytradfri.

We at IKEA would like to inform you about a change to our TRÅDFRI Gateway. We are very happy to see your interest in our gateway and have seen that you are using the CoAP interface. We consider the CoAP interface as our internal interface not developed for third party usage and therefore we do not offer any technical support for this usage. However that does not mean that we want to hinder your work in any way.

There are some security improvements in a soon coming update that we would like to inform you about since it will break your implementation. Technically the improvement is that the TRÅDFRI Gateway will start using DTLS Identities which you will need to handle in your application.

Please use the Following string to connect to the TRÅDFRI Gateway and create a new DTLS Identity.

coap-client -m post -u "Client_identity" -k "SECURITY_CODE" -e '{"9090":"IDENTITY"}' "coaps://IP_ADDRESS:5684/15011/9063"

SECURITY_CODE is what is labelled on the Gateway label, IDENTITY is any string that is representing the connection.
You will then get back a PRE_SHARED_KEY that can be use in all traffic after that.

coap-client -m get -u "IDENTITY" -k "PRE_SHARED_KEY" "coaps://IP_ADDRESS:5684/15001"

We also would like to request that the SECURITY_CODE that is printed on the gateway is never stored permanently in your application.

Best regards
IKEA of Sweden Trådfri team

@lwis

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

Thanks for the heads up, this is something we'll need to support.

@hanpal

This comment has been minimized.

Copy link
Author

commented Oct 31, 2017

Is my assumption correct regarding breaking change? The change was going to be introduced today so users of pytradfri (including use from HASS etc) should postpone the firmware upgrade (if possible) until pytradfri has been updated?

@thomasdelaet

This comment has been minimized.

Copy link

commented Oct 31, 2017

Too late for me ... my gateway upgraded its firmware and I can not use tradfri anymore. This is error I get when using from home assistant:

decrypt_verify(): found 24 bytes cleartext
decrypt_verify(): found 18 bytes cleartext
2017-10-31 13:43:52 ERROR (MainThread) [homeassistant.setup] Error during setup of component tradfri
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/setup.py", line 191, in _async_setup_component
    result = yield from component.async_setup(hass, processed_config)
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/tradfri.py", line 111, in async_setup
    allow_tradfri_groups))
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/tradfri.py", line 125, in _setup_gateway
    api = yield from api_factory(host, key, loop=hass.loop)
  File "/srv/homeassistant/lib/python3.5/site-packages/pytradfri/api/aiocoap_api.py", line 152, in api_factory
    yield from request(Command('get', ['status']))
  File "/srv/homeassistant/lib/python3.5/site-packages/pytradfri/api/aiocoap_api.py", line 117, in request
    result = yield from _execute(api_commands)
  File "/srv/homeassistant/lib/python3.5/site-packages/pytradfri/api/aiocoap_api.py", line 109, in _execute
    api_command.result = _process_output(res, parse_json)
  File "/srv/homeassistant/lib/python3.5/site-packages/pytradfri/api/aiocoap_api.py", line 176, in _process_output
    return json.loads(output)
  File "/usr/lib/python3.5/json/__init__.py", line 319, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.5/json/decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.5/json/decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
@lwis

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

This handshake will require some changes to this lib and to Home Assistant, so don't expect a quick fix :(

@thomasdelaet

This comment has been minimized.

Copy link

commented Oct 31, 2017

@hanpal I'm playing with this but I don't know which coap-client version they are referring to since the libcoap version does not support -k and -u parameters. Any idea?

@lwis

This comment has been minimized.

@hanpal

This comment has been minimized.

Copy link
Author

commented Oct 31, 2017

Added a URL filter for fw.ota.homesmart.ikea.net in my router. Hope this works....

Also blocket TCP/UDP for the gateway LAN IP towards WAN.

@balloob

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

Shouldn't be too hard. I think we open connection to another endpoint, get the key, then use that key moving forward.

@diplix

This comment has been minimized.

Copy link

commented Oct 31, 2017

the update just rolled out. wasn’t quick enough to block wan access for the gateway.
img_2746

@lwis

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

@balloob should this behave more like media_player.bravia ? They've asked that we don't store the security key in any configs. So should we get the user to enter it, which we then exchange for a token, which is stored?

@hanpal

This comment has been minimized.

Copy link
Author

commented Oct 31, 2017

In that case, why not also get the IP address from the user. If that is stored somewhere together with the client PRE_SHARED_KEY, we can get rid of ip/key settings in user scripts.

@ggravlingen

This comment has been minimized.

Copy link
Owner

commented Oct 31, 2017

#92 has been raised by @lwis to find a fix for the problem.

@balloob

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

@lwis correct, in Home Assistant we get the security key as normal, use it to fetch an access key and only store the access key. We will not store the security key.

@hanpal

This comment has been minimized.

Copy link
Author

commented Oct 31, 2017

This should work if your gateway has been accidentally updated before a new pytradfri version has been released:

  1. Block internet access to the gateway
  2. Perform factory reset. The gateway then reverts to a pre-stored version (the one available when it was manufactured). The gateway will try to update to the latest version after factory reset but this will be prohibited according to 1.
@mories76

This comment has been minimized.

Copy link

commented Oct 31, 2017

It looks like the gateway only provides the PSK once. So if you run the first command store the result. I haven't found a way to request the PSK a second time.

@ggravlingen

This comment has been minimized.

Copy link
Owner

commented Oct 31, 2017

@mories76 same here, don't get the psk the second time I run the command

@hanpal

This comment has been minimized.

Copy link
Author

commented Oct 31, 2017

So what if you request the PSK and forget it? Factory reset?

@lwis

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

@hanpal you'll probably have to switch identity.

@ggravlingen

This comment has been minimized.

Copy link
Owner

commented Oct 31, 2017

@mories76 and @hanpal: this magic command gave me a new PSK :)
coap-client -u "Client_identity" -k "gatewaykey" -v 0 -e '{"9090":"GGRAVLINGEN"}' -m POST "coaps://192.168.0.129:5684/15011/9063"

(Seems it generates based on what you send into the 9090-attribute).

@mories76

This comment has been minimized.

Copy link

commented Oct 31, 2017

@ggravlingen in the first request we requested a PSK for the identity IDENTITY, you have now requested a PSK for the identity GGRAVELIGEN, looks like @lwis is using the identity pytradfri

@lwis

This comment has been minimized.

Copy link
Collaborator

commented Oct 31, 2017

I am, but the request from aiocoap isn't working :(

@MiniMeOSc

This comment has been minimized.

Copy link

commented Oct 31, 2017

@hanpal
Thanks for posting this :-). While I wasn't using pytradfri directly but fiddling with my own code atm, I was a bit confused why yesterday's code wasn't working anymore and why the gateway responded with "4.01 Unauthorized" when I tried it with libcoap/coap-client.

Some notes on investigation I did, the NEW_PSK_BY_GW (const 9091 in those decompiled Java sources) appears to survive a reboot, so it's stored permanently on the gateway, at least in the short term, time will show if it'll remain forever. Probably only a factory reset will invalidate it.

Also, you can only request a key for the same CLIENT_IDENTITY_PROPOSED (9090) once. If you try the same again it'll respond with "4.00 Bad Request".

That allowed me to hotfix my scripts pretty quickly by just replacing Client_identity and the permanent key with one it gave me on my request.

@ggravlingen

This comment has been minimized.

Copy link
Owner

commented Oct 31, 2017

Sharing the latest findings when run with aiocoap. Feel free to share what's wrong. 😄

root@patrik-HP-EliteBook-8460p:/usr/src/app# python3 test.py 192.168.0.129
decrypt_verify(): found 24 bytes cleartext
decrypt_verify(): found 52 bytes cleartext
decrypt_verify(): found 291 bytes cleartext
decrypt_verify(): found 239 bytes cleartext
decrypt_verify(): found 235 bytes cleartext
decrypt_verify(): found 237 bytes cleartext
decrypt_verify(): found 212 bytes cleartext
decrypt_verify(): found 304 bytes cleartext
Task was destroyed but it is pending!
task: <Task pending coro=<DTLSClientConnection._run() running at /usr/local/lib/python3.5/dist-packages/aiocoap/transports/tinydtls.py:160> wait_for=<Future pending cb=[Task._wakeup()]>>
Exception RuntimeError('Event loop is closed',) can not be represented as errno, setting -1.
Error received in UDP connection under DTLS: [Errno 9] Bad file descriptor
Exception ignored in: <generator object DTLSClientConnection._run at 0x7f18ab59e4c0>
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/aiocoap/transports/tinydtls.py", line 180, in _run
  File "/usr/lib/python3.5/asyncio/selector_events.py", line 622, in close
  File "/usr/lib/python3.5/asyncio/base_events.py", line 572, in call_soon
  File "/usr/lib/python3.5/asyncio/base_events.py", line 357, in _check_closed
RuntimeError: Event loop is closed

For the sake of documentation, we had the same problem in the past:
home-assistant/home-assistant#9778

@thomasdelaet

This comment has been minimized.

Copy link

commented Oct 31, 2017

Thanks for posting this :-). While I wasn't using pytradfri directly but fiddling with my own code atm, I was a bit confused why yesterday's code wasn't working anymore and why the gateway responded with "4.01 Unauthorized" when I tried it with libcoap/coap-client.

Some notes on investigation I did, the NEW_PSK_BY_GW (const 9091 in those decompiled Java sources) appears to survive a reboot, so it's stored permanently on the gateway, at least in the short term, time will show if it'll remain forever. Probably only a factory reset will invalidate it.

Also, you can only request a key for the same CLIENT_IDENTITY_PROPOSED (9090) once. If you try the same again it'll respond with "4.00 Bad Request".

Would it make sense to generate client identity randomly every time we start? This would make the whole system more robust but not sure if gateway would be able to handle this (will it start deleting unused client identities?)

@Mrono

This comment has been minimized.

Copy link

commented Nov 1, 2017

I'm on 1.2.42 gateway version, it force installed it's self last night. My lights still work except I can't set 255 brightness, 0-254 works but not 255. No error messages and i've rebooted both HA and the gateway

@javefang

This comment has been minimized.

Copy link

commented Nov 1, 2017

@Mrono it broke for me, I am using pytradfri via home-assistant which uses DTLS

@AlCalzone

This comment has been minimized.

Copy link

commented Nov 1, 2017

In case it helps: This is what I'm doing in the ioBroker tradfri adapter and it works:

  1. Connect with Client_identity / security code to check if that is correct
  2. If a stored identity is found, try to reconnect with that one. Otherwise continue with 4.
  3. If (2.) succeeded, we're done. Otherwise, reconnect with the Client_identity / security code and continue with 4.
  4. Generate a new identity/psk pair and store it.
  5. Reconnect with the new identity/psk pair.
@MiniMeOSc

This comment has been minimized.

Copy link

commented Nov 1, 2017

@AlCalzone

This comment has been minimized.

Copy link

commented Nov 1, 2017

@MiniMeOSc I guess you're right. That seems to be the long-term version, however I needed a quick fix to get it working asap.

The response does come with no content-type set, so I'm skipping the usual payload parsing and use JSON.parse directly.

@lwis

This comment has been minimized.

Copy link
Collaborator

commented Nov 1, 2017

Support was added in ed1fea1.

@lwis lwis closed this Nov 1, 2017

Lendar referenced this issue in morzzz007/node-tradfri Nov 1, 2017

@4tnight

This comment has been minimized.

Copy link

commented Nov 2, 2017

And if you are a total Noob in how to implement this on a Rpi, what to do then?

I have had everything working before the Tradfri update but now i don't know what to do to get this back working again.
Can you guys give any tips or a script on what to do to get this working again on my Rpi?

Tia.

@lwis

This comment has been minimized.

Copy link
Collaborator

commented Nov 2, 2017

@4tnight the example_*.py files in the root are a good place to start, and you should just need to install libcoap via the script or requirements.txt via pip depending on if you're using the sync/async api.

@4tnight

This comment has been minimized.

Copy link

commented Nov 2, 2017

@lwis thanks, i will try tonight.

@4tnight

This comment has been minimized.

Copy link

commented Nov 2, 2017

i got the following message:
pi@raspberrypi:~/Downloads/pytradfri-4.0.0 $ python3 example_async.py 192.168.1.122
Traceback (most recent call last):
File "example_async.py", line 19, in
from pytradfri.api.aiocoap_api import APIFactory
File "/home/pi/Downloads/pytradfri-4.0.0/pytradfri/api/aiocoap_api.py", line 9, in
from aiocoap.transports import tinydtls
ImportError: cannot import name 'tinydtls'

@balloob

This comment has been minimized.

Copy link
Collaborator

commented Nov 2, 2017

Please do not use this issue for support.

@linuxlurak

This comment has been minimized.

Copy link

commented Nov 4, 2017

Sorry @balloob to post here against your explicit advise to not use github for support. I did a lot of googling and if someone searches for the problem with tinydtls google links to this issue.

My solution was to reinstall homeassistant completely. My installation was modified too much and I lost track of my changes and all the executed commands.

So to help @4tnight here is my solution:
(as root or per sudo)
uninstall home assistant:
pip3 freeze | xargs pip3 uninstall -y
in your .homeassistant directory remove deps:
rm -rf ./deps/
then reinstall home assistant

pip3 install homeassistant
pip3 install aiohttp
pip3 install websockets

after this start/restart hass. It takes a while because it compiles and installs other libraries (DTLSSocket, psutils, cpuinfo, jsonrpc-async and others) again.

@AnderssonPeter

This comment has been minimized.

Copy link

commented Nov 4, 2017

@linuxlurak I don't think there is a issue with using GitHub for support, but please create a new issue if it's not related.

@4tnight

This comment has been minimized.

Copy link

commented Nov 4, 2017

Thx @linuxlurac for your reply. I installed the home-assistant update and everything is working now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.