Are Tradfri remotes supported without bulbs?

[axxelh]

I'd like to use Tradfri remotes, both the 5-button dimmer and the on/off switch distributed with the remote outlet. I have not yet paired the outlet itself with the remote or with HA. Both switches will pair with zha_new, but the remotes appear to generate no input once joined, either as events in the logbook, or at DEBUG log level in the HA logs.

This is HA 0.84.6 with todays (2019-01-02) ha-zha-new-master.

#36 seems to imply that a pairing with a bulb is required, is that still the case even for the simple on/off switch? Could we get documentation on the exact order of the pairings? Would the outlet serve a similar purpose as the bulb?

[Yoda-x]

for the 5 button remote the only way is currently to pair a bulb. The communication between remote and bulb is via a broadcast group. zha_new reads out the broadcast group from the bulb and then subscribes to this broadcast group.
the peering between bulb and remote happens via the zll near range feature which is not supported by zha_new or zha, As the the needed certificates are not public. its very likely that this will never be supported.
So, the only way to use the remote is to pair it with a bulb and zha_new, and also have the bulb paired with zha

I have no information about the on/off switch and if it uses the same way of communication.
Can you provide the zigbee.db and some debug logs for the on/off switch,

[axxelh]

Based on logs I made earlier, I believe the switch uses the same configuration (broadcast group) as the 5-button remote. I'll get a clean db and logs soon, I don't have the switch with a clean config at hand at the moment.

After opening the issue I had done a little more research into the broadcast group requirement. The remote does expose an output cluster id 4 (groups), but it doesn't appear to respond as expected to the zigpy.Groups methods. Notably, if I add code into the binary_sensor._make_sensor() function to call Groups.get_membership() on the remote the result is Status.UNSUPPORTED_CLUSTER. I see similar results for all other Groups methods. As I understand the Zigbee cluster docs, this seems unexpected, though I may be misunderstanding the library usage.

Recent work in zigbee2mqtt (Koenkk/zigbee2mqtt#102 (comment)) indicates that if zha_new can detect the group directly from the remote it should be possible to skip the bulb step since the group is apparently assigned at reset. Is this something that has been explored? In particular I'm wondering if the group ID you see at the bulb is related in any way to the remote's NWK address, as both are generated when the remote is reset. Since I don't have a bulb (yet) to test, I can't confirm if they are related.

Finally, just to make sure I'm reading the code right, it's the code in Light.async_update() that adds the controller to the group? Specifically the self._endpoint._device._application.listener_event() call?

[Yoda-x]

Hi,
you cant write/read to output clusters. the input clusters represents the server to which a client (output cluster) connects to.

If you look at bulbs and (I assume) the outlet, you find input clusters for groups(4). This can be changed. A (output) client can write to this (input) cluster and enter broadcast groups. But a client can't write to a client. There a group input cluster is needed.

Reading the zigbee2mqtt thread, they do a sniffing with a specific zigbee sniffer. They don't read it out from the remote it self. They snoop the traffic on the air with a different zigbee usb stick to find out the group id. you can do this for development and reverse engineering, but it's nothing for production.
The crux is to find out somehow the groupid the remote uses. I did also this kind of reverse engineering some month ago.

I will check if the groupid is somehow related to the nwk id.
EDT: groupid and nwkid are not related

right, this add the controller to the broadcast group :

                            self._endpoint._device._application.listener_event(
                                'subscribe_group',
                                groups)

This calls a low level function which writes the group to a specific register on the controller.

I can add a configuration item to zha_new for broadcast groups. This would enable users to subscribe well known groupids to zha_new. This is a simple task.
With this, a user would be able to find out somehow the broadcast group (either paring a bulb ) or snooping traffic, and then just enter the group numbers into configuration.yaml. With this, there would be no need to pair the remote with the bulb as it is now.

[axxelh]

Thanks for checking the nwkid and groups relationship, I had hoped it would be related.

The on/off switch bundled with the outlet has the same clusters as the 5-button remote, and appears to behave in the overall same manner. I'll skip the db since its so similar to the remote.

I can get the switch to pair with the outlet and another remote (undocumented feature apparently https://amp.reddit.com/r/tradfri/comments/6x1miq/bind_controller_to_another_controller_to_have/) when not paired to zha_new. I can control the outlet as expected.

Based on this success my assumption was the code to read and subscribe to groups that's used for lights should work for the switch as well. I can pair the button (and remote) to zha_new alongside the switch. However, once paired with zha_new, I cannot get the button (or remote) to peer with the outlet. Without this I can't read the correct group identifier for anywhere.

Before I give up, can you confirm my understanding of the light bulb config is correct? Specifically:

• Pair bulb with zha_new.
• Pair remote with zha_new.
• Peer remote to bulb by holding remote or switch button for >10s in close proximity to bulb.

Which then allows retrieving the group from the bulb?

[Yoda-x]

to be honest I don't understand your question completely

1. the code for light should be easily ported to the switches, it just reads out one cluster. If there is an input group cluster, this should be easy.
2. you need first to pair outlet and remote(button) to the hass, Then connect the outlet and remote/butom
   For the bulbs/remote I do the following.
3. Pair the remote : permit paring in hass, press 4 times the inside button in the remote
4. Pair the bulb: permit paring in hass, 6 times power off/on for the bulb
5. Pair bulb and remote: hold the remote near the bulb and press >10sec the hidden button inside the remote.

The remote has a "standalone" mode in which it acts as its own zigbee controller and a "connected mode" where it part of a zigbee network, connected to zha_new.

Did you get the outlet paired with hass?

[axxelh]

Sorry for not being clear.

I can get the outlet to pair with HA and work as a switch in the web UI. The outlet actually appears as two switches, only one of these controls works and changes the outlet state, the other throws an attribute error when used. I'll look into resolving that if I can get the other things working.

With the outlet and switch I can accomplish both steps 3 and 4 in your list. I can not accomplish step 5. I cannot get the outlet and switch to pair to each other after pairing with the controller.

Using the outlet and the 5-button remote I also can also accomplish steps 3 and 4, but 5 fails in the same way.

This led me to believe that the pairing rules for the outlet may be different than the bulbs as I can pair both the remote and the switch to the outlet when they are not paired to HA.

Thinking this was specific to the outlet, I actually purchased a Tradfri bulb today and tried to follow your steps 3-5 exactly (bulb and 5 button remote). I can pair the remote, but the bulb will not pair. The logs show the bulb device joining the network, but the "discovering endpoints" step times out.

I've reset the pairing (deleted the ZHA database and factory reset the devices) many times. Is there anything that might be lingering from all this testing that I need to do? Some reset of the HUSBZB stick?

[Yoda-x]

I just tested it with the tradfri remote, a tradfri bulb and an osram bulb:

• paired the remote to hass
• paired the bulb to hass
• paired remote to bulbs with 10sec press (bulb is blinking)
  -> working

pairing remote to bulb only works when both are connected to hass or both are disconnected.

For the HUSBZB stick were some faulty batches sold zigpy/bellows#120 (comment), but I don't think it's related to the stick, as the pairing with zha works.
And zha has no role with step 5.
That the bulb not pairs is strange, it's one of the easiest devices, just turn it on and it connects.
How many and what other devices you have in you zigbee network?

[axxelh]

These are my only Zigbee devices, there is nothing else on the network. I'll play with the bulb some more.

In the meantime, here's a DB with three Tradfri devices (outlet, on/off switch, and 5 button remote) joined. Perhaps this will at least let you see if the on/off switch meets your expectations.

Note that the outlet presents in the UI as two separate switches. One works as expected. The other dumps this error when used:

Traceback (most recent call last):
File "/homeassistant/deps/lib/python3.6/site-packages/zigpy/endpoint.py", line 182, in getattr
return self._cluster_attr[name]
KeyError: 'on_off'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/homeassistant/helpers/service.py", line 277, in _handle_service_platform_call
await getattr(entity, func)(**data)
File "/homeassistant/custom_components/switch/zha_new.py", line 78, in async_turn_on
yield from self._endpoint.on_off.on()
File "/homeassistant/deps/lib/python3.6/site-packages/zigpy/endpoint.py", line 184, in getattr
raise AttributeError
AttributeError

zha_new.db.zip

[Yoda-x]

1. the device type is missing from the code. I will make some adjustments.
2. i may found another option to get the group information via the commissioning cluster, but this need some coding. stay tuned.

[Yoda-x]

good news, I'm able to read out the group id from the touchlink commissioning cluster during paring of the remote. I need now to decide where to store the information so it's persistent during a restart.

[Yoda-x]

tradfri remotes are now work without paring to a bulb. please give it a try

[alexeinz]

i am sorry to reopen, but i am not sure how would I add the tradfri remote in the configuration yaml... _?
any example?

[Yoda-x]

Hi

There is no setting needed. If you have any, please remove it and do a full repair(remove from Hass and pair again)

[alexeinz]

i tried many times.. but thats all i get...
it actually appears in hass as device but thats all to it ( no buttons or such available )
Thread) [homeassistant.components.binary_sensor] Setting up binary_sensor.zha_new

2019-01-27 18:01:09 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 18:01:09 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 18:01:09 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 18:01:09 DEBUG (MainThread) [custom_components.zha_new.helpers] discover_group_identifier for 4096: []

2019-01-27 18:01:12 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [2, <Bool.false: 0>, 13556, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 18:01:12 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [2, <Bool.true: 1>, 38704, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 18:01:12 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [2, <Bool.false: 0>, 38704, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 18:01:12 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [2, <Bool.true: 1>, 15881, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 18:01:19 WARNING (MainThread) [homeassistant.components.binary_sensor] Setup of platform zha_new is taking over 10 seconds.

2019-01-27 18:01:19 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [2, <Bool.false: 0>, 15881, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 18:01:23 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.lumi_unknown_02c7b71b_1 is taking over 10 seconds

2019-01-27 18:01:23 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.deckdoor is taking over 10 seconds

2019-01-27 18:01:23 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.bedroomdoor is taking over 10 seconds

2019-01-27 18:01:23 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.lumi_unknown_02d467f1_1 is taking over 10 seconds

2019-01-27 18:01:23 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.lumi_unknown_02386205_1 is taking over 10 seconds

2019-01-27 18:01:23 WARNING (MainThread) [homeassistant.helpers.entity] Update of switch.kitchenwindow is taking over 10 seconds

2019-01-27 18:01:37 ERROR (MainThread) [homeassistant.components.binary_sensor] Error while setting up platform zha_new

Traceback (most recent call last):

File "/config/deps/lib/python3.6/site-packages/bellows/zigbee/application.py", line 381, in request

v = await asyncio.wait_for(reply_fut, timeout)

File "/usr/local/lib/python3.6/asyncio/tasks.py", line 362, in wait_for

raise futures.TimeoutError()

concurrent.futures._base.TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 128, in _async_setup_platform

SLOW_SETUP_MAX_WAIT, loop=hass.loop)

File "/usr/local/lib/python3.6/asyncio/tasks.py", line 358, in wait_for

return fut.result()

File "/config/custom_components/binary_sensor/zha_new.py", line 66, in async_setup_platform

await cluster.bind()

File "/config/deps/lib/python3.6/site-packages/zigpy/device.py", line 96, in request

expect_reply=expect_reply,

File "/config/deps/lib/python3.6/site-packages/bellows/zigbee/application.py", line 385, in request

raise DeliveryError("sendunicast reply timeout error")

zigpy.exceptions.DeliveryError: sendunicast reply timeout error

2019-01-27 18:01:40 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 18:01:40 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 18:01:40 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

[Yoda-x]

Seems you pressed the pair button on the back multiple times. Can you start the join process in Hass, then press 4 times the pair button and let it run. Please provide the logs from this try.
Thanks

[alexeinz]

did remove and 4 times press
i can see it coming to states ui.. but pressing nothin or even generates no log...
states ui:
model
TRADFRI remote control
manufacturer
IKEA of Sweden
lqi
255
rssi
-80
last seen
2019-01-27T13:56:52.805512+09:00
nwk
62911
path
direct
Group id
40900
Level
1
OnOff
0
Scenes
1, 1
last command
toggle

log entry:
`2019-01-27 23:39:14 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [6, <Bool.true: 1>, 15765, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 23:39:15 INFO (MainThread) [zigpy.application] Device 0x3d95 (90:fd:9f:ff:fe:76:91:51) joined the network via 0

2019-01-27 23:39:15 DEBUG (MainThread) [custom_components.zha_new] Device joined: 90:fd:9f:ff:fe:76:91:51:

2019-01-27 23:39:15 DEBUG (MainThread) [custom_components.zha_new] Device initialized: 90:fd:9f:ff:fe:76:91:51:

2019-01-27 23:39:20 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [6, <Bool.false: 0>, 15765, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 23:39:20 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: childJoinHandler : [6, <Bool.true: 1>, 62911, 90:fd:9f:ff:fe:76:91:51, <EmberNodeType.SLEEPY_END_DEVICE: 4>]

2019-01-27 23:39:20 INFO (MainThread) [zigpy.application] Device 0xf5bf (90:fd:9f:ff:fe:76:91:51) joined the network via 0

2019-01-27 23:39:20 DEBUG (MainThread) [custom_components.zha_new] Device joined: 90:fd:9f:ff:fe:76:91:51:

2019-01-27 23:39:20 DEBUG (MainThread) [custom_components.zha_new] Device initialized: 90:fd:9f:ff:fe:76:91:51:

2019-01-27 23:39:21 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: incomingSenderEui64Handler : [90:fd:9f:ff:fe:76:91:51]

/config/deps/lib/python3.6/site-packages/zigpy/util.py:28: RuntimeWarning: coroutine 'BinarySensor.device_announce' was never awaited

method(*args)

2019-01-27 23:39:22 DEBUG (MainThread) [custom_components.zha_new] read attribute: {'model': b'TRADFRI remote control'}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] read attribute: {'manufacturer': b'IKEA of Sweden'}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] manufacturer: type(<class 'bytes'>) b'IKEA of Sweden'

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] manufacturer: type(<class 'str'>) IKEA of Sweden

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] model: type(<class 'bytes'>) b'TRADFRI remote control'

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] model: type(<class 'str'>) TRADFRI remote control

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] discover_endpoint_info:{'manufacturer': 'IKEA of Sweden', 'model': 'TRADFRI remote control'}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf] device init for <class 'str'>(TRADFRI remote control)(90:fd:9f:ff:fe:76:91:51) -> Endpoints: [0, 1], new join

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:0] endpoint init

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] endpoint init

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] node config for 90:fd:9f:ff:fe:76:91:51-1: {}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] Import DH TRADFRI remote control success

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] custom_info for TRADFRI remote control: {'module': <module 'custom_components.device.tradfri_remote_control' from '/config/custom_components/device/tradfri_remote_control.py'>, '_custom_endpoint_init': None, '_custom_cluster_command': <function _custom_cluster_command at 0xaff0ee88>, '_parse_attribute': None, 'custom_parameters': None}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] pre call _custom_endpoint_init: {'module': <module 'custom_components.device.tradfri_remote_control' from '/config/custom_components/device/tradfri_remote_control.py'>, '_custom_endpoint_init': None, '_custom_cluster_command': <function _custom_cluster_command at 0xaff0ee88>, '_parse_attribute': None, 'custom_parameters': None}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] no call _custom_endpoint_init: TRADFRI remote control

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] node config for 90:fd:9f:ff:fe:76:91:51-1: {}

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] config reports skipped for 90:fd:9f:ff:fe:76:91:51, no reports configured

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] 2:profile 49246, component: binary_sensor cluster:[{0, 1280, 1030, 7}, {768, 1280, 4, 5, 6, 8}]

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf:1] Return from component general entity:90:fd:9f:ff:fe:76:91:51

2019-01-27 23:39:23 DEBUG (MainThread) [custom_components.zha_new] [0xf5bf] Exit device init 90:fd:9f:ff:fe:76:91:51

2019-01-27 23:39:23 INFO (MainThread) [homeassistant.components.binary_sensor] Setting up binary_sensor.zha_new

2019-01-27 23:39:24 DEBUG (MainThread) [custom_components.zha_new.helpers] discover_group_identifier for 4096: []

2019-01-27 23:39:27 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 23:39:27 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 23:39:27 WARNING (MainThread) [bellows.zigbee.application] Unexpected message send notification

2019-01-27 23:39:28 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved

Traceback (most recent call last):

File "/usr/local/lib/python3.6/site-packages/homeassistant/helpers/entity_platform.py", line 344, in _async_add_entity

raise HomeAssistantError(msg)

homeassistant.exceptions.HomeAssistantError: Entity id already exists: binary_sensor.ikea_of_sweden_tradfri_remote_control_fe769151_1. Platform zha_new does not generate unique IDs
`

[Yoda-x]

IMHO it's working.

Group is detected and the last command shows 'toggle'
The driver creates events for button presses. Beside this is it has a level and scene and on_off attribute,. If you keep the device details open you should see it changing when you press buttons

[alexeinz]

nope... i can record a video... it definitely communicating but buttons do nothing .., no log entries, nothing at all ... ( log set to debug for zha_new ) ....
if i just do long button press on the pairing button, i can have this response
homeassistant.components.sensor] Updating ring sensor took longer than the scheduled update interval 0:00:30

2019-01-28 07:18:32 INFO (MainThread) [bellows.zigbee.application] Unused callhander received: macFilterMatchMessageHandler : [1, <EmberMacPassthroughType.MAC_PASSTHROUGH_INTERNAL: 128>, 255, -76, b'\x01\xc8a\xff\xff\xff\xff\x8eLQ\x91v\xfe\xff\x9f\xfd\x90\x0b\x00\x0b\x00\x10^\xc0\x11\xc5\x00\xfcj\x17Z\x02\x12']
the screenshot is how i see it in gui.... ( nothing changes with button presses )

[Yoda-x]

Yepp. I see. Let me check. Possible a firmware problem on the device. I have two of this remotes at home. Let me check how my non-test remote looks

[alexeinz]

cheers, let me know if need any other log or scan

[alexeinz]

Hmm still nothing here... I do have 2351 based zqb scanner now... anything i can scan for you?

[Yoda-x]

Can you try my preview branch. I found some issues with remove/repair of devices due to changes in hass.
Beside this both of my tradfi 5button remote disks work fine without need to pair them to a bulb.
If you can scan during the pairing that would be great, just to see whats on the air.

[Yoda-x]

should now work better with tradfri remote