Hue power-on settings

[ebaauw]

Looks like the new firmware for the Philips Hue lights is out, at least here in the Netherlands: https://www2.meethue.com/nl-nl/ondersteuning/versie-informatie/lamps-luminaires.
Note that Philips do a phased rollout - It might take some time before it will be available world wide.

This is the firmware that allows setting a power-on state, i.e. a state for lights after power to it has been restored to it, other than the unholy full light. To enable this, the Hue API v1.28.0 now has a config.startup object, see https://developers.meethue.com/develop/hue-api/supported-devices.

I paired an enhanced color spot (LCT003) to my gen2 Hue bridge (already on firmware 1810251352, which is not mentioned in the release notes). It started updating the spot's firmware immediately. I've got the ZigBee sniffer running. I'm already seeing some interesting things:

• Apparently the Hue bridge has NWK address 0x0001, instead of 0x0000 ?!
• After the light sends a device announcement, the bridge broadcasts a manufacturer-specific 0x03 command to the 0xfc01 cluster;
• I seem to have missed the message where the spot gets its NWK address. [EDIT No I didn't - it's in the IEEE 802.15.4 layer, before the device announcement]. The bridge sends a Link Quality request to the spot, the spot sends a Route record and a Link Quality response. After that, the bridge starts polling the spot's 0x0006, 0x0008, and 0x0300 clusters.
• The bridge sends an OTA Image Notify to the spot; the spot answers with a default response, then does an extended address request and then an OTA Query Next Image request. After that, it starts downloading the image through OTA: image block requests. The Hue app (on iOS) shows that the firmware is downloading to the spot.
• After the download is complete, the Hue app shows an update button. On pressing that, the bridge broadcasts another 0x03 command to the 0xff01 cluster. Apparently that triggers the light to reset and activate the downloaded firmware (or I've missed another message?) - it no longer responds to the polling, until it sends a new device announcement.

The spot's /lights resource now looks like (my sorting):

{
  "capabilities": {
    "certified": true,
    "control": {
      "colorgamut": [
        [
          0.675,
          0.322
        ],
        [
          0.409,
          0.518
        ],
        [
          0.167,
          0.04
        ]
      ],
      "colorgamuttype": "B",
      "ct": {
        "max": 500,
        "min": 153
      },
      "maxlumen": 250,
      "mindimlevel": 5000
    },
    "streaming": {
      "proxy": true,
      "renderer": true
    }
  },
  "config": {
    "archetype": "spotbulb",
    "direction": "downwards",
    "function": "mixed",
    "startup": {
      "configured": true,
      "mode": "safety"
    }
  },
  "manufacturername": "Philips",
  "modelid": "LCT003",
  "name": "Hue color spot 1",
  "productname": "Hue color spot",
  "state": {
    "alert": "none",
    "bri": 254,
    "colormode": "ct",
    "ct": 366,
    "effect": "none",
    "hue": 14988,
    "mode": "homeautomation",
    "on": true,
    "reachable": true,
    "sat": 141,
    "xy": [
      0.4575,
      0.4101
    ]
  },
  "swupdate": {
    "lastinstall": "2018-12-08T09:46:14",
    "state": "noupdates"
  },
  "swversion": "5.127.1.26420",
  "type": "Extended color light",
  "uniqueid": "00:17:88:01:10:3c:9d:0a-0b"
}

Before the firmware update the spot's resource looked like (my sorting):

{
  "capabilities": {
    "certified": true,
    "control": {
      "colorgamut": [
        [
          0.675,
          0.322
        ],
        [
          0.409,
          0.518
        ],
        [
          0.167,
          0.04
        ]
      ],
      "colorgamuttype": "B",
      "ct": {
        "max": 500,
        "min": 153
      },
      "maxlumen": 250,
      "mindimlevel": 5000
    },
    "streaming": {
      "proxy": true,
      "renderer": true
    }
  },
  "config": {
    "archetype": "spotbulb",
    "direction": "downwards",
    "function": "mixed"
  },
  "manufacturername": "Philips",
  "modelid": "LCT003",
  "name": "Hue color spot 1",
  "productname": "Hue color spot",
  "state": {
    "alert": "none",
    "bri": 254,
    "colormode": "ct",
    "ct": 366,
    "effect": "none",
    "hue": 14988,
    "mode": "homeautomation",
    "on": true,
    "reachable": true,
    "sat": 141,
    "xy": [
      0.4575,
      0.4101
    ]
  },
  "swupdate": {
    "lastinstall": null,
    "state": "transferring"
  },
  "swversion": "5.105.0.21536",
  "type": "Extended color light",
  "uniqueid": "00:17:88:01:10:3c:9d:0a-0b"
}

[ebaauw]

Fingers crossed:

$ ph -H philipshue put /lights/8/config '{"startup": {"mode": "powerfail"}}'

Wireshark shows the following:

• The hue bridge sends a Write Attributes command to 0x0006/0x4003 for enum-8 value 0xff. The spot responds with Success;
• I missed a Write Attributes command to the 0x0008 cluster - only captured the Success response;
• The bridge sends a Write Attributes to 0x0300/0x4010 for u16 value 0xffff. The spot responds with Success;
• The bridge sends a Write Attributes to manufacturer specific 0x0300/0x0003 (Current X if not manufacturer specific) for u16 value 0xffff. And similar one for 0x0300/0x0004 (Current Y). The plug responds to both commands with Success.

I cannot find 0x0006/0x4003 nor 0x0300/0x4010 in the ZCL spec nor in the ZLL spec. Yet, the bridge and spot handle them as non-manufacturer-specific attributes. I would theorise they define the power-on state, where 0xff / 0xffff means previous value. @manup do you have the latest ZigBee documentation? I'll try and add them to general.xml anyways.

It would seem they made writeable, manufacturer-specific variants for Current X and Current Y, because there's no standard (yet?) for power-on values of these. @manup I don't think the deCONZ GUI will be able to handle a cluster with standard and manufacturer-specific attributes using the same attribute ID?

The light resource now looks like:

{
  "capabilities": {
    "certified": true,
    "control": {
      "colorgamut": [
        [
          0.675,
          0.322
        ],
        [
          0.409,
          0.518
        ],
        [
          0.167,
          0.04
        ]
      ],
      "colorgamuttype": "B",
      "ct": {
        "max": 500,
        "min": 153
      },
      "maxlumen": 250,
      "mindimlevel": 5000
    },
    "streaming": {
      "proxy": true,
      "renderer": true
    }
  },
  "config": {
    "archetype": "spotbulb",
    "direction": "downwards",
    "function": "mixed",
    "startup": {
      "configured": true,
      "mode": "powerfail"
    }
  },
  "manufacturername": "Philips",
  "modelid": "LCT003",
  "name": "Hue color spot 1",
  "productname": "Hue color spot",
  "state": {
    "alert": "none",
    "bri": 254,
    "colormode": "ct",
    "ct": 366,
    "effect": "none",
    "hue": 14988,
    "mode": "homeautomation",
    "on": true,
    "reachable": true,
    "sat": 141,
    "xy": [
      0.4575,
      0.4101
    ]
  },
  "swupdate": {
    "lastinstall": "2018-12-08T09:46:14",
    "state": "noupdates"
  },
  "swversion": "5.127.1.26420",
  "type": "Extended color light",
  "uniqueid": "00:17:88:01:10:3c:9d:0a-0b"
}

[ebaauw]

@manup, here's the Wireshark capture. The network key is in packet 66.
hue.pcapng.zip

As I've indicated above, I don't think I've captured all packets, but it does give some insight into the working. You might also want to checkout the Group Membership packets: the spot is assigned to the Living Room group (/groups/1), not to any others, but it's a member of two ZigBee groups (the other being the /groups/0 group).

[ebaauw]

Moved the spot to my test network, for testing general.xml. I can set the power-on on/off state from the deCONZ GUI! However, there seems to be a bug in the deCONZ GUI: when an enum8 attribute contains non-continuous values, and a value after the gap is written, a malformed write command is generated, with the value missing.

The following works for writing Off or On, but not for writing Previous:

<attribute id="0x4003" name="PowerOn OnOff" type="enum8" access="rw" default="0x01" required="o">
  <value name="Off" value="0x00"></value>
  <value name="On" value="0x01"></value>
  <value name="Previous" value="0xff"></value>
</attribute>

[ebaauw]

Found the PowerOn attribute for the Level Control cluster: 0x0008/0x4000. A uint8, also standard. Managed to set it through the deCONZ GUI as well.

And as I expected, 0x0300/0x4010 is the PowerOn colour temperature. Also settable from the deCONZ GUI. The manufacturer-specific attributes 0x0300/0x0003 and 0x0300/0x0004 for Current X and Current Y are also settable from the deCONZ GUI, but I need to comment out the standard attributes in general.xml. Otherwise, they're shown alright, but double-clicking on them pops up the attribute dialogue for the standard attribute.

[ebaauw]

Fun fact: the current (pre-Dec 2018) Hue firmware already supports PowerOn Color Temperature.

[ebaauw]

More fun: the IKEA Trådfri and ubisys D1 seem to support the PowerOn OnOff and PowerOn Level attributes. They really must be standard, then. The Trådfri colour temperature bulb does not support PowerOn Color Temperature, however.

My innr, OSRAM, and GLEDOPTO lights don't support any of the PowerOn attributes. Of course, OSRAM have their own way of setting the power-on behaviour.

[ebaauw]

Hm, the Trådfri supports the attributes, but doesn't honour them. Also, they're reset after power-cycling the light. It looks like it always sets the light on when power is re-applied, and that the level restores to the previous state (so as if 0x0006/0x4003 has value 0x01, and 0x0008/0x4000 has value 255).

The attributes on the ubisys D1 seem fully functional.

[oleg77s]

Software update (Today, Russia):
Hue White light LWB010 SW: 1.29.0_R21169 -> 1.46.13_R26312
Hue Ambience light LTW010 SW: 1.29.0_R21169 -> 1.46.13_R26312
Hue Ambience candle LTW012 SW: 1.29.0_R21169 -> 1.46.13_R26312

Hue White light LWB004 SW: 5.105.0.21169 no update
Hue Color light LCT001 SW: 5.105.0.21169 no update

[ebaauw]

I did receive new firmware for all my Hue lights:

• Dimmable white E27 bulb (LWB006): 5.127.1.26420;
• Candle ambiance E14 (LTW012): 1.46.13_r26312;
• Ambiance spot GU10 (LTW013): 1.46.13_r26312;
• Color spot GU10 (LCT003): 5.127.1.26420;
• Color bulb E27 (LCT007): 5.127.1.26420;
• Hue Bloom (LLC011): 5.127.1.26581;
• Hue lightstrip (gen 1) (LST001): 5.127.1.26581.

[oleg77s]

A new FW for Hue Color light LCT001 SW: 5.105.0.21169 -> 5.127.1.26581

[ebaauw]

New version of the Hue app is out now as well, supporting setting the power-on behaviour to default vs Power-loss recovery (i.e. previous value). The API is more elaborate, but quite straightforward:

$ ph put /lights/39/config '{"startup": {"customsettings": {"bri":127, "xy": [0.6376, 0.3563]}}}'
$ ph get /lights/39/config
{
  "archetype": "huebloom",
  "function": "decorative",
  "direction": "upwards",
  "startup": {
    "mode": "custom",
    "configured": true,
    "customsettings": {
      "bri": 127,
      "xy": [
        0.6376,
        0.3563
      ]
    }
  }
}

The Hue app shows the custom settings, but doesn't allow you to set them.

[Larsn1]

Just a short question: Is it possible to do the Firmware update for TRÅDFRI or hue with Deconz or do you connect the bulbs to the hue bridge or Gateway for that?

[ebaauw]

Trådfri: yes, Hue: no. Philips/Signify haven’t published their firmware.

[oleg77s]

Hue White light LWB004 SW: 5.105.0.21169-> 5.127.1.26420 updated too
Hue go LLC020 SW: 5.38.3.19141 no update
Hue lightstrip LST001 SW: 5.23.1.13452 no update

[mwhrtin]

So the IKEA Trådfri bulbs can in theory support restoring to the previous color on power-on but currently (due to firmware) they aren't respecting the values?

[ebaauw]

I'm afraid it might be the other way around: the firmware seems to supports it (for on/off and brightness), but the hardware doesn't. The bulbs seem to lack (enough) non-volatile memory to keep the settings across power cycles. Same for the bindings: they're lost across power cycles. But groups are preserved, so they do have some non-volatile memory.

[Alxandr]

I just bought my first set of hue bulbs and using the deconz gateway with home-assistant, and I have a few questions related to this:

First off, from what I read previously in this issue, it would seem I need a hue bridge in order to do firmware updates, is that correct? And if so, is this the right way to do firmware updates?

1. Reset bulb
2. Connect bulb to hue bridge
3. Update FW
4. Reset bulb
5. Connect bulb to deconz
6. Configure power on state either in deconz gui (if supported), or though the ph utility I see mentioned here (not sure what that is yet).
7. ???
8. Profit.

Also, what is the ph cli utility mentioned here?

[ebaauw]

I need a hue bridge in order to do firmware updates

Yes.

Configure power on state either in deconz gui (if supported), or though the ph utility I see mentioned here (not sure what that is yet).

ph is just a command-line utility to interact with the deCONZ (or Hue) REST API. It is bundled with homebridge-hue, but you can also install it standalone, see https://github.com/ebaauw/homebridge-hue-utils. It won't help you here, since the power-on settings aren't exposed yet through the deCONZ REST API.

If you want to change the settings to values not supported by the deCONZ GUI, and you're comfortable sending binary Zigbee commands, you could try https://github.com/ma-ca/deconz-cli-plugin.

[Alxandr]

And I'm assuming the settings gets lost as I move the bulb from the hue bridge to the deconz gateway?

[ebaauw]

Yes. The setting gets lost when you factory reset the bulb.

[SkorpionNL]

Is there any chance changing the power on behaviour settings will be possible in the Phoscon interface in the future?

[donnib]

@ebaauw do you know if anything has changed in regards to this with IKEA bulbs ? I mean the power-on setting.

I talked to an IKEA engineer some time ago and he said they will support this setting in all new devices that have Zigbee 3.0 but he could not say anything about the old ones. :(

[Alxandr]

Do you think it would be hard to enable the power-on support in the rest API? C++ isn't my strongest suit (by a long shot), but if the code here is one which I can understand, I might take a look at it.

[ebaauw]

It would be a serious amount of work, introducing a config endpoint for /lights resources. Since you typically set this up only once, I’m happy to do this through the GUI, so it’s at the bottom of my prio list.

I don’t think we’ll be able to support the colour settings without changes to deCONZ itself (due to the same attribute ID being used as a standard attribute as well as a manufacturer-specific attribute).

[ebaauw]

@ebaauw do you know if anything has changed in regards to this with IKEA bulbs ? I mean the power-on setting.

They’ll need to add support to their firmware. As far as I can tell they haven’t.

[Alxandr]

I’m happy to do this through the GUI

Wait, I thought there wasn't a GUI available to set this?

[ebaauw]

See my comments from Dec 8.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[donnib]

@ebaauw do you or somebody else have an idea on IKEA trådfri in regards to power-on. I have firmware 1.2.217 on my GU10 bulbs. It seems there are quite some updates https://ww8.ikea.com/ikeahomesmart/releasenotes/releasenotes.html but i don't know if the bulbs are updated to Zigbee 3.0 or i have to get new ones. As i mentioned earlier, the engineer at IKEA told me back in February that they will support this on all Zigbee 3.0 devices since it was mandatory but he could not say regarding current products.

[ebaauw]

I really din’t know. I only have a couple of Trådfri bulbs and they’re not connected at the moment. I haven’t seen any ZigBee 3.0 firmware update for the old (first-gen) lights.

[carver7]

@ebaauw do you or somebody else have an idea on IKEA trådfri in regards to power-on. I have firmware 1.2.217 on my GU10 bulbs. It seems there are quite some updates https://ww8.ikea.com/ikeahomesmart/releasenotes/releasenotes.html but i don't know if the bulbs are updated to Zigbee 3.0 or i have to get new ones. As i mentioned earlier, the engineer at IKEA told me back in February that they will support this on all Zigbee 3.0 devices since it was mandatory but he could not say regarding current products.

IKEA's transition from the Zigbee Light Link standard to the Zigbee 3.0 standard has definitely been happening. From what I understand, IKEA wants it's products (or it's light bulbs at least) to have as much compatibility with the Hue Bridge as Hue is willing to support.

I bought some IKEA lighting products last month, which included a pair of 1000 lumen white Tradfri bulbs, and a pair of Tradfri color bulbs as well.
My bulbs and every product that was bundled with them (like the little remote switches), definitely each got a big update through the Tradfri bridge/hub that I paired them with.

I confirmed via Google that the firmware versions were Zigbee 3.0 before unpairing them from the Tradfri bridge for exploring open-source platforms that support cross-compatibility
(such as our host's @ebaauw 's Homebridge-Hue here, Mozilla WebThings, Home Assistant, etc., by using a Conbee II as an alternative to the Tradfri or Hue bridges)

FYI as all of this is all very recently new to me, in case you find this helpful:
The Tradfri firmware updates from ZLL to Zigbee 3.0 can sometimes take days for people. On your cell phone it can look like the updates are stuck in confused, infinite loops.

My Tradfri bulbs and the Tradfri products that plug into mains outlets completed their updates within an ~1 hour, but the battery-powered little remotes and motion sensor that came with the bulbs took at least 3 days to complete with a couple of failures and restarts.

See this article for a bit more info:
How to get Ikea Trådfri smart light bulbs working with Philips Hue
https://www.the-ambient.com/how-to/ikea-smart-bulbs-on-philips-hue-app-255