Discrete command for turning the light bulbs off

[PetricaM]

Hi Chris,

Is there any way to distinguish, for a physical Milight remote, between a package to turn bulb off and a package sent by a remote that was previously used to send off command?

Specific case: bulb is associated to remote 1 (R1) and remote 2 (R2). After turning the bulb on with R1, if using R2 to send color change commands, then these commands are received by bulb. If pressing the off button on R2, then the bulb would turn off. However, if bulb is already
on from R1 and remote R2 was in off state, the command from R2 would not turn B to off (instead change to the actual color of R2) although MQTT topic published the off state.

After looking at several packages from R2 I think that the Command: 02 doesn't include a state command while Command: 01 does include the state (either on or off).

Below are some of the packets captured (the color settings are different because I couldn't press the exact same point on the color wheel; the expectation was that there was a difference in the full MQTT message).

i. Packet with a color command (will change the color of a bulb that is already on) although the MQTT topic shows state as off:

fut089 packet received (9 bytes):
Raw packet: EA BD BC 10 AB 2F 3A 6A B9

Decoded:
Key : EA
b1 : 25
ID : 550A
Command : 02
Argument : B3
Sequence : 45
Group : 01
Checksum : 0D

MQTT package:
Client mosqsub|15275-hass received PUBLISH (d0, q0, r0, m0, 'milight_gw/state/0x550A/fut089/1', ... (111 bytes))
{"state":"OFF","brightness":150,"hue":37,"saturation":58,"bulb_mode":"color","color":{"r":255,"g":198,"b":107}}

ii. Packet with OFF command (will turn the bulb off):

fut089 packet received (9 bytes):
Raw packet: BC 1C D2 65 A2 12 BA A2 9B

Decoded:
Key : BC
b1 : 25
ID : 550A
Command : 01
Argument : 0A
Sequence : 44
Group : 01
Checksum : C8

Client mosqsub|15275-hass received PUBLISH (d0, q0, r0, m0, 'milight_gw/state/0x550A/fut089/1', ... (112 bytes))
{"state":"OFF","brightness":150,"hue":253,"saturation":58,"bulb_mode":"color","color":{"r":139,"g":107,"b":255}}

Thanks!

[sidoh]

Heya Petrica,

If I'm understanding your question correctly, I don't think there's a way to do this. There's no way to link states, so espMH has no way of knowing that you're using the two IDs for the same bulb.

[PetricaM]

Hi Chris,

Actually it's about how the espmh publishes the state (let's forget the two remote IDs for a minute) as for different commands sent by the same physical remote, the MQTT message published on the state topic is the same, although the actual commands sent by the remote are different: first example of package (color change sent by a remote which was previously set to off and only changes the color of the bulb) has decoded as Command: 02 and the second package (off command sent by the remote and actually turns the bulb off) has Command: 01.

I can confirm that the two commands don't have the same effect; however, the MQTT state topic publishes the same message (ignore the different values for R, G, B).

Thanks!

[sidoh]

State should not change when the bulb is off. I guess I'm not seeing anything unexpected here, but I'm probably missing the point.

Perhaps it'd help if you posted a series of MQTT logs that produce the issue you're noticing, and edit to show what you expect to happen?

[PetricaM]

:) sorry, I think I wasn't clear: in both cases the commands are sent by a physical remote to a bulb that is on.

First command (decoded as Command: 02) will change the color.

The second command (decoded as Command: 01) will turn it off.

From MQTT state point of view the two commands are the same.

However, the actual behavior of the bulb is different and I think MQTT state discarded some info from the actual decoding.

[sidoh]

Gotcha.

What is the state prior to each of these commands?

[PetricaM]

Both physical bulb and MQTT states are "on" (MQTT state is also "on" as I forward the state from the different remotes ID the bulbs are paired to a single remote ID so that 2 or 3 different remotes control the same bulb and the states are in sync).

Indeed, case the physical remote is used to send command "on" then (from a logical point of view) the remote is in "on" state and next commands can be either:

1. change color ({"state":"on","color":{etc}}); changes color;
2. turn on ({"state":"on","color":{etc}}); however, the bulb won't react;
3. turn off ({"state":"off","color":{etc}); turns the bulb off;

If the bulb is "on" and the physical remote was previously used to send "off" command (obviously, this includes the other other remote ID in equation :) ) then the possibilities are:

1. change color ({"state":"off","color":{etc}}); changes color
2. turn on ({"state":"on","color":{etc}}); again, the bulb won't react;
3. turn off ({"state":"off","color":{etc}); turns the bulb off;

Thus my conclusions are:
a. choosing a color from the wheel of a physical remote will change the color of the bulb regardless of the state on/off published by espmh;
b. command "on" sent from a physical remote to a bulb that is already on has no effect for the bulb although the espmh publishes a different color in the state;
c. if the remote previously sent "off" command then the bulb will turn off only if pressing "off" button on the remote (choosing a color from an "off" remote will change the bulb).

My interest is in regard of item 'c' as espmh decodes differently the package for change color from a remote previously put to "off" vs. the command resulted from pressing the "off" button and the bulb behavior reflects it; however, in MQTT topic, the states published are the same ("off") and I think info is missing from the decoding process.

Obviously, a bulb that is physically "off" won't react to change color packages from a remote that was previously used to send "off" command :)

[sidoh]

ok, (1) shouldn't happen, and was probably a regression in 1.9. I'll take a look.

[sidoh]

oh, actually, i already fixed this. i just hadn't released a new version. i'll do that now.

[sidoh]

hopefully fixed in 1.9.0-rc.5. Let me know if I've misunderstood the issue you've noticed.

[PetricaM]

Well, actually this is not an issue, my understanding of the package types was not complete :)

I was confused about espmh publishing state as "on" or "off" for packages that were only in regard of changing colors (or brightness or saturation) as MQTT states published payloads were the same.

Based on my current understanding the packages sent by the remotes are of two types:
i. stateless (change color, change saturation, change brightness); doesn't include "state":"on" or "state":"off";
ii state change (turn_on or turn_off); doesn't include "brightness", "color" or "saturation" fields;

Espmh then publishes the MQTT state based on the state internally kept for each known remote ID: for i. it adds {"state:"on"} or {"state":"off") depending on espmh's internal kept state for that particular remote and for ii. it adds the fields selected in the webpage (such as brightness, color).

Using multiple remotes to associate with a single bulb is where the problem comes in: sending a state change package is correctly interpreted by espmh (decoded "Command:02") as such, however sending a stateless package forces espmh to publish state based on last updated state for the respective remote ID (and if another remote was used to turn the bulb on or off, then the bulb and the remote are no longer in sync).

In order to be able to use multiple remotes with same bulb (and keep them all in sync) espmh should associate all these remote IDs (and use a lot of aliases) which I don't think it is possible (or desired) as it would significantly increase the complexity of the code.

However, I think it can be easily accomplished by espmh publishing an additional field in MQTT state (something like "type") with values "stateless" or "state_changed" (something that doesn't conflict with the other fields). This would allow home automation software to override stateless packages (in HA would work something like below - assuming remote 0x1 is defined in HA and the bulb is also associated with the remotes 0x2, 0x3, 0x4; however, 0x2, 0x3 and 0x4 are not defined as lights).

The full state published by espmh would then be:
{"state":"on or off","type":"stateless or state_change","color":{"r":150,etc}}

and the automations, depending on packet type:

- alias: forward state
  initial_state: True
  trigger:
    - platform: mqtt
      topic: "milight_gw/state/0x2/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/state/0x3/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/state/0x4/rgb_cct/1"
  condition: 
    condition: template
    value_template: '{{ "state_changed" in trigger.payload }}'  
  action:
    - service: mqtt.publish
      data_template:
        topic: "milight_gw/state/0x1/rgb_cct/1"
        payload_template: >
          {{ trigger.payload}}

- alias: sync state
  initial_state: True
  trigger:
    - platform: mqtt
      topic: "milight_gw/state/0x2/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/state/0x3/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/state/0x4/rgb_cct/1"
  condition:
    condition: and
    conditions:
      - condition: state
        entity_id: light.bulb1
        state: 'on'
      - condition: template
        value_template: '{{ "stateless" in trigger.payload }}'
  action:
    - service: mqtt.publish
      data_template:
        topic: "milight_gw/state/0x1/rgb_cct/1"
        payload_template: >
          {{ trigger.payload | replace("OFF","ON") }}

[sidoh]

Ah, that was my original understanding of the issue. There actually are a couple of existing issues for this:

#198, #210

But I tend to agree: it adds a lot of complexity.

Based on my current understanding the packages sent by the remotes are of two types:
i. stateless (change color, change saturation, change brightness); doesn't include "state":"on" or "state":"off";
ii state change (turn_on or turn_off); doesn't include "brightness", "color" or "saturation" fields;

Have you considered using mqtt_update_topic_pattern? It includes only the decoded data for the packet, and none of the internal state. More detail here:

https://github.com/sidoh/esp8266_milight_hub#updates

[PetricaM]

Hi Chris,

Using update topic (partially) works. I'm leaving the automations case of interest for someone else.

There are basically two problems with this:

• using update topic it clobbers the MQTT broker quite a lot as there are now multiple messages from both update and state topics (until these tests I previously did not had setup the MQTT update as full state topic was enough);
• while the update topic for a state change works fine (first automation), for light properties like color or saturation doesn't work as HA expects a full rgb package for state not only "hue" or "saturation" (brightness and color_temp however work in individual packages alongside with "state":"ON"); the solution for the second automation would be to create a sensor that keeps the full state for each of the remotes 2, 3 and 4 and then forward state of that sensor to bulb1 (or creating bulb2, bulb3 and bulb4 corresponding to each remote and forward these bulbs properties to bulb1 only when bulb1 is on) but this requires creating additional entities.

I haven't found a way to use update topic as trigger and then to forward payload from full state topic without going through an additional step of storing full state topic.

I think that adding another (optional) field for the state topic as a flag that identifies a stateless package would be better way (as MQTT update topic would be completely eliminated and only changes in properties would be sent, without the need to create additional entities in the automation software) however you mentioned that it increases complexity.

- alias: forward state
  initial_state: True
  trigger:
    - platform: mqtt
      topic: "milight_gw/update/0x2/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/update/0x3/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/update/0x4/rgb_cct/1"
  condition: 
    condition: template
    value_template: '{{ "state" in trigger.payload }}'  
  action:
    - service: mqtt.publish
      data_template:
        topic: "milight_gw/state/0x1/rgb_cct/1"
        payload_template: >
          {{ trigger.payload}}

- alias: sync state
  initial_state: True
  trigger:
    - platform: mqtt
      topic: "milight_gw/update/0x2/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/update/0x3/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/update/0x4/rgb_cct/1"
  condition:
    condition: and
    conditions:
      - condition: state
        entity_id: light.bulb1
        state: 'on'
      - condition: template
        value_template: '{{ "state" not in trigger.payload }}'  
  action:
    - service: mqtt.publish
      data_template:
        topic: "milight_gw/state/0x1/rgb_cct/1"
        payload_template: >
          {"state":"ON",{{ trigger.payload | replace("{","") }} #in order to have a properly formed payload as HA requires also state field; if light.bulb1 is off there's no need to forward to its topic anyway.

[sidoh]

I don't think I understand what "stateless" means in this context. It's all state. If a message is published to the state topic, something changed. What are the semantics of this field? I'm happy to consider it, but I'm afraid I'm not following what you're getting at.

This is the solution I'd go with if I wanted to use remotes: https://github.com/sidoh/esp8266_milight_hub#updates

Basically:

• Messages sent to the updates topic for remotes (let's say they have IDs 0xA and 0xB are forwarded to the command topic for another ID (let's call it 0xC).
• Bulbs are paired with ID 0xC.
• Now the state for 0xC is the aggregation of updates from either remote, or commands sent directly to 0xC.

It does require that you have updates enabled, and I agree: they're noisy, and that might cause performance issues. But state is handled in a very clean way relative to trying to munge with it yourself.

[PetricaM]

I don't think I understand what "stateless" means in this context.

I should have mentioned that I consider stateless as a package for which no state change occurs (either from on to off or from off to on) thus the remote doesn't send it. Packages that are sent by the remote to change either of the light properties (color, color_temp, saturation or brightness) do not also have the "state":"on" or "state":"off".

• Messages sent to the updates topic for remotes (let's say they have IDs 0xA and 0xB are forwarded to the command topic for another ID (let's call it 0xC).
• Bulbs are paired with ID 0xC.
• Now the state for 0xC is the aggregation of updates from either remote, or commands sent directly to 0xC.

That's fine too, however it relies on multiple parts (HA, ESPMH, Wifi to connect ESPMH, connection between HA and router, etc) that build complexity and add to the potential of failure.

By having multiple physical remotes (let's say 3: one handheld - 0x1, one wall mounted 0x2 and another one that is used to associate all bulbs in the house - 0xA) associated with each bulb it allows for higher reliability case any of the components mentioned above fail and lights can be still used even if HA is off. I now have bulb1 set in HA with remote 0x1 and would basically want to sync 0x2 with 0x1 by showing 0x1 state (on or off) or properties (brightness, color, color_temp, saturation) to be updated in HA if remote 0x2 was used (this is because these two remotes control the same light). To achieve this I started forwarding full states from 0x2 to 0x1 (I only needed full state to be forwarded, touching on the command topic is not needed since the physical bulb already has the desired state; however, forwarding full state from full state topic of remote 0x2 to full state of remote 0x1 has a 50/50 chance of success as full state for remote 0x2 could be with "state":"off" thus the bulb 0x1 would be shown as off in HA although its physical state is on).

My proposal was to have a separate field published in the full state topic that would show if the package changes state (on/off) or only light properties (color, saturation, etc) as following:

I. Remote sends on packet, then ESPMH publishes in full state topic:
{"state":"on","type":"state_changed","color":{etc (the rest of the state remains unchanged)}
for update topic the payload would still remain:
{"state":"on"}

II. Remote sends off packet then, ESPMH publishes in full state topic:
{"state":"off","type":"state_changed","color":{etc (the rest of the state remains unchanged)}
for update topic (no change):
{"state":"off"}

III. Remote sends color change (or saturation, or color_temp, etc) packet then the following full states are the same from physical bulb point of view even if "state" is different:
{"state":"on","type":"state_unchanged","color":{etc"}
{"state":"off","type":"state_unchanged","color":{etc}

in update topic (no change):
{"hue":20} or {"saturation":100}

For cases I and II, full state from physical remote 0x2 can be safely forwarded to full state topic of remote 0x1 (the one created in HA) based on a condition like {{"state_changed" in trigger.payload}}.

For case III, based on a condition {{"state_unchanged" in trigger.payload}} and another one that checks the bulb state in HA, then if the bulb is on, the packet {"state":"on","type":"state_unchanged","color":{etc"} can be forwarded from full state topic of 0x2 to full state topic of 0x1 (otherwise HA automation can substitute off with on).

In conclusion: this is only to have in HA the representation of physical state of the bulb if multiple physical remotes are used (my first automation based on forwarding update topic from one physical remote to full state topic of another remote works with "state":"on"/"state":"off" however the second one doesn't update the color but only brightness and color_temp as state topic in HA expects full rgb not only hue or saturation; in command topic hue or saturation work fine).

[sidoh]

Thanks for clarifying :)

I should have mentioned that I consider stateless as a package for which no state change occurs (either from on to off or from off to on) thus the remote doesn't send it. Packages that are sent by the remote to change either of the light properties (color, color_temp, saturation or brightness) do not also have the "state":"on" or "state":"off".

I think the right way to think about this is that Milight command packets only change one thing at a time. Among the possible fields to change are state (perhaps less confusing to call this "status," haha), hue, saturation, etc.

Perhaps a more general version of what you're suggesting is a field that indicates which other fields changed from the previous state?

If that's the case, I'm not sure it's worth adding. The MQTT subscriber should already be aware of what the previous state is, and can perform the diff themselves if necessary. Am I following?

That's fine too, however it relies on multiple parts (HA, ESPMH, Wifi to connect ESPMH, connection between HA and router, etc) that build complexity and add to the potential of failure.

Any solution that solves the problem you've outlined is going to have a bunch of complexity. :)

[PetricaM]

I think the right way to think about this is that Milight command packets only change one thing at a time. Among the possible fields to change are state (perhaps less confusing to call this "status," haha), hue, saturation, etc.

Fully agree. However, I was referring to packets from full state topic only (not to packets that are sent to command topic) and another (optional) field would be published if chosen.

Perhaps a more general version of what you're suggesting is a field that indicates which other fields changed from the previous state?

If that's the case, I'm not sure it's worth adding.

Fully agree, I don't know what would be the benefits and how this info could be used in automation software.

The MQTT subscriber should already be aware of what the previous state is, and can perform the diff themselves if necessary. Am I following?

I don't think it is possible for two remotes paired to the same bulb as HA cannot identify that these two remotes are, in fact, controlling the same physical bulb.

In the initial message I've also posted two examples of packets sent by the same remote and (meanwhile :) ) I've found that Command: 01 changes status :) from on to off and Command: 02 changes color.

Command: 01 results in update topic in:
delta: {"state":"OFF"}
full: {"state":"OFF","brightness":150,"hue":253,"saturation":58,"bulb_mode":"color","color":{"r":139,"g":107,"b":255}}

while Command : 02
delta: {"hue": 37}
full: {"state":"OFF","brightness":150,"hue":37,"saturation":58,"bulb_mode":"color","color":{"r":255,"g":198,"b":107}}

Full state topics are the same for these two commands and these cannot be easily forwarded to another light state as there's a 50/50 chance the state would be incorrectly shown as off. By adding the status/state changed/unchanged only package from Command: 01 can be selected for forwarding as it is, while the Command: 02 packet can be manipulated to {"state":"ON","brightness":150,"hue":37,"saturation":58,"bulb_mode":"color","color":{"r":255,"g":198,"b":107}} case the other remote ID (used for HA association) is ON if that additional field that tracks the packet type is included.

[PetricaM]

I realized things got very complicated when it should have been quite simple so let's scrap this :)

Was trying to keep things done in HA only (in order to have the least possible effort put on ESPMH). However, forwarding full state from a physical remote topic to another just to have them sync won't do: after pressing any button on one of the remotes, the other remote's full state topic won't be updated and forwarding them will result in a complete mess as none has the correct state.

Simply, in order to have them all in sync, update topic from the other physical remotes need to be forwarded to the command topic of the light defined in HA and ESPMH will update the full state topic of this light.

This is actually the solution you mentioned with multiple physical and virtual remotes and initially I thought that it will result in light flickering; however, it will not change the physical state of the light (as it has already changed from the physical remote) and will only result in the last command being sent to the bulb.

Now just remains the color fidelity reproduction when sending saturation: 0 in color mode as ESPMH treats this as a white light (255,255,255) and all packages afterwards are the same :)

- alias: forward state
  initial_state: True
  trigger:
    - platform: mqtt
      topic: "milight_gw/update/0x2/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/update/0x3/rgb_cct/1"
    - platform: mqtt
      topic: "milight_gw/update/0x4/rgb_cct/1"
  action:
    - service: mqtt.publish
      data_template:
        topic: "milight_gw/0x1/rgb_cct/1"
        payload_template: >
          {{ trigger.payload}}