Native Hue sensor: adjustable scan interval

[olafz]

Home Assistant release with the issue:
0.92

Last working Home Assistant release (if known):
N/A

Operating environment (Hass.io/Docker/Windows/etc.):
Docker

Component/platform:
hue

Description of problem:
Version 0.92 of Home Assistant adds support for native hue sensors, which is really great!

I used to have quite a few of these entries like this that read my motion-, lightlevel- and switch-sensors:

  - platform: rest
    resource: http://a.b.c.d/api/[key]/sensors/18
    value_template: '{{ value_json.state.buttonevent }}'
    name: 'Hue switch bedroom'
    scan_interval: 1

Home Assistant is polling the API of the Hue bridge every second. Key here is the scan_interval.

At the risk of being flamed in trying to add to a year-old discussion: for lights, it's fine to have a 5-second poll interval; but especially for the motion sensor, the default 5-second scan interval for the Hue component is (too?) long to be useful.

At the time of the mentioned issue, Hue only supported lights and anything else basically required it to run it as a custom component. Now that native sensors have been added to the Hue platform, would it be an option to reconsider the statement "the scan interval is not coming back"?

[ghost]

Hey there @balloob, mind taking a look at this issue as its been labeled with a integration (hue) you are listed as a codeowner for? Thanks!

This is a automatic comment generated by codeowners-mention to help ensure issues and pull requests are seen by the right people.

[balloob]

Scan interval is still not coming back. Instead we should wait for Hue to release this https://www.reddit.com/r/Hue/comments/agfqks/ama_with_george_yianni_founder_of_hue/ee76mmo?utm_source=share&utm_medium=web2x

[dasb00ter]

I just would like to make sure I understand before I roll back to a previous version. Now that the motion sensor is fully integrated in Home Assistant it defaults to the the scan interval of the hue component which is 5 seconds which makes it at best impractical for use to light a room when you walk into it. Is there a way to keep using the custom component with 0.92 without receiving the errors that I received upon upgrade? Hopefully the api is released soon...I guess

[warcanoid]

This integration is unusable, to slow for automation. Also the old custom component is still needed for switches.

[mitchellrj]

I just would like to make sure I understand before I roll back to a previous version. Now that the motion sensor is fully integrated in Home Assistant it defaults to the the scan interval of the hue component which is 5 seconds which makes it at best impractical for use to light a room when you walk into it. Is there a way to keep using the custom component with 0.92 without receiving the errors that I received upon upgrade? Hopefully the api is released soon...I guess

That would be up to the maintainers of the customer component to address. Could you share a link to it?

too slow for automation

That depends on your use case, but I see your point. If you really need to override the scan interval, you could add a customer component to monkey-patch the SCAN_INTERVAL value, but from @balloob's comment, I don't think it is going to be something that will be supported officially.

[Mariusthvdb]

@dasb00ter

Is there a way to keep using the custom component with 0.92 without receiving the errors that I received upon upgrade?

Not sure about your errors, but using the new integration alongside the CC is posing no issues what so ever on this side. Motion sensors are quick as ever. Have them listed with and both work fine. (have 12)

as for the responsiveness of the new integration sensors: fwiw, in my setup they are changing state as swiftly as the CC's sensors in the frontend, must admit I havent yet changed setup to see if they turn-on lights with a noticeable lag. Sometime fronted lags behind live action.

these 5 seconds have always been a miracle of sorts, since the sensors report motion if and when happening. Also within the timeframe of 5 seconds

[andrewrobinsonuk]

Scan interval definitely too slow for sensors to be usable. Rely heavily on them for light automation and the delay is much too long.

[fondberg]

@balloob Waiting for https://www.reddit.com/r/Hue/comments/agfqks/ama_with_george_yianni_founder_of_hue/ee76mmo?utm_source=share&utm_medium=web2x will be a long waiting game. The comments after the one in the link (which is 110 days old) states lack of a standardized communication protocol.

Wouldn't it be possible to reduce the 5s interval today 1s to at least get the motion sensors usable?

[balloob]

It will generate a lot of traffic and Hue hubs also have a max reqs/second, so it might impact turning on lights etc. We should not go below 5 seconds.

[olafz]

Hue hubs also have a max reqs/second

Can you share documentation about that? Because what I have now (and what I had in the past before the upgrade) is quite a few entries like these (one per sensor and switch):

 - platform: rest
    resource: http://hue.ip.address/api/[key]/sensors/18
    value_template: '{{ value_json.state.buttonevent }}'
    name: 'Hue switch bedroom'
    scan_interval: 1

So, in addition to the new integration I also have plenty of sensors that are individually hitting my Hue hub with rest requests - every second.

It will generate a lot of traffic

I just checked briefly; I have 10 Hue sensors/switches in Home Assistant (like the one above). All have a scan_interval: 1. So I assume I'm currently doing 10 requests every second to the Hue hub?

That is, apart from the "once every 5 seconds" of the new integration.

So, old style I'm doing a lot more than I would with a lower interval in the new integration?

[fondberg]

@balloob many of us have been running the custom component successfully for quite some time and that has 100ms as interval. As these requests should be async local in the users home I don't think a scan interval of 1s second would hurt anything OR let the user change this through a configuration. What will happen otherwise is that the custom component will resurface and continue with the 100ms interval polling.

Ref: https://github.com/robmarkcole/Hue-sensors-HASS/blob/master/custom_components/huesensor/binary_sensor.py#L24

I can offer to make this configurable and create a PR including updateding any documentation with appropriate warnings aso, unless you have string opinions against

[nikolak]

@balloob

From what I can gather the limit seems to be 10 requests per second. I honestly don't really see how reducing the polling interval will impact lights turning on in a meaningful way, as far as I understand it (and someone please correct me if I'm wrong), turning on lights would result in individual calls.

Oversimplifying it by ignoring other calls being made and focusing just on two, sensor update and lights turning on - but the principle is the same:

So reducing the update to 1 second, would basically impact the users ONLY if they have exactly 10 lights that they're controlling - if they have more they already run into rate limits, and if they have less it will not impact them. On top of that for those users with exactly 10 lights they can be impacted already if they make the request at the same time as the polling for sensors.

But let's say that it does in fact impact users, and one request takes longer to process - the delay would be 100ms, if we honor the rate limit and send basically one request every 100ms not to exceed 10 requests per second (when for example there's 15 in queue).

I honestly think that adding 100ms to a request under specific circumstances, is an acceptable compromise to have motion sensors be usable for, what I'd argue is their main purpose that is automation.

Even if we add other calls being made to the mix, we're still "only" adding 100ms delay in case of multiple calls being sent, for an important feature.

Ideally this would be configurable for users that might want some different values (I'd personally lower it to 100-500ms myself).

Disclaimer: I'm not familiar with hass code and these assumptions may be flawed.

[Mariusthvdb]

@olafz

So, in addition to the new integration I also have plenty of sensors that are individually hitting my Hue hub with rest requests - every second.

maybe slightly off topic, pardon me if it is (still, its all about requests to the Hue hub, so rather relevant):

exactly for that reason the CC is so great to use. It only requests the Hub once and updates all sensors and switches. No more need for individual rest sensors.

fwiw: doing away with all my individual rest sensor (I had them for all switches and sensors), did improve on the Hue hub being unavailable in HA a lot! So you might want to be sparse with them.

btw, the CC is configurable, in that you can set the update interval in the component code yourself:

SCAN_INTERVAL = timedelta(seconds=0.1)

[balloob]

fwiw: doing away with all my individual rest sensor (I had them for all switches and sensors), did improve on the Hue hub being unavailable in HA a lot! So you might want to be sparse with them.

This so much here. I personally got a lot of attacks and shit for the hub being unavailable all the time, my code being bad. Turns out that people are just hitting the hub with a ton of requests all the time. We should not increase the request rate, as it will get us unavailable hubs again, and I get all the blame.

It's not our fault has an Hue that is not suitable for motion sensors. If you want to have a super fast rate, feel free to use the custom component or feel free to contribute a request manager for Hue that maintains rate limit, backs off sensor requests, prioritizes commands etc. But don't come in with requests to just lower the scan interval, as that's not a solution.

[Mariusthvdb]

Hi @balloob ,

Hope I read you correctly, and you are NOT in disagreement of my advise against the use of plenty rest sensors, and that I am most certainly not requesting the lowering of scan interval...

There seem to be 2 discussions going on related to scan interval:
1.- about the native sensors apparently needing a higher scan interval to be reliable motion sensors ( not agreeing on that yet here, mine are perfectly reliable, both native Hue/Hassio and CC).
2.- about scanning the Hub causing the lights and Hub (note, not light groups...) to go unavailable at frequent times.

The latter we discussed in depth without it ever being resolved. As a matter of fact, just today I experienced serious effects of that, and not to pollute this thread, point to the post on the community, which I hope you will find the time for to read it: https://community.home-assistant.io/t/philips-hue-changed-to-unavailable/65556/71?u=mariusthvdb

Cheers!

[johansmitsnl]

Why not make it adjustable with a note in the documentation about the availability. I ran the custom component for some time without any issues. The default could still be 5 seconds but let users adjust if they want.

[Rmafive]

Why not make it adjustable with a note in the documentation about the availability. I ran the custom component for some time without any issues. The default could still be 5 seconds but let users adjust if they want.

This seems like the simplest solution, at least for the short-term. If Phillips does release an update to the API to allow outbound communication (to better support sensor automation), this could be rolled back.

[johansmitsnl]

Spoke to a collegae of mine and he confirms that with the V1 HUE bridge polling is an issue. But I have a V2 bridge and don't have issues when polling much more frequently. So for V2 bridges it seems to be no issue to reduce the time. This might be handy to note with the documentation.

[knudsvik]

Would very much appriciate lower scan interval as well. Lights triggered by motion detector cannot wait 5s

[dasb00ter]

@Mariusthvdb

Yes I currently have both the integration and the custom component going at the same time. If polling is an issue then this seems like a less than optimal set up. I don't , however, seem to be experiencing issues with this set up. To confirm though I have an actual delay in the activation of a light when switching to the integration motion sensors and it is alleviated by switching back to the custom component. I'm not holding my breath waiting for hue to release an open API.

[Mariusthvdb]

HI @dasb00ter

thanks, still breathing here too ;-) Philips has been asked, begged maybe, frequently, on their own community forum, to change their Api, and response has been weak....

I don't have issues though with the motion sensors yet. I wonder if people here really are experiencing 5s delay, or they are simply mistaking the polling time for response time, without actually seeing that in real life response.

[knudsvik]

HI @dasb00ter
thanks, still breathing here too ;-) Philips has been asked, begged maybe, frequently, on their own community forum, to change their Api, and response has been weak....
I don't have issues though with the motion sensors yet. I wonder if people here really are experiencing 5s delay, or they are simply mistaking the polling time for response time, without actually seeing that in real life response.

I experience everything from 0.1 - 5s delays which were not present when the Hue hub was controlling the lights. I guessed this was due to the fact that polling data was set at 5s intervals, sometimes I enter the room 1s before the scheduled data fetch, other times 4s before the scheduled data fetch which will seem like random lag to the person observing this.

[Mariusthvdb]

HI @knudsvik

which were not present when the Hue hub was controlling the lights

Just to be precise: Hue hub is still controlling the lights, the Hue HA integration perse doesn't change that. Only if you set the motion-sensors to do nothing in the App, and have the lights turned_on/off in HA automations based on detected motion in HA, could any lag be suffered.

Is this the case in your setting?
For testing purposes I made a group/card with the CC sensors and the new native sensors, and checked whether they had different response times. Not that the front-end is in any way an exact measurement tool, but timing differences didn't worry me, and never got to the level you report. Which is bad of course.

Would you maybe try that too and see? Fwiw, in the CC the timing can be changed, so you'd be able to experiment with that. Because of the ridiculous amount of timing issues that were caused by the 0.1 frequency before, people did up that frequency, and didn't notice any negative results in response time. Maybe test in your own setting?

[andrewrobinsonuk]

Echoing @knudsvik - sometimes it’s instant, others I’m half way up/down the stairs in the dark before the lights kick in. It does appear “random” but I know it’s whenever it polls for updates.

…

On 21 May 2019, at 10:19, Marius ***@***.***> wrote: HI @knudsvik <https://github.com/knudsvik> which were not present when the Hue hub was controlling the lights Just to be precise: Hue hub is still controlling the lights, the Hue HA integration doesn't change that. Only if you set the motion-sensors to do nothing in the App, and have the lights turned_on/off in HA automations based on detected motion in HA, could any lag be suffered. Is this the case in your setting? For testing purposes I made a group/card with the CC sensors and the new native sensors, and checked whether they had different response times. Not that the front-end is in any way an exact measurement tool, but timing differences didn't worry me, and never got to the level you report. Which is bad of course. Would you maybe try that too and see? Fwiw, in the CC the timing can be changed, so you'd be able to experiment with that. Because of the ridiculous amount of timing issues that were caused by the 0.1 frequency before, people did up that frequency, and didn't notice any negative results in response time. Maybe test in your win setting? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#23502>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACMJVZTW6QCIOVKNCFM5MHDPWO5DPANCNFSM4HI554GA>.

[dasb00ter]

@Mariusthvdb

Didn't the original maker of the CC say somewhere along the line the hue bridge was limiting it's answering to the polling so changing the interval scan beyond a certain point didn't have an effect.

Also I mentioned that I have a delay in when my lights turn on with the new integration after I enter a room because that is precisely what is happening. As it's mentioned elsewhere why wouldn't this be expected (but unusable behaviour). If a person enters a room at the beginning of a polling interval it will be another 4-5 second's before home assistant reaches out for the info from the hue bridge and actuates the lights. Or am I missing something. I think I need a custom hue component That replaces all of the hue integration. If I understand things correctly a custom component can override a Built in one but our custom component is only a partial one

[knudsvik]

HI @knudsvik

which were not present when the Hue hub was controlling the lights

Just to be precise: Hue hub is still controlling the lights, the Hue HA integration perse doesn't change that. Only if you set the motion-sensors to do nothing in the App, and have the lights turned_on/off in HA automations based on detected motion in HA, could any lag be suffered.

Is this the case in your setting?

Yes, you are correct. The reason for me to use the HA integration instead of only the Hue app is to change the brightness level after everyone in the house is in bed at night. Through the Hue app I can only change modes at a certain time.

For testing purposes I made a group/card with the CC sensors and the new native sensors, and checked whether they had different response times. Not that the front-end is in any way an exact measurement tool, but timing differences didn't worry me, and never got to the level you report. Which is bad of course.

Would you maybe try that too and see? Fwiw, in the CC the timing can be changed, so you'd be able to experiment with that. Because of the ridiculous amount of timing issues that were caused by the 0.1 frequency before, people did up that frequency, and didn't notice any negative results in response time. Maybe test in your own setting?

Sure I can test something in my setup, but I need some guidance.

[gonzotek]

HI @dasb00ter

thanks, still breathing here too ;-) Philips has been asked, begged maybe, frequently, on their own community forum, to change their Api, and response has been weak....

FWIW, George Yianni (Hue Founder) replied in a Reddit AMA in January that they've built something for the bridge to do outbound calls (specifically in reply to a question about motion sensor data), but it's still in private beta with their big partners, with no specific timetable:
https://www.reddit.com/r/Hue/comments/agfqks/ama_with_george_yianni_founder_of_hue/ee759rm/

At least we know it's actively being worked on.

[johansmitsnl]

Added a PR #24147 waiting for the CI to pass to finish the PR.

[johansmitsnl]

For the people interested in testing the PR is done.

[dasb00ter]

I'm interested! How does one go about utilizing. I'll hit up that link in the mean time to see if I can figure it out.

I'm clearly naive to the development of code procedure. It looks like a PR to have a scan interval reinstated into the official hue component was made. It looks like baloob isn't too happy about that. Maybe I have misunderstood. Or is this a branch of the main code. Sorry to have to ask. The hue component is a mainstay of ha so I could see how it might be a source of friction. Hopefully not though. Flexibility and standardization is a really difficult thing

[bturrie]

I've been looking through this thread after coming here from the Custom Component thread. As I said there, at a five second scan interval I'm nearly out of the room before the lights come on. That makes the native motion sensors essentially useless to me. I have about 30 hue bulbs and even the custom component takes a second or so to turn my lights on. Perhaps there's a bet of a queue. If so then the important thing is getting the request into the queue without overloading the hub. My question is why can't the sample rate be adjustable by the user.? Then they are responsible if they overload their hub, in which case they can back off on the sample rate. I certainly have no issue with the CC rate of .1 second. But if I did I'd either back off on the sample rate or buy a second hub exclusively for motion sensors. My concern at this point is that Rob Mark Cole will stop supporting the CC and then I'll be stuck with motion sensors unusable in HA

[frenck]

As in the issue linked in the OP and per the latest result of PR #24147, I can safely say that this is not going to be changed/happen.

Based on that, I'll go ahead and close this issue.

PS: Users looking for this kind of response times are probably better of using ZHA, deCONZ or Zigee2MQTT (Better choice for this purpose since those rely on push instead of polling).

[skylord123]

I just wrote an article about how to get around this delay by using Node-RED with Home Assistant:
https://skylar.tech/faster-hue-motion-events-in-home-assistant-using-node-red/

I hope this helps people out.