Continuous transition management

[Aqoush]

First off, thanks for creating tools related to controlling Hue lights. I've previously used f.lux to manage lights to transition my lights throughout the day but am exploring more customizable options.

As a disclaimer I haven't used hue-scheduler yet but plan to play around with it and read through the documentation.

I'm interested in creating a configuration where when lights are turned on they are set to a setting relevant to the sun's position.

In the current discrete state configuration it's my understanding that hue-scheduler will identify the state matching the current time, set the lights to the defined parameters, and then start transitioning to the next state if there exists a tr-before value defined.

For example in the example configuration:

Office    civil_dawn   ct:6500  bri:150  tr:10s 
Office    sunrise      ct:5000  bri:200  tr-before:90min
Office    12:00        bri:255  tr-before:90min
Office    sunset       ct:3000  bri:200  tr-before:90min
Office    civil_dusk   bri:150  tr-before:90min
Office    23:59        ct:2000  bri:100  tr-before:90min

Let's assume sunset is at 18:00. If the Office light group is turned on at 16:00 then the Office lights will be set to ct:5000, bri:255 as hue-scheduler is taking control of the lights. Then 90 minutes before sunset (16:30) the lights will begin to transition to ct:3000, bri:200. However if the lights were turned on at 17:30 then the lights would still be set to the same ct:5000, bri:255 as the previous scenario and hue-scheduler will execute the same full state-to-state transition in 30 minutes to match the sunset state.

My ideal execution of the transition (when lights are turned on at 17:30) would be for hue-scheduler to calculate the mid-transition state, set the lights to that state, and then continue the transition to the next state. In other words, it doesn't matter when the light was turned on, the light configuration is based on the time of day (relative to the sun constant times).

Not to overload this issue but I'm assuming that tr-before cannot reference sun states (e.g. tr-before:civil_dusk-sunset). Also I believe a "solar_noon" constant would be useful :).

Interested in what you think about this proposed behavior.

[cswrd]

I was about to post the same thing at some point, too. Actually, I am using a workaround that does the job for now but requires more configuration and doesn't make use of tr-before. Instead, the desired light interpolation can be scheduled statically with baby step sized changes. Due to the flexibility of hue scheduler and it's easy configuration it can be done within minutes.

Let's assume you want to transition the brightness from 200 to 140 within 60 minutes before sunset. Instead of:

Office  sunset  br:140  tr-before:60min

you'd configure:

Office  sunset-60  br:199
Office  sunset-59  br:198
Office  sunset-58  br:197
...

Even larger baby steps are usually not noticable, especially when you add a little tr-before. That way you can also find your own tradeoff between a fast reaction time and the amount of configuration you want to maintain.

Of course, it'd be nice if this desired transition behaviour would be integrated, natively. For me it'd be sufficient if it'd be the only one (replace the current transition behaviour) or be configurable per schedule entry (having it as default would be preferable though).

[Aqoush]

This is a good, if verbose, workaround. I will try to experiment with it when I can.

I believe the current configuration states map more natively to the way the Hue devices expose their capabilities and what is available through the Hue app. The sort of continuous lighting change we're after is almost like a very long-running light effect. For this reason I'm not sure it would be a good fit for being the default but maybe an additional configuration type that could be supported.

In the end internally it could just look like this:
Continuous light configuration -> Automatically generated discrete states -> hue-scheduler execution

In the end it will always be discrete on some time-step. One minute intervals doesn't seem too bad.

[stefanvictora]

Thank you for your interest in the tool and thank you both for starting the discussion about light state transitions. Right now Hue Scheduler does not support any custom interpolations, but rather reuses the transition system already provided by Hue lights.

You were almost right about how the transitions, or more specifically the custom tr-before implementation works. The main difference to your interpretation is that hue scheduler basically just uses tr-before to modify the start time of the sate by substracting the duration of the transition from the defined start time. In addition to that, the scheduler then dynamically shortens the transition time based on the time the lights are turned on after the set time. So your "tr-before" example could also be written like this at the given points in time:

Office    sunset       ct:3000  bri:200  tr-before:90min

# Equals at 16:30:
Office    sunset-90    ct:3000  bri:200  tr:90min
# Equals at 17:30:
Office    sunset-90    ct:3000  bri:200  tr:30min
# Equals at 18:00:
Office    sunset-90    ct:3000  bri:200

As a consequence this means that the scheduler does not explicitly set the previous state (again) when starting the transition but just keeps the current state of the light and uses the native transition feature of Hue lights.

My ideal execution of the transition (when lights are turned on at 17:30) would be for hue-scheduler to calculate the mid-transition state, set the lights to that state, and then continue the transition to the next state. In other words, it doesn't matter when the light was turned on, the light configuration is based on the time of day (relative to the sun constant times).

While you correctly identified the issue that hue scheduler currently does not support this kind of interpolation, I do like the approach and appreciate your suggestions and detailed descriptions of your uses cases. I will to look into adding some linear interpolations for tr-before over the next couple of days. If you have some more details on how you expect this feature to work, I'm happy to hear your thoughts. One issue that currently comes to mind is interpolating between states that use different color modes (i.e. ct vs hue and sat vs x and y).

Finally, I also have some small notes on the following statement of yours:

Let's assume sunset is at 18:00. If the Office light group is turned on at 16:00 then the Office lights will be set to ct:5000, bri:254 as hue-scheduler is taking control of the lights.

If we look at the relevant part of your example again:

Office    sunrise      ct:5000  bri:200  tr-before:90min
Office    12:00        bri:254  tr-before:90min
Office    sunset       ct:3000  bri:200  tr-before:90min

If the Office light group is turned on at 16:00, the lights will be just set to the maximum brightness level of 254. Since you didn't specify a color temperature for the "12:00" state, the scheduler will not modify any ct values of the lights at this point in time. If you want to make sure the ct is set to that value, e.g. after manually changing it at 11:00 and then resetting the manual override by turning off and on the lights at 13:00, you also need to specify the ct value in your state configuration.

Not to overload this issue but I'm assuming that tr-before cannot reference sun states (e.g. tr-before:civil_dusk-sunset). Also I believe a "solar_noon" constant would be useful :).

And you are right, such tr-before definitions are currently not supported, but I could think about them at some point. But for now I would focus on improving how tr-before works in general.

And thank you for pointing out that there was no "noon" sun time constant, I will include that in the next release.

[stefanvictora]

Instead, the desired light interpolation can be scheduled statically with baby step sized changes. Due to the flexibility of hue scheduler and it's easy configuration it can be done within minutes.

And you are right, a workaround for this could be to just define all the interpolations yourself. Since there is no limit on the number of state you can define, it's absolutely possible to create your own continuous transitions like this. This would even work with dynamic sun times as you could just use the offset feature for that.

I'm however also interested in provide a less labor-intensive support for this natively.

I believe the current configuration states map more natively to the way the Hue devices expose their capabilities and what is available through the Hue app. The sort of continuous lighting change we're after is almost like a very long-running light effect. For this reason I'm not sure it would be a good fit for being the default but maybe an additional configuration type that could be supported.

You are right, the configuration of the light states directly map to the Hue API, with some additional support for features like tr-before and dynamic sun times. So one idea could of course be to create some different state configuration format or type to support these continuous transitions implemented by some of the popular alternatives.

To begin with, I would however look into improving the existing tr-before feature, but I'm open for suggestions on how the desired features could be implemented without requiring too much manual configuration, while still allowing great flexibility.

[cswrd]

My geometrical interpretation of the problem domain (1 and 2 are scheduler entries):

The blue and green behaviour are implemented. Red is desired in addition to green. Both are meaningful.

I agree with you, it should be possible to have either of them as default and let the other one apply via a new schedule entry keyword.
While the current behaviour is more pleasing to the internals, I assume the desired behaviour is what users are looking for, usually. It's also what brought me here btw.

I'm just thinking about how fast the light should keep up with the intermediate state really (the red arrow pointing to the top). Like the icing on the cake: an eye-pleasing transition (~500ms - 2000ms ?) might be preferable to an instantanous and potentially huge light change. edit: for Tradfri bulbs the answer is clear, though: literally 0.

[stefanvictora]

Hi, I wanted to let you know that I have now released version 0.9.0 of Hue Scheduler which adds interpolations between states as one of its main feature. Thank you @cswrd for providing the visualization of the problem. I managed to implement this behavior into the existing tr-before logic and added the additional interpolate:true property for convenience, if you want back-to-back transitions between your states without having to explicitly configure it. Basically interpolate:true acts as a more advanced version of tr-before as it dynamically uses the start of the previous state, all while considering day cross overs and day-of-week configurations.

To summarize, if you turn on your lights in the middle of the transition, Hue Scheduler now calculates the mid-transition point using the previous state and the already passed time, sets the interpolated value and continues the transition from this point on. This way it can ensure that the lights are always at the correct state, regardless when they are turned on.

Additionally, it's now possibel to reference absolute and solar times when using tr-before, so you can use the following configuration now:

Home  nautical_dawn      bri:160  ct:390  tr-before:20min          tr:3s
Home  civil_dawn         bri:254  ct:199  interpolate:true         tr:3s
Home  golden_hour        bri:254  ct:333  tr-before:15:00          tr:3s
Home  sunset             bri:170  ct:366  tr-before:golden_hour+5  tr:3s
Home  nautical_dusk      bri:40  x:0.6024  y:0.3433  interpolate:true  tr:5s

Note how its also possible to transition / interpolat between different color modes (CT to XY)

To solve the question regarding how long the transition time of the interpolated call should be, Hue Scheduler has a default value configured (400 ms), which can be adapted using the new command line argument --interpolation-transition-time. However, if the previous state already has a tr property defined, Hue Scheduler just reuses this value also for the interpolated call. In the above examples this would be tr:3s i.e. 3 seconds.

If your goal is to enable interpolations between all states per default, you can just provide the --interpolate-all command line flag, which implicitly sets interpolate:true for all states. This can however be overridden on a case-by-case basis for individual states by either setting interpolate:false or by providing a custom tr-before property, which always takes precedence over interpolate:true.

To work around the limited transition time of Hue lights (max ~100 minutes), Hue Scheduler automatically splits up long transitions into multiple chunks and uses the already mentioned interpolation feature to calculate multiple interpolation points to transition between. This also means that all interpolations are done using native Hue light transitions, without periodically setting their state every few seconds as e.g. Adaptive Lighting does. However, I may look into supporting other modes of interpolation / transition, if needed. As not all lights might support such transitions (e.g. Ikea Tradfri bulbs #5).

If you have any questions or feedback on this new behavior, feel free to reach out -- I'm happy to provide some more insights into the inner workings. And thank you again to both of you for all your valuable input and suggestions to further improve this tool. I really appreciate it! You can find the new release and full list of changes here: https://github.com/stefanvictora/hue-scheduler/releases/tag/v0.9.0

[cswrd]

@stefanvictora Thank you very much for this new version. I run it in my own Docker container again and it works perfectly so far. I was able to reduce the configuration a lot. interestingly, the tradfri bulbs have a noticeable faster response time now. Even faster than the original hue bulbs. While the philips ones still take about 3-4 seconds, the tradfri ones react within 2 seconds now.

[stefanvictora]

Hi @cswrd, thank you for the positive feedback! Glad to hear it's still working as expected for you and even helps you reduce the configuration effort. I hope the growing number of configuration properties are not getting too confusing. And that's interesting indeed, as I haven't really changed much that could improve such response time when physically turning on lights.

I was actually a bit worried about the Tradfri bulbs as the new interpolate functionality is using native transitions which as you reported are not without issues for the IKEA bulbs. Have you been able to workaround these issues? Or are you mainly transitioning single properties of the lights?

One possible future solution that came to mind was adding multiple interpolation approaches by e.g. updating lights every few seconds instead of using native transitions. Similar to what other tools like Adaptive Lighting are currently doing. But I haven't put much thought into this for now.

[cswrd]

Hi @stefanvictora I am still using the more verbose configuration for the tradfri bulbs with tr:0. Maybe they work faster now due to the new hue API you trigger and some changes you've made regarding group events?
The approach you mentioned is the only I could think of too. I guess Ikea will never fix their bug.

Yes, the configuration is getting more complex, now. I think it might get easier for users just by shrinking down the readme already. I tend to write long descriptions by myself.