Some new config files Hue lights

[smonesi]

I'm trying to automatically create as many files for LUT config as possible using a modified measure.py script (using an TP-Link HS110 smart plug), however for some light I can only provide basic linear config.

[nepozs]

It wolud be nice if you can publish also modified script fot HS110 using name eg. measure_hs110.

[Leatherface75]

Would be nice with modified measure script for zwave and tuya plugs too.
Then maybe i can fix for my Philips Hue Lightstrips i have here.

[bramstroker]

@smonesi, thanks awesome job!

Few remarks.

• I see your power measurements in the CSV files have more than 2 decimals precision. Not sure if this would cause any issues somewhere. I don't do any rounding in my component (yet). I would prefer to convert these values to have 2 decimal precision as this is consistent across all CSV files and also makes them smaller.
• You are using much less measure points compared to other lights. for HS mode you have 1429 and most other lights 23167. This is because your step sizes of brightness and saturation are much larger. This will make measuring a lot faster but lacks precision. I'm not sure if this is precise enough as the component will choose the closest value and is not doing any interpolation. So for example your brightness goes from 1 to 51 to 101. Imagine the user sets his brightness to 73, than the power for 51 would be selected. This could be 0.3 watt difference for example. Not really sure what to do, as I don't want you to ask to redo all the lights (as this will take a lot of time). @nepozs what do you think?
• Could you add the new mandatory properties for measuring information to model.json? See adopt to new model schema #86 for example.

[nepozs]

It is really important to have consistent measurement base; it is even more important now, when HA has official "energy panel" available for every user.
So maybe it needs clear rules how to collect data.

Few remarks.

@smonesi @bramstroker

ad 1. No more than 2 decimals is really needed (and rather never be), as precision of devices used to measure isn't really big, but also there are other circumstances which distort the result.
@bramstroker I've found somewhere (one of LUT files, but I don't remember which one) notation without decimals - will this be interpreted as integer instaead of float? (if yes I will find it; I think I've always used 2 or 1 decimals notation so it must be file from other contributor)

ad 2. Brightness is most important part (so when I used to measure manually it was always 100 steps of brightness), in case of brigtness it is really sufficient (no need to use all 255 possible values), but due to nonlinear parts of power characteristics it is hard to predict how many measurements will be enough count .
Theoretically there is option to produce less dense files, but due to nonlinear character of color space it will be really hard to make proper script (it could eg. use predefined values table instead of loop with constant step).

ad 3. I'm not 100% sure if my files are OK, maybe better to add just example in documentation, and maybe new subdirectories for sc ript working with other platforms?

@Leatherface75
It would be nice, but it depends on contributors using their own scripts if they decide to pubish them or not.
Just not every contributor is programmer :D

[smonesi]

I will definitely fix the 2 decimals, I wrote a script that reads the CSV and writes it back rounded. I will always add decimals (as in sprintf %.2f) but I think no decimals should be supported.

The first time I run the script with the "default" settings (as in measure.py) it was too slow and I feared some automation would ruin the calculation (eg. the measured light temporarily turning off if I leave the house!), I really thought the component would interpolate between two values.

I wrote a script that interpolates the values and generates a new hs file with much more data, I am currently doing some sampling to verify how good the interpolation is and, for now, everything looks quite good. I'm actually more worried about the precision of the measurements of my HS110 (eg. hue changes should be completely irrelevant when saturation is 1, but the results are all slightly different)...

I can share those scripts together with the measure_kasa.py script, I'm not a python programmer so the code might now be as good as it should be!

[bramstroker]

I will definitely fix the 2 decimals, I wrote a script that reads the CSV and writes it back rounded. I will always add decimals (as in sprintf %.2f) but I think no decimals should be supported.

Ok great!

The first time I run the script with the "default" settings (as in measure.py) it was too slow and I feared some automation would ruin the calculation (eg. the measured light temporarily turning off if I leave the house!), I really thought the component would interpolate between two values.

Haha, you don't want that after 8 hours of measuring ;-). But you still can have this issue when the measure procedure takes 1-2 hours, right? I did disable my light automations in HA (toggle) while measuring so there was no risk this happens.

I wrote a script that interpolates the values and generates a new hs file with much more data, I am currently doing some sampling to verify how good the interpolation is and, for now, everything looks quite good. I'm actually more worried about the precision of the measurements of my HS110 (eg. hue changes should be completely irrelevant when saturation is 1, but the results are all slightly different)...

Nice!
Need to have another thought how to do the interpolation in the component. Because the lookup file is over 3 dimensions this is not really easy (at least I was not able to figure out the algorithm before). Since brightness is the most important factor I might have an idea how to do it. The relevant code is in

homeassistant-powercalc/custom_components/powercalc/strategy_lut.py

Line 114 in 3729aa6

if color_mode == COLOR_MODE_HS:

. So it also needs some refactoring.

I can share those scripts together with the measure_kasa.py script, I'm not a python programmer so the code might now be as good as it should be!

I am also not a Python programmer by heart ;-). Did a lot of PHP, C# and Javascript. But honestly this is my first Python project. Besides the measure scripts and utitlities need to do there job. maintainability or clean code is not the most important factor here.

[bramstroker]

You can set this to manual, and then description to "From specs"

[nepozs]

@smonesi
I'm pretty sure, that LTG002 does not work linear (I don't have it but it is typical, that this kind of device works non-linear).
Linear interpolation is good only for situations where measurement is impossible (like portable lights using built in rechargeable battery), so this model is ideal to use in documentation describing how to create configuration without measurement with data printed in the manual, box or on the device, but I'm not sure if something like that should be in ready made database.

[smonesi]

I'm pretty sure, that LTG002 does not work linear (I don't have it but it is typical, that this kind of device works non-linear).
Linear interpolation is good only for situations where measurement is impossible (like portable lights using built in rechargeable battery), so this model is ideal to use in documentation describing how to create configuration without measurement with data printed in the manual, box or on the device, but I'm not sure if something like that should be in ready made database.

I do have a LTG002 but I have no way to measure it (I might have other lights in the same situation, while for most of them I can and I am measuring via script). I would still like to use this component to have a rough idea of the power used (and I guess all the owners of such a light will). Do you think there is a way to produce a "good-enough" configuration, somehow (eg. a supported_mode="typical-led-interpolation")?

Maybe we can take the most similar device that we can measure (LTA001?) and scale the values in the LUT file to the 0.2-5.5W of the LTG002? I think an approximation is still better than nothing (or than linear, as you noticed).

[nepozs]

Yes, similar model could be an approximation, but it shouldn't be published here, maybe it is time to discuss secondary/temporary database for models without measurements.
If you need some solution "just for now" I've described workaround in #67 (powercalc also have built-in other possibilities to workaround such situation).

I think as temporary solution could be few linear models with calibrate points
https://github.com/bramstroker/homeassistant-powercalc#advanced-precision-calibration
but as of today we have still not enough data to prepare them.

[bramstroker]

@smonesi I agree with @nepozs that the linear mode for LTG002 is not accurate enough, and we should not include it in the repository. We should only use linear mode when there is no other option (for example the portable battery powered light Hue Go).
I measured the GU10 spots myself with some separate GU10 fitting I had laying around which I connected to a 230v wire/plug and ultimately to the Shelly plug. Just removed the spot temporary from the ceiling bracket, and placed it back when done measuring. Maybe you are able to do the same?

Regarding another mode "typical-led-interpolation", not sure how to do it, and I don't want to add unnecessary complexity to the component for this kind of edge cases. It will also take development effort and maintenance.

You can include the LTG002 files in your own HA instance using custom_model_directory configuration option. https://github.com/bramstroker/homeassistant-powercalc#sensor-configuration
Or you can just define linear mode on the powercalc sensor configuration.
Hope this helps.

So regarding the state of this PR, to get it merged two things need to be resolved.

• LTG002 files need to be removed
• Interpolated values need to be added to the CSV's so they have enough measure points. And sample based be manually checked for correctness.

Thanks again for your efforts, much appreciated.

[bramstroker]

One final question. Is it correct that the Iris and Bloom lights only support hs mode? Can you not change the color temperature for that light in Home Assistant? You can also verify this in Developer Tools, and than check the supported_color_modes

[smonesi]

One final question. Is it correct that the Iris and Bloom lights only support hs mode? Can you not change the color temperature for that light in Home Assistant? You can also verify this in Developer Tools, and than check the supported_color_modes

Those are very old (first/second gen) Philips Hue lights that only support hs, both look like:

effect_list:
  - colorloop
  - random
supported_color_modes:
  - hs
friendly_name: ***
supported_features: 61

[bramstroker]

Ok let's get it merged

[bramstroker]

Just merged, but I see the build on master is failing now.
https://github.com/bramstroker/homeassistant-powercalc/runs/3389233387

Not sure why the github actions in the PR were not failing.
There is a problem with signify/LCT024/hs.csv.gz.

Could you have a look please?

[nepozs]

@smonesi @bramstroker

There is a problem with signify/LCT024/hs.csv.gz

I've reviewed hs.csv file inside and it contains "-0" values, so I think probably it is the reason for errors (and it can be fixed quick and simple by manual edit).

But there is second and even worst problem: hs.csv contains unbelievable power values - they just can't be lower than standby_power (I know, there are sometimes some fluctuations which are measured as 0W but it means only that pauses in script were much too small).