Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added DPT-3s and corresponding sensor types (second try) #167

Merged
merged 12 commits into from
Dec 13, 2020

Conversation

buergi
Copy link
Contributor

@buergi buergi commented Jan 11, 2019

Added all missing DPT-3 datatypes, DPT 3.007 and 3.008, each in their stepwise and startstop mode, see ABB Applications Manual for details.
I'm using them to control my Hues with my Busch Jäger push button-coupling 6108/02.

@coveralls
Copy link

coveralls commented Jan 11, 2019

Coverage Status

Coverage increased (+0.1%) to 94.77% when pulling 586316e on buergi:master into 5957eb0 on XKNX:master.

@nodefeet
Copy link

Thank you very much @buergi , that's what was missing in xknx for me.
I think the relativ dimming should additionally be implemented in the device class Light to cover the basic functionality someone would expect from a KNX Light.

@marvin-w marvin-w requested a review from farmio January 18, 2019 11:52
@farmio
Copy link
Member

farmio commented Jan 18, 2019

@nodefeet could you please explain your usecase for DPT-3 in the Light module? My only usage of xknx is through home-assistant - which has no concept of holding down buttons / switches. Are you using xknx outside of HA?

@nodefeet
Copy link

Yes and it's really just that for that: dimming directly with KNX switches outside of the HA interface. A lot of visualizations for smart homes offer the relative dimming, but I would agree that as soon as you have a virtual slider or knob it’s not really necessary. But with KNX compatible switches it is very useful to dim the light directly from your physical switch (the fastest app is still no app).

From my experience the stepwise dimming is rarely used in practice, partly due to that it is swiftly flooding the bus with excess telegrams, but, the start stop dimming is imho part of any basic KNX installation with lights. After the brightness is modified via relative dimming (i. e. after receiving the stop telegram or reaching 0 or 100 % brightness ) KNX dimming actuators typically return the actual brightness to the bus (so that interfaces like HA can update their status).

Let me know if I can provide any additional input.

@nodefeet
Copy link

@farmio I see now that you were specifically asking about the use case of DPT-3 in the class Light and not just about the use case of DPT-3 in general.

So let me add that since I think DPT-3 or relative dimming is essential for an abstraction of a KNX Light I would expect to not just find the group_address_brightness(_absolut), but also a group_address_brightness_relative and the associated functionality in the device class Light.

@farmio
Copy link
Member

farmio commented Jan 19, 2019

Adding a stepwise dimming GA to the Light class would be no big deal. As far as im concerned it is a stateless action so we wouldn't even need a state GA or a current_* property. It would just forward the given value to the GA.
Functionally this would be no different to adding a Sensor with DPT-3 type. I see that for the sake of completeness it could be added to the Light class.
I'd say its @Julius2342 call, since he doesn't want the light module to get too bloated.

Copy link
Collaborator

@Julius2342 Julius2342 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The only thing i wonder, do you think it would be possible to encapsulate the bitshifting a bit more? (not having a good idea though).

(And i have to admit I can't really say anything about the DPT-3xx functionality, i don't have such devices...)

@buergi
Copy link
Contributor Author

buergi commented Jan 24, 2019

Which bitshifting do you mean? The few lines inside the to_knx method? After some pondering I couldn't make up any better representation that shows the origin of the values better of course I could encapsulate the bitshifting like this, if you prefer this, but I wouldn't consider this to increase the readability

MakeDPT3 = lambda sign, stepcode: DPTBinary(sign << 3 | stepcode)

Copy link
Collaborator

@Julius2342 Julius2342 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @buergi sorry for my late reply. I hope I am not too picky.

sign = 0 if value < 0 else 1
if self.invert:
sign = 1 if sign == 0 else 0

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would add a local function within to_knx():

def get_knx_value(value):
        """"Returns the knx value according to the theshold defined on of the XXX-knx standard."""
        if abs(value) >= 100:
            return 1
        elif abs(value) >= 50:
            return 2
        elif abs(value) >= 25:
            return 3
        elif abs(value) >= 12:
            return 4
        elif abs(value) >= 6:
            return 5
        elif abs(value) >= 3:
            return 6
        elif abs(value) >= 1:
           return 7
       return 0  # <--- not sure about this, that should be the behaviour from line 53: ret = DPTBinary(0)

def format_knx_bitmap(sign, value):
    return sign << 3 | value & 7

knx_value=get_knx_value(value)
return DPTBinary(format_knx_bitmap(sign, knx_value)

What do you think?

def from_knx(self, payload):
"""Convert current payload to value."""
if payload.value & ~0x0F != 0: # more than 4-bit
pass # raises exception below
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should raise the Exception here.


def from_knx(self, payload):
"""Convert current payload to value."""
if payload.value & ~0x0F != 0: # more than 4-bit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not either if payload.value > 15: or if payload.value >> 4: ?

if payload.value & ~0x0F != 0: # more than 4-bit
raise CouldNotParseTelegram("payload invalid", payload=payload, device_name=self.device_name)
# calculated using floor(100/2^((value&0x07)-1))
value = [0, -100, -50, -25, -12, -6, -3, -1, 0, 100, 50, 25, 12, 6, 3, 1][payload.value & 0x0F]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I personally find this hard to read :) why not doing something like

THRESHOLDS = [0, 100, 50, 25, 12, 6, 3, 1]
invert = self.invert ^ (payload.value & 0x8) > 0
value = threshold_array[payload.value  & 0x07]
return -value if invert else value

But well, not sure if this makes it better.

def from_knx(self, payload):
"""Convert current payload to value."""
if payload.value & ~0x0F != 0: # more than 4-bit
pass # raises exception below
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should raise the Exception here.


def from_knx(self, payload):
"""Convert current payload to value."""
if payload.value & ~0x0F != 0: # more than 4-bit
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not either if payload.value > 15: or if payload.value >> 4: ?

Would at least make it more clear that you are testing for the 4 right bits.

"""Convert current payload to value."""
if payload.value & ~0x0F != 0: # more than 4-bit
pass # raises exception below
elif payload.value & 0x07 == 0:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just that i understand this right, this evaluates for value=8 and value=0 ...?

(I'm not too good in bitwise calculations, bit i think we should be more explicit here:

if payload.value in [0, 8]: if this is intended.

pass # raises exception below
elif payload.value & 0x07 == 0:
return self.Direction.STOP
elif payload.value & 0x08 == 0:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

payload.value < 8

@buergi
Copy link
Contributor Author

buergi commented Jan 25, 2020

Integrated your most if not all of your remarks and ported the remote value to a new dpt class. What do you think?

@farmio
Copy link
Member

farmio commented Jan 26, 2020

Thank you for updating it. This is very much appreciated.

I think remote_value_sensor should be used for numeric datatypes only. I would add a new class (maybe "remote_value_control") for this - and maybe DPT2 in the future. This would also enable us to pass tuples holding control and step values so we can use them too.
I'd also create a new device subclass for this. This would be a cleaner imho.

@buergi
Copy link
Contributor Author

buergi commented Jan 27, 2020

I see your point, having a sensor for a control state is a bit weird, especially as you cannot set the value for controlling it.
I have a remote_value_dpt3 implementation in my desk drawer, however, it currently does not use the DPT3 class. However, it would be no problem adapting it.
However, I have no idea, how I'd integrate this remote value class into a usable device in home assistant.
So, what do you suggest, how to go on? I remove the sensor integration, add a remote value class and we then probably need a new device type, e.g. control.

As shortly explained in the docstring, there are two control types stepwise dimming and start/stop dimming. The frontend for the start/stop could probably look similar to the cover control. For the stepwise dimming it would be similar, plus evtl. another slider to allow setting the step size.

For easy use of the control without any further automatisation the control's state would probably include an internally accumulated absolute value of the dimming/cover state.

@farmio
Copy link
Member

farmio commented Jan 27, 2020

You're right, the remote_value class is really not a big deal - its mostly boilerplate.

I like xknx classes to be explicit. HA is another issue - we have to adapt to the available platforms. We could

  • integrate the Control device in the HA integration
  • or maybe the RemoteValueControl in the xknx Sensor class.

In ExposeSensor we have something similar:

if value_type == "binary":

For stepwise dimming it would also be an option to make the step size configurable via yaml. Just like KNX switches where the stepsize is set once via ETS. It could always be bypassed by eg. HA-automations with an optional argument.

@recMartin
Copy link
Contributor

I made similar work and only now saw that there exists this still unmerged PR.
Perhaps someone which has a better view of the complete xknx than me can look at both?
See #262 .

@buergi
Copy link
Contributor Author

buergi commented Feb 4, 2020

Thanks @recMartin for your implementation. I looked over the code and like the idea of having DPTControl implemented as "control + stepcode". It is closer to the KNX standard as my "increment" implementation and especially as it is bijective (invertable).

However, from a user perspective I'd expect to work with increment values, e.g. make the light 25% brigher. I'm currently consider moving the increment conversion to a corresponding RemoteValue type and stripping the DPT implementation down to the control+stepcode approach.

As @recMartin said he won't do any changes on his PR at the moment I'd suggest I keep working in this PR and try to merge both PRs.

@recMartin
Copy link
Contributor

I had a look in the code, its a different approach than mine (#262), but also good work!
Some comments and suggestions:

  • I like that this solution does not need a list (as mine does) but only one value
  • Maybe we should make increment_to_stepcode(value) inverse of stepcode_to_increment(stepcode, sign) this means:
    • stepcode_to_increment(stepcode, sign) -> value
    • increment_to_stepcode(value) -> (value, sign) , calculate 'sign' in this function
  • DPT3 looks like only a base-class, but it is also used for both, stepwise-dimming and controlling blinds
  • the derived classes 'DPTControlDimming' and 'DPTControlBlinds' can only be used for start/stop dimming.
    All step dimmings +/-1 to +/-100% are mapped to +/-100% and output as DECREASE or INCREASE or STOP, so some information is lost in from_knx (not the correct step is output).
  • Could we allow/handle both types of values in the DPT classes or handle this only in remote_control?
  • Since DPT 3.00x is not for sensors, but for actuators/controls, it should be incorporated in e.g. "remote_value_control" (as farmio suggested) and not in remote_sensor.

@buergi
Copy link
Contributor Author

buergi commented Feb 6, 2020

Thanks @recMartin for your input. I've been working on the code yesterday and a lot has changed. I've not pushed it yet as I have some open points I still have to ponder about. Especially as I already have a future control device implementation in mind when developing.

I currently implemented a base DPTControl class based mostly on your code with a few esthetical changes. From this I derived a DPTControlStepwise which is mostly my code. I renamed the DPTControlDimming/Blinds to DPTControlStartStopDimming/Blinds (which I actually already had in the original code of this PR). I still like to add an intermediate DPTControlStartStop class to reduce the amount of duplicated code, but I'm not done with it yet
The implementation of the RemoteValueControl is very similar to your implementation (with a DPTMap as in RemoteValueSensor).

I already added unit tests for all important cases I could think of.
I guess this should tackle all your valid points of critisism you mentioned, but I'm open to new comments once I pushed the code.

I currently don't plan to add a new Control class in this PR, but I have some ideas and will make another PR when I'm ready. In short I plan to make a Control with an integrated absolute counter which can be configured to stepwise or startstop mode. The first case is simple and the absolute value is changed directly when a step-message is received. For the second case I have an implementation in mind which is similar to the one of the cover device, which uses a method which is added as a job to home assistant (see cover.py)

@buergi
Copy link
Contributor Author

buergi commented Nov 21, 2020

I again rebased the PR and updated the DPT mapping in the RemoteValueControl to match that of RemoteValueSensor.

There is a slight problem with the dpt_main_number and dpt_sub_number convention. It is meant to be unique, however, in this case it isn't. There are two different interpretations for DPT3 values, stepwise and start/stop. A +100% message e.g. in the context of light dimming can either mean "turn the brightness to maximum" or "start increasing the brighness (until a 0% message is received)". Therefore both are handled via separate classes so only one can carry the dpt_*_number flags. I decided for the stepwise class as it carries more information than the start/stop class.

Currently, this value class cannot be used from within HA. I still plan to add a new kind of device which has an internal value that can be controlled with this type of messages. However, in the meantime, and also for flexibility reasons I would suggest extending the RemoteValueSensor type to allow DPTBinary payloads just by adding or isinstance(payload, DPTBinary) to the return statement of its payload_valid method and a slight adaption in its to_knx method. This makes no sense for stepwise controls but would allow using start/stop switches in custom automations. However, as I thought you might not like this idea I did not include it in the PR.

@farmio
Copy link
Member

farmio commented Nov 23, 2020

Hi!
Thanks for your contribution. We have some bugs currently that keep me form reviewing this PR.

All I can say right now is that I'd not be in favour of adding or isinstance(payload, DPTBinary) to RemoteValueSensor.payload_valid() . I'd rather use a new RemoteValue subclass and use it form Sensor() - just as done in ExposeSensor for binary type.

Copy link
Member

@farmio farmio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have some readability/style suggestions.
The suggested type annotations are just for better overview.

xknx/dpt/dpt_1byte_control.py Outdated Show resolved Hide resolved
xknx/dpt/__init__.py Outdated Show resolved Hide resolved
xknx/dpt/dpt_1byte_control.py Outdated Show resolved Hide resolved
@classmethod
def _decode(cls, value):
"""Decode value into control-bit and step-code."""
control = 1 if (value & cls.APCI_CONTROLMASK) != 0 else 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
control = 1 if (value & cls.APCI_CONTROLMASK) != 0 else 0
control = (value & cls.APCI_CONTROLMASK) >> 3

I think this would be easier to read here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would be possible, however, I don't like it a lot as it is not directly obvious what possible values control could get. One first has to look up the value of the mask and count the bits to understand where the 3 is coming from. Especially having in mind the implementation for the DPT2 class we would only need to change the APCI_xxxMASK values to reuse it for a 2-bit value.

Comment on lines 37 to 41
def _encode(cls, control, step_code):
"""Encode control-bit with step-code."""
value = 1 if control > 0 else 0
value = (value << 3) | (step_code & cls.APCI_STEPCODEMASK)
return value
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def _encode(cls, control, step_code):
"""Encode control-bit with step-code."""
value = 1 if control > 0 else 0
value = (value << 3) | (step_code & cls.APCI_STEPCODEMASK)
return value
def _encode(cls, control: bool, step_code: int):
"""Encode control-bit with step-code."""
return (control << 3) | (step_code & cls.APCI_STEPCODEMASK)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't like this at all, as you put in the intrinsic assumption that control is always either 0 or 1. You added the type hints, so the type checker should detect this, but users of the xknx library might not use type checking in their code. Just imagine I would pass control=3.
It would be possible to use use (control & cls.APIC_CONTROLMASK) << 3, but as above this diminishs the abstraction, as the 3 contains redundant information which is already contained in APIC_CONTROLMASK. So I'll add the type hints, but would prefer to keep the rest.

xknx/dpt/dpt_1byte_control.py Outdated Show resolved Hide resolved
Comment on lines 99 to 102
if invert:
control = 0 if control > 0 else 1

return {"control": control, "step_code": step_code}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if invert:
control = 0 if control > 0 else 1
return {"control": control, "step_code": step_code}
return {"control": control, "step_code": step_code ^ invert}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you meant control ^ invert. Would be possible, but do you really consider this easier to read? I find it somewhat crypting whereas the explicit version is more obvious. Plus, it would cause a mess if invert is not 0 or 1, which, however, is safe to assume as long as a boolean is passed.

xknx/dpt/dpt_1byte_control.py Outdated Show resolved Hide resolved
xknx/dpt/dpt_1byte_control.py Outdated Show resolved Hide resolved
xknx/dpt/dpt_1byte_control.py Outdated Show resolved Hide resolved
@farmio
Copy link
Member

farmio commented Dec 12, 2020

Hey! I think this can be merged now, but for some reason this is showing 99 files edited now. There are some commits in the changeset that are already merged a while ago. Can you please clean this up (hopefully) a last time?

@buergi
Copy link
Contributor Author

buergi commented Dec 13, 2020

Upps, what did I do there, git keeps on surprising me. I now cherry-picked all my commits again upon the current head. I'm not sure why the coverage test fails. I don't understand the output of this unit test.

@farmio farmio merged commit be8de09 into XKNX:master Dec 13, 2020
@farmio
Copy link
Member

farmio commented Dec 13, 2020

Thank you very much!
I'm looking forward to see this integrated to HA! 😃

@nikolamomchev
Copy link

@buergi thanks for contributing this!

I am trying to figure out how to control my dimmers with HA and I found this change. It seems that the DPTControlStartStopDimming should do exactly what I need.

I am, however having trouble figuring out how to use this in HA. You are only providing a sensor in this change and it's not obvious to me how this helps with HA integration.

If you managed to control your dimmers with HA, I'd really appreciate if you shared how you did it.

@farmio
Copy link
Member

farmio commented Oct 23, 2021

@nicolamomchev there is no built in entity type in HA that would properly fit to send these.

If you want other integrations entities to action received DPT3 telegrams you could use this blueprint: https://community.home-assistant.io/t/knx-relative-dimming-for-lights/273473

Or try to do it via the select entity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants