Skip to content

4.0.0 Typed and tidy

Choose a tag to compare

@farmio farmio released this 15 Oct 09:29
· 30 commits to main since this release
bdf5ea4

Breaking changes

  • Drop support for Python 3.10. XKNX requires Python 3.11 or newer now.

  • Remove xknx.util. Its only member was asyncio_timeout - a backport of asyncio.timeout for Python versions not providing it - so use asyncio.timeout directly.

  • RawValue takes payload_length as its first argument after xknx, before name - name is optional now and Python doesn't allow a required argument after an optional one. RawValue(xknx, "Name", 2, ...) becomes RawValue(xknx, 2, "Name", ...).

  • Sensor, NumericValue and ExposeSensor take value_type as their first argument after xknx, before name, and require it - like payload_length of RawValue above. Sensor(xknx, "Name", value_type="temperature") becomes Sensor(xknx, "temperature", "Name"). value_type never had a usable default: passing None raised ConversionError. Notification keeps its optional value_type, now defaulting to "string" instead of None.

  • RemoteValueSensor, RemoteValueNumeric and RemoteValueString take value_type as their first argument after xknx and require it. RemoteValueString no longer defaults to DPTString; pass value_type="string" explicitly.

  • Every argument of a device after xknx is keyword-only now. Switch(xknx, "Kitchen", "1/2/3") becomes Switch(xknx, name="Kitchen", group_address="1/2/3"). Devices carry a lot of similar looking arguments - most of them group addresses - so a positional call was easy to get subtly wrong, and it pinned the argument order as public API. The argument order named in the entries above is therefore only about the signature; it doesn't affect how a device is constructed anymore.

  • Remove ha_device_class from DPTBase subclasses, RemoteValueSensor, RemoteValueByLength, Sensor and NumericValue. Home Assistant shall maintain this itself.

  • Remove RemoteValue.__eq__(). It compared __dict__ attributes - a leftover of the YAML config handling removed in 1.0 - and raised AttributeError when compared to an object without a __dict__. Remote values compare by identity now, which also makes them hashable again, so they can be used in sets and as dict keys.

  • All classes in xknx.remote_value define __slots__ now. Setting an attribute that isn't declared by the class raises AttributeError instead of silently creating it.

    rv_1 = RemoteValueSwitch(xknx, group_address="1/2/3")
    rv_2 = RemoteValueSwitch(xknx, group_address="1/2/3")
    
    rv_1 == rv_2  # before: True (equal attributes); now: False (distinct objects)
    hash(rv_1)  # before: TypeError: unhashable type; now: works
    rv_1.my_own_attribute = 1  # before: works; now: AttributeError
  • Remove Device.__eq__() - same reasoning as RemoteValue.__eq__() above: it compared __dict__ attributes and was a leftover of the YAML config handling removed in 1.0. Devices compare by identity now, which also makes Device hashable again, so devices can be used in sets and as dict keys. The same applies to Light.red, .green, .blue and .white.

  • Remove device lookup by name or index from xknx.devices. xknx.devices["NameOfDevice"] and xknx.devices[0] are gone - device names are never checked for uniqueness, so the lookup could return any one of several devices sharing a name. Keep a reference to the device object instead, or iterate xknx.devices to find it. device in xknx.devices takes the Device object now instead of its name.

  • Devices.async_add() and Devices.async_remove() raise ValueError when the device object is already registered, or not registered at all. Adding a device twice had it receive every telegram twice and fire its callbacks twice; removing an unregistered one cancelled its tasks and unregistered its state updater before failing.

  • Scene.scene_value is a RemoteValueSceneControl (DPT 18.001) instead of a RemoteValueSceneNumber (DPT 17.001), so its value carries the learn bit next to the scene number. Telegrams on the wire are unchanged: DPT 18.001 encodes an activation to the same octet DPT 17.001 does, and decodes one back the same way. The device callback is called for received learn telegrams of the devices scene_number now, not only for activations - the new Scene.learn_requested tells both apart.

  • DPT 9.xxx: value_max is 670433.28 for all 2 byte float types - it was 670760.96 on DPT2ByteFloat and 670760 on most subtypes, which encode to 0x7FFF. KNX v02.02.01 - Datapoint Types 03.07.02 - §3.10 reserves that pattern: "For all Datapoint Types 9.xxx, the encoded value 7FFFh shall always be used to denote invalid data", and gives the range as [-671 088,64 ... 670 433,28]. So xknx sent the invalid-data pattern as a regular value and decoded a received one to 670760.96 instead of rejecting it - from_knx() raises ConversionError on 0x7FFF now, and to_knx() rejects anything above 670433.28. A device sending 0x7FFF - a thermostat with an unconfigured setpoint for example - is reported as a conversion error instead of a nonsensical reading.

  • DPT 9.xxx lower bounds follow the same table: DPTTemperatureDifference2Byte (9.002), DPTTemperatureA (9.003), DPTTime1 (9.010) and DPTTime2 (9.011) accept -671088.64 instead of -670760, and DPTPartsPerMillion (9.008) - specified as 0 ppm ... 670 433,28 ppm - rejects negative values now like the other subtypes with a 0 minimum do.

  • Rename xknx.management.management.MANAGAMENT_ACK_TIMEOUT and MANAGAMENT_CONNECTION_TIMEOUT to MANAGEMENT_ACK_TIMEOUT and MANAGEMENT_CONNECTION_TIMEOUT - they were misspelled.

  • Remove nm_invididual_address_write from xknx.management.procedures - the misspelled alias of nm_individual_address_write, deprecated since 3.18.0. Use nm_individual_address_write instead.

  • Remove xknx.management.application_layer_enum. The module only re-exported ReturnCode, which moved to xknx.telegram.apci in 3.17.0 and has been deprecated there since. Import ReturnCode from xknx.telegram.apci.

Features

  • Add a command line interface. Installing xknx puts an xknx command on the path (also available via python -m xknx) providing scan, group read, group write and group monitor commands. See xknx --help.

  • Devices tell the cause of their latest update in Device.last_update, set before device updated callbacks are called. DeviceUpdate.telegram is the incoming or outgoing telegram that caused the update. Updates run later from a task keep the telegram that started it - eg. the periodic travel updates of a Cover, debounced individual color updates of a Light or the counter updates of a BinarySensor after context_timeout. It is None for updates not caused by a telegram, like RemoteValue.update_value(). Work scheduled from a device updated callback doesn't inherit the cause - an application action started from there is attributed to its own telegram_context(), not to the telegram that triggered the callback. last_update always describes the latest update, so copy it inside the callback if work scheduled from there needs the cause of this specific update.

  • Telegram.context carries an application defined object identifying the origin of a telegram. xknx never reads it. Outgoing telegrams created inside with xknx.telegram.telegram_context(context): - or in tasks started from it, like the auto stop of a Cover - carry context. For incoming telegrams it may be set by a telegram received callback, which runs before devices process the telegram. DeviceUpdate.context is the context of the causing telegram, or of the active telegram_context() for updates not caused by a telegram. This lets an application relate a device update to its own action or to the received telegram - eg. to show which KNX device, or which user, changed a state.

    with telegram_context(my_context):
        await light.set_on()
    
    def light_updated_cb(light: Light) -> None:
        light.last_update.context  # my_context - once the outgoing telegram was processed
        light.last_update.telegram  # the outgoing telegram

Devices

  • The name of a device is optional now and can be changed after instantiation. When no name is given it defaults to the class name of the device - Sensor, NumericValue and ExposeSensor append their value type, eg. "Sensor temperature". Assigning Device.name passes the new name down to the devices RemoteValue instances, so log messages and exceptions use it too - previously the name was only copied to the remote values on instantiation and assigning a new one left them stale. Renaming a Climate renames its mode device as well.

    light = Light(xknx, group_address_switch="1/2/3")
    light.name  # "Light"
    light.name = "light.kitchen_ceiling"  # eg. in Home Assistants `async_added_to_hass()`
  • Scene: add learn() to send a telegram with the learn bit set, telling actuators to store their current state as this scene. Received learn telegrams are decoded instead of logging a "Can not process" warning, so a Scene can serve as scene actuator: restore its state from the device callback when learn_requested is False, store it when it is True.

Management

  • Add dmp_interface_object_read_r, dmp_interface_object_write_r, dmp_interface_object_scan_r and dmp_interface_object_verify_r to xknx.management.procedures - KNX v02.01.02 - Management Procedures 03.05.02 - §3.27.2/§3.25.2/§3.28.2/§3.26.2, reading, writing, discovering and verifying Properties of a device's Interface Objects on an already-open P2PConnection. Read, write and verify split a request into as many A_PropertyValue_Read/_Write exchanges as it needs, bounded both by the 4-bit nr_of_elem field and by max_apdu_length - the device's PID_MAX_APDU_LENGTH, defaulting to the 15 octets of an L_Data_Standard frame. read_r also serves the start_index=0 element count query. A read-back that doesn't match raises the new PropertyVerificationError; scan_r returns a list of ScannedInterfaceObject.
  • Add dmp_mem_read_r_co, dmp_mem_write_r_co and dmp_mem_verify_r_co to xknx.management.procedures — KNX v02.01.02 - Management Procedures 03.05.02 - §3.18.2/§3.16.2/§3.17.2, reading, writing and verifying a device's 64 KiB A_Memory_* address space over an already-open P2PConnection. Add dmp_user_mem_read_r_co, dmp_user_mem_write_r_co and dmp_user_mem_verify_r_co — §3.21.2/§3.19.2/§3.20.2, the same three operations for the 1 MiB A_UserMemory_* address space. All six transparently chunk transfers exceeding the wire APDU length (12/11 octets respectively for the default 15 octet frame, or the count field's own 63/15-element limit if that binds first at a larger negotiated APDU length), validate the address range doesn't overflow before sending anything, and check each response's echoed address matches what was requested. The write procedures take an optional verify to read back and compare each chunk immediately instead of leaving a caller-supplied write_delay for the device to finish programming, since A_Memory_Write/A_UserMemory_Write are not confirmed at the application layer; a mismatch raises the new VerificationError (PropertyVerificationError's new, more general parent, so both families can be caught uniformly).
  • dm_restart_r_co and dm_restart support a Master Reset now, via master_reset=True, erase_code and channel_number - KNX v02.01.02 - Management Procedures 03.05.02 - §3.7.3 DM_Restart_RCo, which covers both restart kinds as one Management Procedure distinguished by its mpp_RestartType parameter. Unlike a Basic Restart, a Master Reset is confirmed at the application layer by an A_Restart_Response-PDU, so both functions return the parsed apci.RestartMasterResetResponse for a Master Reset, or None for a Basic Restart as before; a non-zero error code raises ManagementConnectionError. erase_code defaults to 01h "Confirmed Restart" and is validated to 01h-08h (KNX v02.01.02 - Management Procedures 03.05.02 - §3.7.1.2.3.1, Table 4 - the only defined values; 00h and 09h-FFh are reserved and a Management Client must not send them). The response's process_time is not merely informational - the spec mandates it as the minimum wait before the Management Client may retry communication.

Connection

  • Threaded interface: a request still pending when the interface is stopped raises CommunicationError instead of never returning. stop() stopped the connection thread's loop without finishing the tasks on it, so the caller - and the executor thread blocked waiting for the result - hung forever, holding up the shutdown of the host application. Results from the connection thread are awaited with asyncio.wrap_future() now, so no executor thread is blocked, and cancelling the caller also cancels the request on the connection thread.
  • Threaded interface: cancelling start() - by asyncio.timeout for example - stops the connection thread it already started instead of leaving it running. The connection thread's event loop is closed when the interface is stopped, and stop() waits for the thread to end in an executor instead of blocking the calling event loop.
  • KNX IP Secure transports discard unencrypted frames instead of passing them to their callbacks. A secure session accepts a plain frame only for the handshake - SessionRequest outgoing, SessionResponse incoming - and raises IPSecureError when anything else is sent before the session is initialized. Secure routing keeps forwarding plain discovery and self description frames (SearchRequest, SearchResponse, DescriptionRequest and DescriptionResponse, extended variants included) since these services are never secured and share the multicast endpoint, but now drops every other plain frame - previously only RoutingIndication was dropped, so a plain RoutingBusy from any sender could still throttle outgoing telegrams. Frames that may not be encapsulated at all - a nested SecureWrapper and the Remote Configuration and Diagnosis service family - are discarded when received inside a SecureWrapper.

Bugfixes

  • Management: don't acknowledge a connected telegram from a device xknx has no open point-to-point connection with. The acknowledgement was sent before the connection was looked up, so a device that was not talking to us received a T_Ack and was then reported as "No active point-to-point connection for received telegram". KNX v01.02.03 - Transport Layer 03.03.04 - §5.4: no style of the connection oriented state machine acknowledges anything while CLOSED - including the styles that accept incoming connections.
  • Management: only telegrams of a point-to-point connection are handled by an open connection to their sender. The sender address alone decided that before, so an unnumbered telegram from a device we happened to hold a connection with - a broadcast response for example - was acknowledged, never reached the broadcast context it was meant for, and could be mistaken for the response of a pending point-to-point request.
  • Management: a repeated sequence number is acknowledged again instead of being acknowledged and dropped, and a sequence number that is neither the expected nor the previous one is answered with a negative acknowledgement instead of a positive one - KNX v01.02.03 - Transport Layer 03.03.04 - §5.3 A3 and A4.
  • apci.PropertyValueWrite is an APCIRequest[PropertyValueResponse] now, like PropertyValueRead already was. It carries a KNX-spec-defined response (KNX v02.01.01 - Application Layer 03.03.07 - §3.4.4.2) and was missed when the other point-to-point request services were converted, so P2PConnection.request() sent it without verifying or narrowing the response type.
  • apci.PropertyValueRead, PropertyValueWrite and PropertyValueResponse raise ConversionError for a start_index above 4095 instead of encoding it into the neighbouring nr_of_elem field - to_knx() silently emitted a frame addressing a different element with a different count.

Internals

  • Telegram is now generic over its payload type (Telegram[GroupValueWrite], etc.), defaulting to Telegram behaving exactly as before when left unparametrized - payload is None only for that default/unparametrized case (control telegrams like ACK/Disconnect); parametrized as Telegram[SomeAPCI], payload is SomeAPCI, never None. Device.process() now hands process_group_write()/process_group_response()/process_group_read() a Telegram narrowed to the APCI type it already verified via isinstance, propagated through RemoteValue.process() and every device's process_group_* override. No behavior change - RemoteValue.process() keeps its own isinstance check since, unlike the management case below, nothing enforces the payload type before it's called directly.
  • Add APCIRequest to xknx.telegram.apci - an APCI subclass carrying the KNX-spec-defined response type of a request service as its type argument, eg. class MemoryRead(APCIRequest[MemoryResponse]). RESPONSE_TYPE is derived from that argument. All 21 point-to-point request services with a spec-defined response were converted; group communication services (GroupValueRead, GroupPropValueRead) stay plain APCI since they are never sent via P2PConnection.request(). request() no longer takes an expected= argument - it infers and verifies the expected response from the payload's type and returns the correspondingly typed Telegram[ResponseType], so procedures no longer need assert isinstance(response.payload, ResponseType) (or even a None check) to get a typed .payload.
  • Add APCIBroadcastRequest to xknx.telegram.apci - the broadcast counterpart of APCIRequest, carrying its response type the same way. IndividualAddressRead, IndividualAddressSerialRead, DomainAddressRead, DomainAddressSerialNumberRead, NetworkParameterRead and SystemNetworkParameterRead derive from it. It is a sibling of APCIRequest, not a subclass, so a broadcast service can not be passed to P2PConnection.request() - it would be sent over a point-to-point connection and time out.
  • RequestResponse is now generic over the response body it awaits, eg. class Connect(RequestResponse[ConnectResponse]). start() gives way to request(), which returns that response instead of leaving it on the instance, and raises the new RequestResponseError when none arrived or the server answered with an error status.
  • DescriptionQuery and SearchExtendedQuery derive from RequestResponse now. Their start() and gateway_descriptor attribute are replaced by request_gateway_descriptor().
  • Broadcast communication is encapsulated in a Broadcast class, reached as xknx.management.broadcast. Management.send_broadcast() becomes xknx.management.broadcast.send() and the Management.broadcast() context manager becomes xknx.management.broadcast.context().
  • Add Broadcast.request() - broadcasts a request service and yields the responses to it, eg. async for telegram in xknx.management.broadcast.request(apci.IndividualAddressRead(), timeout=3). It replaces the context, send() and receive() trio for the common case, listening for the response type the payload declares and narrowing each telegram to it. Any number of devices may answer, so it yields until timeout elapses rather than returning a single telegram like P2PConnection.request().
  • BroadcastContext.receive() takes the APCI class to listen for, eg. bc_context.receive(apci.IndividualAddressResponse, timeout=3), and yields only telegrams carrying it, narrowed to Telegram[ThatAPCI], instead of everything on the broadcast channel. Called without an argument it behaves as before.
  • BroadcastContext.receive()'s timeout is armed around the wait for the next telegram instead of around the whole iteration. Previously a caller leaving the loop early - after finding the response it was looking for - left the timeout armed on the abandoned generator, cancelling that caller once it elapsed. timeout still bounds the whole iteration.
  • Add xknx.exceptions.VerificationError, a new parent of PropertyVerificationError for "the device's read-back data doesn't match what was expected" independent of which service produced it. dmp_mem_write_r_co/dmp_user_mem_write_r_co's verify and dmp_mem_verify_r_co/dmp_user_mem_verify_r_co raise it directly, so a caller can catch VerificationError uniformly across Property and Memory operations instead of PropertyVerificationError only matching the former. Non-breaking: existing code catching PropertyVerificationError or ManagementConnectionError is unaffected.
  • CEMILData.flags is a CEMIFlags dataclass now instead of a 16 bit int, with a field per control field value: priority (new CEMIPriority enum), repeat_on_error, system_broadcast (both named for the positive meaning; inverted on the wire), acknowledge_request, confirm_error, hop_count - which replaces the removed CEMILData.hops property - and the received frame_type / frame_format. Frame Type and Address Type are derived when serializing, from the NPDU length and from the type of the destination address (CEMILData.address_type), so flags can no longer disagree with the frame that is put on the wire. CEMILData(flags=...) is optional now. The bit constants moved from CEMIFlags to xknx.cemi.flags.
  • xknx.secure.data_secure_asdu.block_0(), SecureData.init_from_plain_apdu() and SecureData.get_plain_apdu() take address_type: CEMIAddressType and frame_format: CEMIFrameFormat instead of frame_flags: int. Only those two fields of Ctrl2 ever reached the CCM input; the value on the wire and the one fed to the MAC now come from the same place.
  • Devices keeps a group address index of its registered devices instead of scanning every device on every incoming telegram. Devices.devices_by_group_address() is a dict lookup now - its result is unchanged, devices are still returned in registration order and a device carrying one group address on several of its RemoteValues is still returned once. This relies on a devices group addresses being fixed when its RemoteValues are created, which the library guarantees - assigning RemoteValue.group_address after Devices.async_add() was never supported and would now leave the index stale.
  • Dependencies are declared in pyproject.toml only - the library's own in [project.dependencies], the development tooling in the dev group of [dependency-groups] - and pinned, including transitive ones, in uv.lock. The requirements/ directory and tox.ini are removed; contributors need uv now: uv sync to set up, uv run pytest to test.
  • Git hooks are run by prek instead of pre-commit, from the same .pre-commit-config.yaml. Install them with uv run prek install, run them with uv run prek run --all-files. ruff, ruff format, mypy and pylint are local hooks executed via uv run --frozen, so their versions come from uv.lock alone - ruff is no longer pinned a second time in the hook config. script/run-in-env.sh is removed with them. The check-json hook is dropped - the repository tracks no JSON files.
  • Releases are built with uv build and uploaded by pypa/gh-action-pypi-publish using PyPI Trusted Publishing instead of twine with a stored username and password. Distributions now carry PEP 740 attestations.
  • Add support and test for Python 3.15.

Thank you

A big thank you to everyone who contributed code to xknx since 3.0.0:

@alexkn, @cpha-trifork, @enkore, @farmio, @kewde, @martinhoefling, @NHanser, @opieters, @pdgendt, @philippwaller, @richardpolzer and @somdoron