Skip to content

Add PostgreSQL backend and storage backend selection for KNX telegrams - #175673

Merged
farmio merged 10 commits into
home-assistant:devfrom
martinhoefling:pg-addition-rebase
Jul 14, 2026
Merged

Add PostgreSQL backend and storage backend selection for KNX telegrams#175673
farmio merged 10 commits into
home-assistant:devfrom
martinhoefling:pg-addition-rebase

Conversation

@martinhoefling

@martinhoefling martinhoefling commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Breaking change

No, builds on the introduced library for KNX Telegram storage in 2026.07

Proposed change

Build on the SQLite telegram store by adding an external PostgreSQL storage backend and a storage-backend selector to the options flow. Primary use case is to store long term history of KNX Telegrams in a separate time series database from HA for querying and multi tenancy with other analysis tools like Spectrum KNX.

Details

  • Add a storage backend dropdown ("Internal storage" / "PostgreSQL") to the communication settings step. SQLite routes to the existing store step; PostgreSQL routes to a new step collecting host, port, user, password, database and TLS, assembled into a DSN.
  • Validate the PostgreSQL connection (with a 10s timeout) before saving, mapping connection error kinds to translated form errors. TimescaleDB is optional since knx-telegram-store 0.9.0: it is auto-detected and used when available (hypertable + native compression), with a fallback to plain PostgreSQL otherwise.
  • Reuse the stored password when the PostgreSQL password field is left blank and mask credentials when building/parsing the DSN.
  • Construct a BufferedPostgresStore when the PostgreSQL backend is selected and surface the backend in the group monitor websocket base data.
  • Redact the telegram DSN in diagnostics.
  • Pull in the postgres extra of knx-telegram-store (bumped 0.3.2 -> 0.9.1). 0.9.0 percent-decodes the database name in DSNs and makes TimescaleDB optional with automatic detection; 0.9.1 translates libpq's sslmode DSN parameter to asyncpg's ssl connect argument so TLS connections work. Release notes: https://github.com/XKNX/knx-telegram-store/releases
  • Add tests for the PostgreSQL options flow, DSN parsing and round-trip, connection failure, malformed DSN handling, real expiry of the connection-check and store-init timeouts, and the init failure path.

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

Checklist

  • I understand the code I am submitting and can explain how it works.
  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.
  • Any generated code has been carefully reviewed for correctness and compliance with project standards.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies a diff between library versions and ideally a link to the changelog/release notes is added to the PR description.

To help with the load of incoming pull requests:

@home-assistant

home-assistant Bot commented Jul 5, 2026

Copy link
Copy Markdown
Contributor

Hey there @Julius2342, @farmio, @marvin-w, mind taking a look at this pull request as it has been labeled with an integration (knx) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of knx can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant mark-draft Mark the pull request as draft.
  • @home-assistant ready-for-review Remove the draft status from the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign knx Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant update-branch Update the pull request branch with the base branch.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component, problem in config, problem in device, feature-request) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component, problem in config, problem in device, feature-request) on the pull request.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the KNX integration's telegram storage subsystem (introduced for 2026.07) by adding an external PostgreSQL backend alongside the existing SQLite store, and a backend selector in the options flow. When PostgreSQL is chosen, a second options step collects host/port/user/password/database/TLS, assembles a DSN, validates the connection (10s timeout) with translated error mapping, and stores the DSN in config entry options. The DSN is redacted in diagnostics, the group monitor websocket base data now reports the concrete backend, and the knx-telegram-store dependency is bumped to 0.4.0 with the postgres extra. A config entry migration to v2.2 defaults existing entries to the SQLite backend and moves any DSN from entry data to options.

Changes:

  • Add a telegram_db_backend selector and a telegram_store_postgres options step with DSN build/parse/validation helpers and password reuse.
  • Select BufferedPostgresStore/BufferedSqliteStore in Telegrams, surface the backend via isinstance in the websocket, and redact the DSN in diagnostics.
  • Bump knx-telegram-store[sqlite,postgres]==0.4.0, add constants/typed options, migrate entries to v2.2, and add tests for the new flow and init paths.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
homeassistant/components/knx/config_flow.py Adds backend selector, PostgreSQL step, and DSN build/parse/validate helpers
homeassistant/components/knx/telegrams.py Selects SQLite or PostgreSQL store based on configured backend
homeassistant/components/knx/websocket.py Reports concrete backend via isinstance checks on the store
homeassistant/components/knx/__init__.py Adds v2.2 migration setting default backend and moving DSN to options
homeassistant/components/knx/const.py New backend/DSN constants and typed option fields
homeassistant/components/knx/diagnostics.py Redacts the PostgreSQL DSN in diagnostics
homeassistant/components/knx/strings.json Adds selector, form, and error translations
homeassistant/components/knx/manifest.json Bumps dependency to knx-telegram-store[sqlite,postgres]==0.4.0
requirements_all.txt Mirrors the dependency bump
tests/components/knx/test_config_flow.py Tests the PostgreSQL options flow, DSN parsing, failure/timeout paths
tests/components/knx/test_telegrams.py Tests invalid backend and PostgreSQL init-failure paths
tests/components/knx/test_init.py Tests the v2.xv2.2 migration
tests/components/knx/conftest.py Adds default backend to the mock config entry options
tests/components/knx/snapshots/test_diagnostic.ambr Updates snapshots for the new option

Comment thread homeassistant/components/knx/config_flow.py Outdated
@martinhoefling
martinhoefling marked this pull request as ready for review July 5, 2026 13:22
Copilot AI review requested due to automatic review settings July 5, 2026 13:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Comment thread homeassistant/components/knx/websocket.py
Copilot AI review requested due to automatic review settings July 5, 2026 14:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Comment thread homeassistant/components/knx/config_flow.py Outdated
Comment thread homeassistant/components/knx/manifest.json Outdated
Copilot AI review requested due to automatic review settings July 5, 2026 14:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Comment thread homeassistant/components/knx/config_flow.py
Comment thread tests/components/knx/test_config_flow.py
Comment thread tests/components/knx/test_config_flow.py
Comment thread tests/components/knx/test_config_flow.py Outdated
Comment thread tests/components/knx/test_config_flow.py
Copilot AI review requested due to automatic review settings July 13, 2026 19:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comment thread homeassistant/components/knx/manifest.json Outdated
martinhoefling and others added 8 commits July 13, 2026 21:57
Build on the SQLite telegram store by adding an external PostgreSQL
storage backend and a storage-backend selector to the options flow.

- Add a storage backend dropdown ("Internal storage" / "PostgreSQL") to the
  communication settings step. SQLite routes to the existing store step;
  PostgreSQL routes to a new step collecting host, port, user, password,
  database and TLS, assembled into a DSN.
- Validate the PostgreSQL connection (with a 10s timeout) before saving,
  mapping connection error kinds to translated form errors.
- Reuse the stored password when the PostgreSQL password field is left
  blank and mask credentials when building/parsing the DSN.
- Construct a BufferedPostgresStore when the PostgreSQL backend is selected
  and surface the backend in the group monitor websocket base data.
- Redact the telegram DSN in diagnostics.
- Pull in the postgres extra of knx-telegram-store (bumped to 0.4.0).
- Add tests for the PostgreSQL options flow, DSN parsing, invalid backend
  handling, connection failure, timeout and the init failure path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
- Drop DSN data->options migration that never shipped; comment when
  the 2.2 migration was introduced
- Parse current DSN once in the postgres options step
- Import knx-telegram-store at module level in config flow
- Use else for the sqlite backend branch; remove unreachable
  invalid-backend handling and its test
- Clarify that the TLS option does not verify the server certificate

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AC3osqhYegB84hvFNJLofB
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
- Make the store init and DSN check timeouts patchable constants and
  exercise real timeout expiry in tests instead of injecting TimeoutError
- Annotate knx_setup fixture parameters as AsyncMock
- Cover the malformed-DSN ValueError path in the options flow
- knx-telegram-store 0.9.0 percent-decodes the database name in DSNs and
  makes TimescaleDB optional; drop the now unreachable missing_timescaledb
  form error

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KGK6rMuaq1Zs44Ry6Wk7XS
Copilot AI review requested due to automatic review settings July 13, 2026 19:57

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comment thread tests/components/knx/test_websocket.py
Copilot AI review requested due to automatic review settings July 13, 2026 20:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Comment thread homeassistant/components/knx/config_flow.py
Comment thread homeassistant/components/knx/manifest.json Outdated
0.9.1 translates libpq's sslmode DSN parameter to asyncpg's ssl connect
argument, fixing connections with TLS enabled. Size the write buffer to
cover a full flush interval at the maximum KNX TP line rate so telegrams
are not dropped between flushes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KGK6rMuaq1Zs44Ry6Wk7XS
Copilot AI review requested due to automatic review settings July 13, 2026 20:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

@farmio farmio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thank you very much! 👍

@farmio
farmio merged commit 9eb29b8 into home-assistant:dev Jul 14, 2026
49 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 15, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants