OpenWebNet-HA
RFC: Privacy-preserving real-world OpenWebNet evidence collection #536
anotherjulien
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
RFC: Privacy-preserving real-world OpenWebNet evidence collection
We increasingly rely on real-world OpenWebNet captures to document behaviour that is absent, incomplete, or ambiguous in official documentation.
Those captures are extremely valuable: they can establish device behaviour, previously undocumented dimensions and values, gateway differences, firmware-specific behaviour, and relationships that cannot be recovered from documentation alone.
At the same time, raw captures from real installations can contain persistent device or gateway identifiers. We need a repeatable way to collect useful evidence without asking contributors to publish their installation identities.
This discussion proposes a common model for collecting, pseudonymizing, preserving, and contributing real-world OpenWebNet evidence.
The intention is to agree on the model first. Implementation can then be split between MyHOME, OWNd, the Encyclopedia, and related tooling.
What we want to achieve
The goal is a collection process which can:
Nothing would be uploaded automatically.
Proposed responsibility split
OWNd
OWNd remains responsible for the OpenWebNet communication layer:
Research collection should not implement a second OpenWebNet stack.
A small future OWNd addition may be useful to expose exact application-level TX/RX observations, including command ACK/NACK frames, after authentication has completed.
MyHOME
MyHOME is a natural host for the collector because it already knows the configured gateways, holds their credentials, manages shared-bus topology, and has active gateway sessions.
The research subsystem would use those existing connections and command queues. It would not receive or persist gateway passwords.
The existing Bus Monitor remains a diagnostic/raw-traffic tool.
Research collection would be a separate subsystem with a separate user interface and a stronger privacy contract.
Encyclopedia / OWN Machine KB
The Encyclopedia remains the human-facing repository for established protocol knowledge and provenance.
The Machine KB provides a structured representation of that knowledge.
Captured evidence can support both, but neither should depend on undocumented local/private mappings held by a contributor.
Persistent research identity
The proposed identity hierarchy is:
These are random persistent UUIDs generated locally.
A bus UUID represents the physical SCS bus, not a particular gateway. Replacing a gateway should therefore not change the bus identity.
Two gateways connected to the same physical bus share the same bus UUID.
A separate, non-interconnected bus receives a different bus UUID even when it belongs to the same installation.
Protocol identifiers that must remain within constrained representations, such as 8-hex-character device IDs, receive appropriately shaped pseudonyms in captured frames. The UUID remains the durable research identity.
Privacy model
The collector would pseudonymize every field currently known to carry a persistent installation, gateway, or physical-device identity.
Examples include known hardware/device identifiers, full gateway MAC addresses, network addresses, and similar installation-specific identifiers.
Where useful, semantically significant portions may be retained. For example, a gateway MAC pseudonym may preserve the vendor OUI while replacing the installation-specific suffix.
Unknown protocol data is preserved
A fundamental goal of these captures is discovering things we do not yet understand.
For that reason, an unknown dimension, value, parameter, or frame form should not be discarded merely because its semantics are unknown.
The working rule would be:
At present we have no evidence that further identifier-bearing fields exist beyond those we know about, but that cannot be guaranteed.
A future capture may teach us that a previously undocumented field is in fact an identifier. Captures created before that discovery will necessarily contain the original value because the collector could not have known its meaning at the time.
This limitation should be explicit rather than hidden behind a claim of complete anonymity.
Privacy rules should therefore be versioned. If a new identifier is discovered, future captures can pseudonymize it and existing local captures can be reprocessed before export.
Repository captures can also record which privacy-profile version produced them.
Preserve the unknown as evidence
A device fingerprint should not be a fixed collection of only the dimensions we currently understand.
Instead it should contain:
For example, an unknown DIM47 response remains an observation:
If DIM47 is understood later, the original evidence remains available and can be reinterpreted without requiring the contributor to reproduce the installation.
Likewise, previously unseen values of known dimensions must not be constrained to our current enumerations.
Two research capture kinds
The proposal distinguishes two capture kinds while keeping one common capture format.
Fingerprint capture
An active, controlled interrogation.
The collector executes a versioned set of safe, read-only queries and records:
Fingerprinting also provides the bridge between protocol observations and identified real-world hardware. Where the available evidence is sufficient to identify a known device SKU or gateway model, subsequent captures and observed behaviour can be tied to that identified hardware rather than merely to an anonymous bus address or pseudonymous device.
This is particularly valuable when comparing behaviour between SKUs, hardware revisions, firmware versions, configuration methods, or gateway families: a passive observation can be associated with the fingerprinted device that produced it.
Fingerprint captures answer questions such as:
Fingerprinting is deliberately different from the existing diagnostic "Sweep Bus" operation.
Usage capture
A passive observation of normal bus behaviour.
No research commands are sent.
A contributor might capture:
The capture continues in the Home Assistant backend even if the browser is closed.
This lets us collect realistic sequences rather than only deliberately triggered test cases.
Optional predefined context such as
normal use,lighting,covers,heating,scenarios, oraudiocan be attached without requiring free-form personal descriptions.Proposed Home Assistant experience
Research collection would be opt-in and disabled by default.
Under the MyHOME integration configuration:
The explanation should make clear that:
Enabling research collection creates the local identity registry but does not automatically start collecting traffic.
Separate Research card
Rather than overloading the existing Bus Monitor, the proposal is to provide a separate OpenWebNet Research card.
The Bus Monitor remains an engineering/diagnostic tool with raw traffic, filters, diagnostic sweep, and direct command transmission.
The Research card would instead provide bus-centric actions such as:
It would have no arbitrary raw-command control.
Local storage
Three different classes of data should remain distinct:
Raw Bus Monitor
Transient in-memory diagnostics.
May contain real identifiers.
Not part of the research evidence system.
Research identity registry
Persistent and strictly local.
Contains the information needed to recognize the same installation, bus, gateway, or device in later sessions and assign stable pseudonyms.
This mapping is never exported.
Where possible, real identifier values themselves need not be stored: a keyed local digest can be used to recognize them and map them to random pseudonyms.
Research captures
Persistent but already pseudonymized.
They should never depend on possession of the private identity registry in order to be understood.
A possible layout is:
Fingerprint and usage captures use the same underlying capture schema and privacy machinery.
They are separate capture kinds because their purpose, collection process, and derived metadata differ.
Time and correlation
Repository evidence generally does not require the contributor's absolute wall-clock time.
Individual frames can therefore use relative timestamps from the beginning of the capture while preserving ordering and timing relationships.
Stable installation, bus, gateway and device pseudonyms make it possible to correlate observations:
The same public pseudonymous identifiers should remain stable for that local research installation unless deliberately reset.
Evidence provenance
One reason for formalizing this is to improve provenance in the Encyclopedia.
Rather than saying:
we should eventually be able to say:
A documentation claim can then be traced back to the actual pseudonymized observation without publishing the contributor's real installation identity.
What this proposal deliberately does not include yet
For an initial version:
The capture format should be sufficiently documented that other collectors can eventually produce equivalent evidence.
Questions for contributors
The main points I would like feedback on are:
Is the distinction between known identifiers and unknown protocol data the right privacy boundary for useful reverse-engineering evidence?
Are random persistent installation/bus/device identities preferable to deriving those identities from captured values?
Should gateway MAC pseudonyms preserve the real vendor/OUI portion?
Does separating fingerprint captures and usage captures as capture kinds, while keeping one common format and identity space, make sense?
Are timed passive captures useful? Which default durations would be most practical?
Are there other known OpenWebNet fields that should already be classified as identifying?
What additional metadata would make a capture meaningfully usable as evidence without revealing information about the installation?
Should the capture format be specified independently of MyHOME from the beginning so standalone/bench collection tools can produce the same format?
How should repository captures be organized and cited from the Encyclopedia?
The intent here is not to finalize every implementation detail in this thread. The useful outcome would be agreement on the evidence model and privacy contract, after which implementation work can be split into focused issues/PRs.
All reactions