Machine KB 0.1.0 is now available

[anotherjulien]

The first public release of the OpenWebNet Machine KB is now available: Machine KB 0.1.0.

It is a deterministic, machine-readable projection of the Encyclopedia, with 7,449 atomic claims, 1,423 reference records and 1,173 retrieval chunks, together with provenance, applicability, epistemic status, cautions and unresolved questions.

There is also now some consumer-oriented documentation covering direct LLM use, RAG/indexed retrieval and MCP-style adapters.

@GreenGrassBlueOcean, this might be a good opportunity to try having the MCP server ingest it and see how well it behaves in practice. Feedback from a real consumer would be particularly useful at this stage.

One caveat for that test: DALI/WHO24 knowledge is not in the Machine KB corpus yet, so anything relying on DALI should currently be expected to be missing.

Release: machine-kb-v0.1.0

Any feedback on the structure, ingestion experience, or what would make it easier to consume is very welcome.

[GreenGrassBlueOcean]

Thanks, and congrats on 0.1.0. The numbers and the provenance/epistemic-status fields look like what a consumer needs.

For context on where we are: the openwebnet-mcp server is only loosely coupled to the Encyclopedia today. It has a hand-curated WHO catalog that cites Encyclopedia pages, plus a doc index that can point at a local checkout of the markdown. It does not ingest the Machine KB yet, so we can't give real-consumer feedback so far. The MyHOME/OWNd cross-reference in #32 came from reading the code and docs side by side, not from the KB.

I'd like to take you up on the suggestion and try it. Plan:

1. Ingest the machine-kb-v0.1.0 release into the MCP as a retrieval source next to the current catalog. Claims and chunks would be the retrieval units, and provenance, applicability and cautions would come back with every answer.
2. Run the discrepancies from Cross-reference with MyHOME / OWNd: deviations, missing coverage and documentation gaps #32 as test questions (WHO 2 255, WHO 16 volume, WHO 4 x10/x11, WHO 15 CEN) and check that the KB gives the right answer with the right epistemic status.
3. Report back here on the ingestion experience and on where the schema helps or gets in the way.

Noted that DALI/WHO 24 isn't in the corpus yet, so I'll treat those gaps as expected and won't count them as failures.

[GreenGrassBlueOcean]

Thanks again for 0.1.0. We've now run it through a real consumer, and it went very well.

What we did

openwebnet-mcp now loads machine-kb-v0.1.0 (openwebnet-mcp#10). It follows CONSUMING.md closely: manifest first, version gate, SHA-256 check, paths taken from the manifest, and the id-registry honoured. The consumer docs made this easy. Loading takes about 0.5 s and a search 10-20 ms, so speed is a non-issue.

Quality: it holds up

We built a golden set from the discrepancies in #32. The expected answers come from the Encyclopedia pages cited there, and CI fails if quality drops. On 0.1.0:

• On the 11 questions we scored as passing, the right record was in the top 5 every time and ranked first for 82%.
• Every hit kept its epistemic status, applicability, provenance and cautions, and the schema made that easy to do. The KB never returned an answer for DALI/WHO 24 or a non-existent WHO, so the documented gap behaves as documented.
• We track 3 questions as known misses (WHO 16 volume down 1101, and the WHO 5 "state values are not commands" rule). They're ranking gaps on our side, and we'll work on them.

Making it world class: real traces

The Encyclopedia is already a great foundation. The next step is to back it with real bus captures. Today 2,703 claims are specified and 3,098 are corroborated_interpretation, and 1,379 are observed. Only 1 of 7,449 is experimentally_confirmed. That is where the opportunity is: a claim that says "confirmed on this gateway, this firmware, this date" beats any amount of careful reading.

We have real captures from MH200 hardware (dimmers, WHO 4 zone state, F422 scope commands, and an undocumented WHAT 19 fault). We'd be glad to contribute them, and we'd tag our golden cases as trace-backed so the harness can flag any claim that contradicts a capture.

A few ideas, if useful:

1. A claim could reference its evidence (capture id, hardware, firmware, date), so experimentally_confirmed becomes specific and checkable.
2. Which format do you prefer for contributed traces?
3. A per-value claim for the WHO 16 WHAT table would let a query like "volume down" find 1101 directly.
4. Do you have a timeline for the truncated claim statements (Clean internal Markdown AST data from generated claim statements #37)? We fall back to the chunk text for now, and that works well.

Thanks again. We'll share the trace-backed results here as we get them.

[anotherjulien]

Thanks, this is exactly the sort of real-consumer feedback I was hoping for. It is very reassuring to hear that the ingestion model, provenance, applicability and epistemic metadata are holding up in practice.

The retrieval results are particularly encouraging. The WHO 16 1101 miss is useful feedback too: I agree that some tables probably need finer claim granularity so individual semantic values can be retrieved directly rather than only through a larger table-level claim. That feels worth looking at as a general extraction rule rather than a WHO 16 special case.

On the truncated claim statements: #37 is already fixed on main. The 1,120 affected statements have been repaired, a fail-closed generated-text hygiene check was added, and the rebuilt corpus has no remaining AST-repr leakage. I have deliberately left the issue open until the fix is included in the next patch release.

The trace-backed evidence point also lines up very closely with the proposal in OpenWebNet-HA discussions #509.

If the Encyclopedia and Machine KB are proving useful enough for real-world use, I think the cleanest model is:

field capture → canonical sanitized evidence → Encyclopedia → Machine KB

The Encyclopedia repository would hold the publishable evidence behind its claims, with stable evidence IDs and enough context to know what hardware, firmware, configuration and test produced an observation. MyHOME/OWNd's Golden Corpus could then consume that evidence for its regression fixtures rather than being its canonical home. Synthetic implementation-specific fixtures would stay where they are.

That also gives us a clean distinction between observed and experimentally_confirmed: a preserved trace is an observation; a controlled, repeatable test can support experimental confirmation.

I am also considering a small dedicated collection tool based on the diagnostics work already documented in the Encyclopedia. It could persistently fingerprint an installation, its buses and devices, take successive configuration snapshots, and associate later passive captures or controlled tests with the same pseudonymous devices.

The real Device IDs, MACs, IPs and other installation identifiers would remain local to the collector. Repository exports would contain stable pseudonymous identities and semantically valid synthetic protocol values, so longitudinal captures remain correlatable without exposing the real installation.

Your MH200 captures sound like excellent material for testing this model. Rather than defining another ad-hoc capture format, I would be very interested in using them as an early contribution through the unified evidence path proposed in #509, assuming you also think the Encyclopedia / Machine KB have now demonstrated enough value to justify becoming that common evidence layer.

[GreenGrassBlueOcean]

Thanks, the field capture → canonical sanitized evidence → Encyclopedia → Machine KB model makes sense to me, and I agree with the observed vs experimentally_confirmed distinction. A preserved trace is an observation, and a controlled, repeatable test is what supports confirmation.

I see two ways for traces to reach the evidence layer, and I think we should support both.

1. Accepted pull requests (the #509 path)

• A contributor sanitizes a capture and submits it as a PR.
• A maintainer reviews it, and the merged evidence gets a stable ID.
• This is the reviewable, publishable route, and it fits traces that already exist, such as our MH200 captures for dimmers, WHO 4 zone state, F422 scope commands and the WHAT 19 fault.
• The weak point is provenance. The submitter has to state which hardware, firmware and configuration produced a trace, and a reviewer can't easily verify that.

2. A procedure built on the gateway MCP

• openwebnet-gateway-mcp gives an assistant live access to a gateway. It can connect, monitor the bus, parse frames, scan and send raw frames.
• I have used it extensively for debugging, testing and capturing traces. A scripted procedure records the context (gateway, firmware, bus, timestamp) next to the frames, and that is what could take a claim to experimentally_confirmed, because someone else can repeat it.
• The server is read-only by default. It classifies every frame and blocks anything that isn't a status or dimension request (*#WHO*WHERE*DIM##), so an assistant can't switch a load or move a cover by accident.
• It can be unlocked, with OPENWEBNET_ALLOW_WRITE=1 or --allow-write, and then it will send any frame to a live bus. That is the mode I used for controlled tests. The current mode is reported by get_connection_status.
• Caveat: I have only run it against my own MH200 (with an F422 scope), in my own installation. I haven't tested it on other gateways, firmware versions or bus layouts. Write mode is the part I'd be most careful about on someone else's installation.
• Passive capture and status reads are the part I'd recommend for wider use. Write-mode tests would need an explicit opt-in per test and a stated expected effect.

How I'd combine them

• Existing or passive captures go through path 1 and are tagged observed.
• Procedure-generated captures go through path 2. Their evidence record includes the test definition, so another installation can repeat it.
• Both feed the same evidence record (capture id, hardware, firmware, date), so a claim doesn't need to know which path produced it.
• Your collector idea fits path 2 well. The pseudonymisation would happen in the collector, before anything reaches the repository.

I'm happy to start with the MH200 captures as an early contribution under #509. In parallel I can write up the read-only capture procedure. Before it is presented as a reference, the gateway MCP repo needs some tidying: I opened #1 (stale ownd import path) and #2 (examples/ hardcode IP and password and mix read and write scripts).

[anotherjulien]

Thanks, I think we're converging quite nicely here.

I agree that both contribution paths are useful. Existing captures should absolutely be contributable through normal review, while a more structured acquisition procedure can give us much better context and reproducibility for new evidence.

One nuance I would add is that I would not tie observed vs experimentally_confirmed directly to the acquisition method. A scripted test on one installation is certainly stronger than an isolated trace, but it may still only establish behaviour for that particular gateway, firmware, bus topology and device configuration. Depending on the scope of the claim, I think experimentally_confirmed may need reproduction across genuinely independent contexts - different installations, buses, gateways and/or firmware - with the confirmation remaining explicitly scoped to what was actually tested.

That is also why I am increasingly interested in the collector being a little more general than a procedure built specifically around the gateway MCP. The MCP looks like a very useful acquisition backend, but I think the persistent identity, fingerprinting, snapshots, pseudonymisation, safety policy and evidence export should belong to a dedicated research collector above it. That leaves room for gateway MCP, OWNd or other acquisition methods without making the evidence model depend on any one of them.

The collector could also solve the provenance problem you point out: rather than relying entirely on a contributor saying "this was gateway X, firmware Y, configuration Z", it could diagnostically fingerprint as much of the installation as possible, associate later captures with the same pseudonymous installation/bus/device identities, and preserve successive configuration snapshots. Real Device IDs, MACs, IPs and similar identifiers would remain local; exported captures would use stable, semantically valid pseudonyms.

I would also distinguish passive capture, explicitly allowlisted non-destructive diagnostics, and active tests requiring deliberate opt-in, rather than treating every dimension request as inherently safe or unlocking arbitrary writes for general collection.

I've opened OpenWebNet-HA discussion #536 specifically to work out that centralized fingerprinting / research-data collection side without turning this release-feedback thread into the design discussion for the collector.

So in my mind the pieces are becoming:

collection / fingerprinting (#536)
        ↓
canonical sanitized evidence (#509)
        ↓
Encyclopedia
        ↓
Machine KB
        ↓
consumers / regression and retrieval tests

Your existing MH200 captures still look like very good first material for exercising that path, and the gateway MCP may well be one of the first useful acquisition backends for the collector.