Watch upstream_docs for changes and flag affected drivers
The problem
A driver decodes a device by following the vendor's own reference material: a
Modbus register map, a parameter/register changelog, a protocol manual. That
material lives on the vendor's site and changes without telling us — a
register renumbered, a parameter added, a scaling factor corrected. When it
does, the driver keeps running against a map that is quietly out of date, and
nobody notices until a value looks wrong on a real site months later.
We now record those documents in the manifest. The companion PR adds an optional
upstream_docs field (url, title, kind) and populates it for the two NIBE
drivers, both pointing at NIBE's myUplink register changelog:
upstream_docs:
- url: "https://www.nibe.eu/webdav/files/myuplink_changelog/nibe-n.pdf"
title: "NIBE S-series myUplink register/parameter changelog (nibe-n.pdf)"
kind: changelogThe field is inert on its own. This issue is about the half that gives it value:
a bot that routinely checks those URLs, surfaces when one changes, and asks
whether the driver is affected.
Proposal
A scheduled watcher (weekly schedule: GitHub Action is enough) that, for every
upstream_docs entry across all manifests:
- Fetches the URL and records a content hash +
Last-Modified/ETag.
- Compares against the last-seen state stored in the repo (e.g.
upstream-docs-state.json, append-only like driver-history.json).
- On change, surfaces it: opens (or updates) a tracking issue titled after
the driver + document, labelled upstream-doc-changed, linking the old and
new versions so a maintainer sees it in one place.
- Optionally reviews impact: attaches a first-pass assessment of whether the
change touches registers/parameters the driver actually reads, so triage
starts from "these three registers moved" rather than "a PDF changed".
The output is a nudge for human review, never an automatic driver edit — same
posture as the rest of the repo (a watcher proposes, a maintainer decides).
Design details and open questions
Change detection. A PDF's bytes can churn without the content changing
(rebuild timestamps, metadata). Hashing raw bytes over-fires; extracting text
first (pdftotext) and hashing that is more honest but adds a dependency. Start
with a raw-byte hash + ETag/Last-Modified short-circuit, and refine only if
noise is a real problem.
State storage. Keep last-seen {driver, url, sha256, checked_at, http_etag}
in a committed JSON file so history is auditable and diffs are visible, matching
how driver-history.json treats published versions as append-only facts.
Impact review. When a watched doc changes, the useful question is "does this
change any register/parameter this driver reads?" That can be a diff of extracted
text scoped to the register ids the driver references, optionally summarised by an
LLM step. Kept clearly advisory.
Robustness. Vendor URLs rot and rate-limit. The watcher must degrade quietly:
a fetch failure is logged and retried next run, never a false "changed". A
persistently-dead URL is its own signal (the doc moved) and worth surfacing after
N consecutive failures.
Surfacing venue. Issues here are the obvious home (Discussions are disabled
on this repo). A label + a stable issue title per (driver, doc) keeps it to one
thread per document rather than a new issue every run.
Schema follow-ups this may want. A per-entry sha256/checked_at could move
into the manifest itself, but that couples slow-moving metadata to a fast-moving
fact — the append-only state file is probably the better home. Worth deciding
before the watcher is wired up.
Scope / next steps
Companion PR: adds the upstream_docs field and the first two entries.
Watch
upstream_docsfor changes and flag affected driversThe problem
A driver decodes a device by following the vendor's own reference material: a
Modbus register map, a parameter/register changelog, a protocol manual. That
material lives on the vendor's site and changes without telling us — a
register renumbered, a parameter added, a scaling factor corrected. When it
does, the driver keeps running against a map that is quietly out of date, and
nobody notices until a value looks wrong on a real site months later.
We now record those documents in the manifest. The companion PR adds an optional
upstream_docsfield (url,title,kind) and populates it for the two NIBEdrivers, both pointing at NIBE's myUplink register changelog:
The field is inert on its own. This issue is about the half that gives it value:
a bot that routinely checks those URLs, surfaces when one changes, and asks
whether the driver is affected.
Proposal
A scheduled watcher (weekly
schedule:GitHub Action is enough) that, for everyupstream_docsentry across all manifests:Last-Modified/ETag.upstream-docs-state.json, append-only likedriver-history.json).the driver + document, labelled
upstream-doc-changed, linking the old andnew versions so a maintainer sees it in one place.
change touches registers/parameters the driver actually reads, so triage
starts from "these three registers moved" rather than "a PDF changed".
The output is a nudge for human review, never an automatic driver edit — same
posture as the rest of the repo (a watcher proposes, a maintainer decides).
Design details and open questions
Change detection. A PDF's bytes can churn without the content changing
(rebuild timestamps, metadata). Hashing raw bytes over-fires; extracting text
first (
pdftotext) and hashing that is more honest but adds a dependency. Startwith a raw-byte hash +
ETag/Last-Modifiedshort-circuit, and refine only ifnoise is a real problem.
State storage. Keep last-seen
{driver, url, sha256, checked_at, http_etag}in a committed JSON file so history is auditable and diffs are visible, matching
how
driver-history.jsontreats published versions as append-only facts.Impact review. When a watched doc changes, the useful question is "does this
change any register/parameter this driver reads?" That can be a diff of extracted
text scoped to the register ids the driver references, optionally summarised by an
LLM step. Kept clearly advisory.
Robustness. Vendor URLs rot and rate-limit. The watcher must degrade quietly:
a fetch failure is logged and retried next run, never a false "changed". A
persistently-dead URL is its own signal (the doc moved) and worth surfacing after
N consecutive failures.
Surfacing venue. Issues here are the obvious home (Discussions are disabled
on this repo). A label + a stable issue title per (driver, doc) keeps it to one
thread per document rather than a new issue every run.
Schema follow-ups this may want. A per-entry
sha256/checked_atcould moveinto the manifest itself, but that couples slow-moving metadata to a fast-moving
fact — the append-only state file is probably the better home. Worth deciding
before the watcher is wired up.
Scope / next steps
upstream_docsfor other drivers with a public register source.Companion PR: adds the
upstream_docsfield and the first two entries.