Repository navigation
Releases: ztomer/multitop
Releases · ztomer/multitop
Release list
v0.67.0
v0.67.0 — the agent's MCP server and the Ops view's client are rmcp (2026-10-26)
Changed — MCP is the official SDK on both ends (owner ruling OW.60, queue MCP.MON)
multitop-agent mcpis rmcp 3.7 instead of a hand-rolled JSON-RPC loop:
2026-07-28 statelessly throughserver/discover, the older versions
throughinitialize. The wire was pinned against the real binary first
(crates/agent/tests/mcp_stdio.rs, green on 0.66.0 and on this release).- Every tool states readOnlyHint, destructiveHint, idempotentHint and
openWorldHint, and declares anoutputSchema; a destructive tool is never
listed or called. goh's MCP wire gate now judges the server on every push. - The Ops view's client opens each session with rmcp's Discover lifecycle
and sends every request through it; the kept tool list, the absent
reasons and the catalogue digest are read exactly as before. Proven
againstmcp_host1.99.25 on all three hosts and against the 0.66.0
agent; an older multitop is answered by the new agent too. - Upstream's deltas: a line that is not JSON is dropped (was -32700); a
stateless request without the 2026-07-28_metais -32602. - The musl agents grow (x86_64 774K to 1.6M, aarch64 700K to 1.4M): rmcp
and its runtime, uploaded once per build per host.
Added — the phone reads the server (ROADMAP A2.5)
- The paired screen of the Android companion shows the server's own count
(/api/health) and one line per host (/api/hosts), read over the pinned
TLS and nothing else; each line carries its state in a glyph and words.
Proven end to end against a realmultitop --servethe test starts.
Added — the Ops view draws the machine's verdict (servers ROADMAP O92)
machine.triageis the first row of every Ops card: what needs the
owner's hand, one row per item naming its stack, with the owner's own fix
quoted (or the plain fact that there is none) and the sources' words
under it; then each stack's quiet line, saying what was read there.
A cache kept by an earlier version does not parse and is asked again.
Fixed — a host nobody had read was counted healthy
/api/hostsgave a host with no reading"health": 100, and/api/health
counted it inhealthy: a server that had read nothing said every host was
well.healthis now absent until there is a reading,/api/healthadds
unread, and/api/hostsadds the host'snameandbreaching.
v0.66.0
v0.66.0 — the agent's tick is read from the cadence the estate publishes (2026-10-21)
Changed — the installed agent paces ITSELF, and the launchd job carries no interval (servers ROADMAP E53)
- THE TICK WAS A NUMBER THIS REPO CARRIED, AND LAUNCHD FIXES IT WHEN
THE JOB IS LOADED. The installed agent
(com.ztomer.multitop.ops-poll) ran on aStartInterval 390written by
tools/install_ops_poller.sh— a number in this repo, and an interval
launchd cannot change while a job is loaded. So the tick could not follow
the estate: had the publication moved from every 15 minutes to every 5,
the agent would still have read every 6.5 minutes — slower than the
publication, so its verdict could be two publications behind — and the
client's own warning would have fired on every tick, correctly, with
nothing anyone could do about it but reinstall the agent. - THE AGENT NOW READS THE CADENCE AND WAITS THE RATE IT IMPLIES.
multitop --ops-agentis the poll in a loop: one pass, then the tick.
The tick is the shortest cadence any host published, halved (the
rule, DECISIONS D15), less a five-second margin so the tick's own jitter
can never land a second past the rule — and the cadence is read from what
the estate published with its verdict (health://latest), through the
answers this client already keeps. A publication that moves is followed a
cycle later, with no reinstall and nothing for the operator to do. - THE INSTALLED JOB CARRIES NO INTERVAL AT ALL. The launchd job the
template writes keeps the agent ALIVE (RunAtLoad+KeepAlive) and
nothing else: there is no rate in this repo for the estate's cadence to
move under.--intervalis REFUSED by the template — a rate handed
to it is the defect the tick was raised about — and the only number in the
job is the restart throttle for a failed agent, which says nothing about
how often the agent reads. Installing over a job that carried one says
what is being dropped: "the job being replaced carried an interval of
390s - dropped: the agent paces itself by the published cadence". - THE OTHER WAY ROUND WAS NOT CHOSEN, AND WHY. The tick could have been
re-installed when the cadence moved: the agent rewrites its own plist and
re-bootstraps it. That was not chosen becausebootoutkills the job
that is running it — the agent would have to be killed from inside its
own pass — and because a failed swap leaves the estate with no agent at
all, which is a worse failure than a tick that is one cycle behind. The
other way round is now impossible to get wrong: there is no interval to
install, and no way to ask for one. - A CADENCE THE AGENT COULD NOT READ IS SAID, AND THE RATE IS KEPT. A
host that published no cadence keeps the rate this client was keeping for
it, and the log says so in the client's own words ("the cadence could not
be read: this client keeps its rate"); a host this client has never read
keeps the client's fallback rate, and the log says that too. Neither is
called a published rate, because neither was published — and the tick's
own line names the cadence it followed, the host it read it from, and how
old that reading is. - THE AGENT PICKS UP A REPLACED BINARY. The agent is a long-lived
process — that is what lets it pace itself — so an upgraded binary would
otherwise leave it running the old one, and the old one keeps the old
rule. It hands the process over to the binary that is installed (exec),
so the estate is read by the binary on disk and the launchd job keeps its
identity.
Measured (2026-10-21, on the live estate)
- BEFORE — the tick was the launchd interval this repo carried.
Measured on this machine from theateach kept answer carries (the
epoch second the PASS STARTED — the moment the client itself measures
its sampling rate against): 396 s, against aStartIntervalof 390.
The same tick read from the cache's write times: 392 s, 393 s, 395 s,
395 s, 395 s — the write lands when the gather lands, and the gather
takes a different number of seconds each cycle. - AFTER — the tick is the rate the published cadence implies: 445 s
(the rule, 450 s, less the five-second margin), measured the same way
from theatthe kept answers carry: 446 s, 446 s (the tick is
445 s, and the second is the sleep's own overshoot). - THE CADENCE IT FOLLOWS, AND WHERE IT READ IT FROM — the agent's own
log line, every cycle:
tick every 7.4 min (445 s) · the shortest cadence any host published: every 15 min (900 s), read from health://latest on 192.168.0.33, 5 s ago
— the900 sis the estate's own published cadence (cadence_sin
health://latest), not a number this repo carries, and the client keeps
it atrule_every_s: 450— so its own warning does not fire. - THE FALLBACK, MEASURED — the local host publishes no cadence (it has
no health suite), so the agent's log carries, every cycle:
127.0.0.1 · the cadence could not be read: this client keeps its rate (every 7.5 min)— the rate it was keeping, said in words, and not passed
off as a published rate. - THE HANDOVER, MEASURED — the agent noticed the installed binary had
been replaced under it and said so:the installed binary was replaced: this agent is now the new one.
v0.65.0
v0.65.0 — a timed-out call reads as UNKNOWN, and the estate's other readings stand (2026-10-21)
Fixed — the poll's timed-out call is a GAP in the reading, not a fault of the estate (servers ROADMAP E74)
- THE LOCAL HOST'S READING WAS THROWN AWAY WITH THE CALL THAT MISSED
ITS DEADLINE. The launchd poll's local host failed intermittently with
tools/call: no answer in 20s— 17 of 125 passes in the poll's own
log (measured 2026-10-21: 9 at atools/calldeadline and 8 at a
server/discoverone, so 13.6% of the passes). A missed deadline
ENDED the session, so the host's whole reading was discarded, the host
was reported as failed, and the sections the estate had answered were
never asked. The estate's health is read from these answers, so an
intermittent local failure was an intermittent blind spot — and it
was read as "a flaky tool", which is a failure that is not a fault. - A MISSED DEADLINE IS A GAP IN ONE ANSWER, AND THE SESSION SURVIVES
IT. A late answer carries the id of the request it answers, and this
client matches that id — so a late answer is read and discarded, never
taken for the next one. The old reason for ending the session ("a late
reply could be mistaken for the next one's") could not happen, and
paying for it was the whole of E74: the estate's other readings are
read again. A test drives a server that answers late and reads the
NEXT answer back, to keep the id match honest. - THE SECTION THAT MISSED IT READS UNKNOWN, AND NAMES THE CALL. The
frame draws⚠ containers unknown: docker.containers: no answer in 20s— the TOOL is named, where the old wording named the JSON-RPC
method (tools/call: no answer in 20s) and so named no tool at all: a
log of timeouts said nothing about which call had missed its deadline.
A section is now in one of three states and they are not the same:
ABSENT (the host does not offer it, and says why), FAILED (the
estate's own report —failed: ...— a finding), UNKNOWN (the
answer did not arrive —unknown: ...— a gap, never an all-clear and
never a fault of the estate). - THE HOST IS NOT FAILED, AND THE PASS IS NOT ALL-CLEAR. The pass
gathers the host, keeps the reading, and names the fact that is
missing:127.0.0.1 · gathered · containers unknown: docker.containers: no answer in 20s. The pass's exit code is still a failure while a gap
stands — a gap is not an all-clear, and a fresh answer that still
carries one says so too. - THE 20-SECOND BUDGET IS THE ESTATE'S, AND IT WAS NOT RAISED. The
deadline is the estate's own (OPS_TIMEOUT), the agent was not given a
longer or a second one of its own, and a missed deadline is reported
rather than waited out.
Measured (2026-10-21, on the live estate)
- THE LOCAL CALLS ON AN AWAKE MACHINE NEVER MISS IT. 100 runs of the
local agent'sdocker.containers: max 5.039 s, 1 over the
agent's own 5-second IO timeout (the daemon did not answer, and the
tool said so), 0 over 20 s. 100 runs ofcron.status: max
0.094 s, 0 over 20 s. The 20-second timeouts in the poll's log
are therefore not the call being slow — they are the machine not
answering within the estate's budget, which is also why 8 of the 17
failures were aserver/discoverdeadline: the agent did not even
answer the first request. - THE SAME ESTATE, THE SAME TIMEOUT, BEFORE AND AFTER. With a local
agent that answers everything exceptdocker.containers, on the live
estate:- before (v0.64.0):
127.0.0.1 · failed: tools/call: no answer in 20s, exit 1, and no answer file at all — the reading was
discarded, and the card went stale; - after (this build):
127.0.0.1 · gathered · containers unknown: docker.containers: no answer in 20s, exit 1, and the answer is
kept —crona value,containers{"Unknown": "docker.containers: no answer in 20s"}, and the absences keep their own reasons. - the frame it draws:
✓ cron 1 job okbeside⚠ containers unknown: docker.containers: no answer in 20s.
- before (v0.64.0):
Verified
tools/gate.sh --fullgreen (14/14); the Rust suite and the pytest
suite as the gate of record ran them.- The red before the fix: the four tests that carry E74 failed on the
shipped behaviour — "tools/call: no answer in 200ms" (the tool not
named), "a missed deadline does not end the session", "the section
that could not be read does not say so", and "the host was thrown
away with the call that timed out".
v0.64.0
v0.64.0 — the kept catalogue is checked against the server's own DIGEST (2026-10-21)
Changed — the kept list is checked against the server's WORD, not a guess (servers ROADMAP E72)
- THE KEPT LIST WAS CHECKED BY A GUESS. The client decided that its
kept copy of a host's tool list was still the truth by INFERring it from
two things it could see cheaply: the server's advertised version and
the set of tools the server says it does not offer. Both are hints,
not facts — and the guess was wrong often enough to measure: it sent the
client back for the whole catalogue at every release, and four of the
last ten releases touched no tool table at all (v1.99.13→14, 15→16,
16→17, 17→18), so those downloads were needless — and, the other way
round, a table that moved WITHOUT a release was invisible to it. - THE SERVER NOW PUBLISHES THE DIGEST OF THE EXACT LIST
tools/list
ANSWERS, inserver/discover, beside the absent map this client
already reads:dev.multitop/catalogueDigest, the SHA-256 of the
list's own canonical bytes, in the estate's spelling (sha256:and 64
lowercase hex). That is the server's own word about the list it serves,
and it is the fact the client checks its copy against — the server's
half ismedia_server7c3b02b1, shipped there and not written here. - A DIGEST THAT MATCHES KEEPS THE KEPT LIST — ACROSS A RELEASE. The
version moving no longer costs a download: the digest says the list is
byte-for-byte the one the client holds, so a release that touched no
tool table leaves the catalogue alone (149,997 B a gather measured
on the live estate today). The client asks for the list only when the
digest says the list moved. - A DIGEST THAT CHANGED PUTS THE KEPT LIST ASIDE, and the host is
asked again — and this is the case the guess could not see at all: the
same binary, the same absent set, a different digest. The frame says
so: "tool list: re-listed, the server's catalogue digest changed". - THE DIGEST IS REMEMBERED, NOT COMPUTED. It is of the whole list
— names and schemas — and this client keeps only the names, so it
cannot compute it again: it keeps the digest it was handed, beside
the list the word was given for. A digest over the names alone would be
a smaller fact and a weaker one (a schema change would go unseen); the
server publishes the stronger one, and the client takes it. - A DIGEST THE CLIENT CANNOT CHECK SAYS THE FACT IS ABSENT. A server
that publishes none — one too old to know the field, or the local
agent, which publishes none — has said nothing about its list: the kept
copy is then checked the OLDER way (the version and the absent set),
the frame says "the server published no digest", and that is not an
all-clear: an absent fact is stated, never drawn as a copy-is-fine. A
digest that is not in the estate's spelling is not a digest either, and
is treated the same way. - AND THE CLIENT DOES NOT RE-LIST EVERY SESSION. The absent digest
does not turn into a download on every pass — that is the defect this
closes, not a new one. A kept list taken before the server published a
digest is put aside once (it cannot be checked), and the digest is
remembered with the list from then on. - THE KEPT LIST IS STILL THE WHOLE LIST — an optimisation, never a
filter (E61's rule): every tool of it, every page of it, by name.
Measured (2026-10-21, on the live estate and on the shipped binary)
- THE ESTATE'S HALF OF THE BYTES: the digest costs +105 B per
session per host on the wire — measured on the live estate, the
server/discoveranswer grew by exactly that on every host when the
fleet shipped v1.99.21: 667 → 772 B on.33, 1,097 → 1,202 B on
.90, 1,711 → 1,816 B on.158— and it saves the 149,997 B a
gather the version inference forced at every release: the three
tools/listanswers, 65,557 B on.33, 51,716 B on.90, 32,724 B on
.158. A release that touched no tool table now costs 315 B (three
digests) instead of 149,997 B (three catalogues): 149,682 B a
gather saved. - THE CLIENT'S HALF, ON DISK: the kept list grows by 87 B per host
(672 → 759 B on.33— the 71-byte digest and the JSON around it) and
the answer the poll keeps grows by 108 B per host per gather (its
catalogue field, 39 → 147 B — it now carries the proof). Neither is on
the wire. - THE PASS, WITH AND WITHOUT THE KEPT LISTS, measured with the
counting shim in front ofssh(tools/measure_poll.py): 149,024 B
a gather with the kept list (notools/listline in the per-answer
table at all) against 299,023 B with them dropped — the same
149,997 B of catalogue, measured twice. - AND THE FRAME SAYS IT, ON THE LIVE ESTATE: "tool list: kept 24m
old (window 1h), the server's digest matches". - THE MIGRATION, ON THE LIVE ESTATE: the fleet shipped the digest
while this was being written, and the lists this client had kept under
the older servers carried no digest — so they were put aside ONCE
(the one path where a kept copy cannot be checked at all), and the
digest was remembered with the list from then on. Measured on.158:
the answer saidRelisted/Uncheckableand the frame said "the kept
list carried no digest to check against"; the next session said
"tool list: kept 4s old (window 1h), the server's digest matches",
and the host was not asked for its list again.
Verified
tools/gate.sh --fullgreen; the Rust suite green.- The tests were red first: a changed digest did not put the kept list
aside, a release that changed no tool re-listed the catalogue anyway,
the digest was not kept with the list, and the frame said nothing about
the digest — all four failed against the old code, and pass now.
v0.63.0
v0.63.0 — the Ops poll stops re-downloading the TOOL CATALOGUE (2026-10-21)
Added — the tool list is kept, keyed to what determines it (servers ROADMAP E61)
- THE OPS POLL PULLED THE WHOLE CATALOGUE, EVERY GATHER. The list of
tools a host offers is the biggest thing this client pulls off a host —
65,467 B on.33, 51,716 B on.90, 32,724 B on.158, measured on
the live estate — and it is the SAME list every time: it changes at a
RELEASE, not every six and a half minutes. At the scheduler's own rate
(a pass every 390 s) that is 149,907 B a gather, 33.1 MB a day, of
one immutable list, over and over. - THE KEY IS (the server's advertised version, the set of tools it says
it does NOT offer), and BOTH are read from the LIVE server on every
session, so the kept list cannot hide a change: a release moves the
version, and a host that gains or loses a tool moves the absent set. A
kept list taken under another binary, or under another tool set, is put
aside and the host is asked again — and the reasons the server gives for
what it does not list are read live every session, so they are never
the kept copy's. - THE SERVER'S OWN
ttlMsBOUNDS IT — an hour, for a list — so this
client never keeps an answer longer than the answer's owner says it is
good for. A window of zero is the protocol's "a live fact": there is
nothing to keep, so there is nothing to use, and nothing is kept. A
server that names no version is not keyed at all, so nothing is kept
either: the version is the only thing that says which binary holds the
tool table. - THE CACHE IS AN OPTIMISATION, NEVER A FILTER. What is kept is EVERY
tool the server listed, every page of them: a tool the kept list omitted
would lose its section from the panel, and a list that kept only the
first page would lose every tool on the second. - WHERE THE LIST CAME FROM IS ON THE PANEL. The list is the panel's
SHAPE — a tool the list does not name has no section — so the frame says
it: "tool list: kept 12m old (window 1h)", "tool list: from the
server (window 1h)", and, when a kept list was put aside, WHY: "tool
list: re-listed, the kept list was for 1.99.19, the server is 1.99.20",
"… the kept list had expired (2h old, window 1h)". A list nobody can
date is a shape nobody can trust, and a list quietly replaced would hide
the very thing the operator needs to know.
Fixed — the kept lists cannot be taken for the answers
- THE OTHER CLIENT READS THIS CACHE DIRECTORY TO FIND WHAT THE MONITOR
KNOWS, and it takes the FIRST file whose name holds the host — so a
tool list kept beside the answer, named for the same host, would have
been taken for the answer itself, and the parity check would have read a
tool list where it expects the estate's own answers. The lists live in
the cache directory's owncatalogue/directory now: out of that
reader's way, and the directory still holds one answer per host and
nothing else.
Measured, on the live estate (2026-10-21)
- THE LIST, A GATHER:
tools/listcosts 65,467 B on.33,
51,716 B on.90and 32,724 B on.158— 149,907 B a gather
across the estate. - THE PASS, BEFORE AND AFTER, measured on the shipped binary with a
counting shim in front ofssh(tools/measure_poll.py): the whole
poll cost 298,728 B a gather (66.0 MB a day) with the released
client, and 148,817 B a gather (32.9 MB a day) with the kept list —
and the kept list is pulled an hour, not a gather (3.6 MB a day). So
the poll's inbound bytes fall from 66.0 MB a day to 36.5 MB a day,
and the catalogue's own share from 33.1 MB a day to 3.6 MB a day. - THE FALL IS THE CATALOGUE, TO THE BYTE: 298,728 − 148,817 =
149,911 B a gather, against the 149,907 B the threetools/list
answers cost on their own — and the warm pass's per-answer table has no
tools/listline in it at all. The discover answer (667–1,711 B) is
still read every session, because that is what says whether the kept
list is still the truth. - A RELEASE STILL RE-LISTS: every kept list was given another binary's
version, and the next pass asked every host for its list again — the
whole poll's bytes back to the cold figure (298,726 B) — and the
client said why, in its own answer ("Relisted — the kept list was for
1.99.19-before-a-release, the server is 1.99.19") and on the panel
("tool list: re-listed, the kept list was for 1.99.19-before-a-release,
the server is 1.99.19").
Reported, not written — the better place may be the server
- THE CATALOGUE IS A PER-BINARY CONSTANT, AND THE SERVER KNOWS IT. The
estate's plan says the better place for this may be the server
(media_server's lane), and it is: the tool table is fixed in the
binary, so the server could publish the list's DIGEST beside the absent
map it already publishes inserver/discover— a few dozen bytes — and
then any client could check its copy against the server's own word
instead of inferring it from the version and the absent set. That is
reported to the estate's plan; nothing was written in that repo.
Verified
tools/gate.sh --fullgreen; the Rust suite green.- The kept lists are the client's own copy and nothing else: the cache
directory still holds one answer per host, and the tool lists live in
catalogue/, where the reader that looks for the answers does not look.
v0.62.0
v0.62.0 — the rate is READ from the estate, and the Ops frame can be PRINTED (2026-10-21)
Changed — the client's sampling rate (servers ROADMAP E53)
- THE RATE WAS A NUMBER COPIED INTO THIS CLIENT.
MAX_SAMPLING_INTERVAL_S
(450 s) was half of a fifteen-minute publication, written down here, and
the due rule and the staleness threshold were written against it. The
estate's cron can move — and if it moved to every five minutes, this
client would have kept sampling every 7.5 minutes and neither of its
own staleness thresholds would have noticed. The server now publishes its
publication cadence with the verdict (health://latest, beside the age:
cadence_sandcadence), and this client derives its rate from
what it read: half of the cadence the server published. - THE WARNING IS DRIVEN BY THE PUBLISHED CADENCE, not by the local
copy: a client that samples slower than half the cadence the server
published says so where the verdict is shown — "sampled every 15 min,
server publishes every 5 min: two publications behind". The age on the
line cannot say it: a verdict two publications behind looks exactly like
a fresh one. - A CADENCE THE SERVER COULD NOT READ IS NOT A WARNING. It is a gap in
the reading, not a fault: the client says the rate could not be read
("the cadence could not be read: this client keeps its rate (every 15
min)") and keeps the rate it had. - THE STALENESS THRESHOLD FOLLOWS THE CADENCE TOO — the published
cadence plus a grace for a suite that is still running — so a verdict is
stale against the estate's own publication, not against a number typed
here. - THE CACHE CARRIES THE RULE AND THE TRUTH. A kept answer holds the
rate the client KEEPS (the rule) and the rate the answer was TAKEN AT
(the rule, raised to the gap when the client missed its schedule). A
missed cycle is not a new rate: a client that missed one must not keep
sampling at half the rate for ever.
Added — the Ops frame can be printed (servers ROADMAP O90)
multitop --ops-frame <HOST>prints the Ops frame for one host,
uncoloured: the same lines the Ops view draws, from this client's own
answers (the cache the poll writes), and it asks the estate for nothing.
The host is named by the name in its[[servers]]entry or by its
address. The other client's parity check (routines parity) reads it
through--monitor-cmd, so the two renderings can be compared without
reading a terminal — the monitor draws its Ops view in its TUI and keeps
no copy of it, so the check had to read the monitor's screen and say out
loud that the rendered half was a stand-in.
Fixed — a machine that declares nothing is not a fault (servers ROADMAP E26)
- THE
vpnLINE READ A HEALTHY MACHINE AS A FAULT. The server's own
reason for not offeringnet.egressreads "the machine's instance
declaration is missing or unreadable (…): its instances are unknown",
and the monitor repeated it. E26's rule is that a machine which declares
nothing has one instance — itself — and says so; the routines card says
"1 stack: adguard_server · no declaration". The monitor now says what
it knows, in the card's words, and a declaration that EXISTS and could
not be read still keeps the server's own reason — that one IS a fault.
Measured, on the live estate (2026-10-21)
- THE CADENCE IS PUBLISHED, on all three hosts:
cadence_s: 900,
cadence: "every 15 min"(.33,.90and.158, read over ssh with
resources/read health://latest). The client's rate is therefore
450 s, and the answers say so:rule_every_s: 450. - THE GATHERS A DAY: 221 per host — the launchd agent's tick is 390 s
and every tick finds a gather due (measured over six ticks:
390–391 s between them), against the rule's own floor of 192
(450 s). - THE BYTES A DAY: 60.3 MB across the estate, from 153,111 B
(.33), 77,273 B (.90) and 42,341 B (.158) per gather —
measured withpython3 tools/measure_gather.py <label> <ssh-target> <mcp-command>, which asks the host's ownmcp_hostover ssh for exactly
the requests the gather makes and counts the bytes each reply crossed the
wire for. - THE FRAME FOR A HOST WITH NO DECLARATION (
.158), from
multitop --ops-frame pihole:
· vpn no tunnel on this machine: adguard_server, no declaration
— where it read "no Docker on this machine: no daemon socket; the
machine's instance declaration is missing or unreadable
(/home/ztomer/prj/adguard_server/config/instances.tsv): its instances are
unknown". The machine's own facts are on the line above it
(· stacks adguard_server: 1 instance, no declaration), and the
containers section still says whether the machine has Docker at all.
Verified
tools/gate.sh --fullgreen (14/14); the Rust suite (1,794 tests across
the workspace) and the pytest suite as the gate of record ran them.- The other client's parity check now reads the monitor's RENDERED frame
(routines parity --monitor-cmd "multitop --ops-frame"): 23
differences across 3 hosts, every one named, and the note that the
rendered half was a stand-in is gone.
v0.61.1
v0.61.1 — the poll's rate lands INSIDE the rule, whatever the scheduler does (2026-10-21)
Fixed — the poll's schedule
- launchd's timer wheel has a granularity of its own. Measured on this
Mac on 2026-10-21: a 420 sStartIntervalfired 425 to 470 s after
the last run, so the client's read rate was sometimes 7 minutes 50
seconds — outside the rule's 7.5, and the card would have said so on the
days it drifted. The agent's tick is now 390 s, which puts every
gather between 6.5 and 7.5 minutes: inside the rule every time. - THE DUE RULE FOLLOWS THE SCHEDULER. The age a host must reach to be
gathered is now 90 s under the poll's interval, so a tick that lands early
still finds a gather DUE: the client's read rate cannot halve because a
second was lost. - AN ANSWER SAYS WHEN THE CLIENT MISSED ITS SCHEDULE. The rate an
answer carries is the poll's own — unless the gap since the last answer
is LONGER, in which case the gap is the truth. A client that slept
through a cycle says so, rather than carrying a rate it did not keep.
Measured, on the live estate (2026-10-21, after the fix)
- The gathers: every 390–450 s on
.33(measured: 406 s between
two gathers), against the retired rate's 30 s. - The answer carries 450 s, so the aliasing warning stays silent
because the client IS inside the rule — and it fires the moment the
client is not.
Verified
tools/gate.sh --fullgreen (14/14); the Rust suite (1,773 tests across
the workspace) and the pytest suite as the gate of record ran them.
v0.61.0
v0.61.0 — the Ops poll is a JOB: it runs whether or not anyone is watching (2026-10-21)
Changed — the Ops poll (servers ROADMAP E36)
- THE POLL NEVER RAN. It was spawned by the Ops VIEW's keypress and by
nothing else, so the estate's fingerprint tools fired 0.3 a day while
the shared tools fired 87: the Ops view was shipped, installed, and nothing
asked the estate for it. The code said so out loud — "Polled every 30 s
while the view is up, and never while it is not" — so this was a design,
not a bug, and the design was the defect. - THE POLL IS A JOB NOW.
multitop --ops-pollis one pass: it gathers
the hosts whose answer is due and keeps what it found beside the
config.tools/install_ops_poller.shinstalls the launchd agent that runs
that pass every 7.5 minutes. A pass that exits cannot be left half-dead;
the next pass opens a fresh session. - THE RATE IS THE RULING'S. DECISIONS D15 (owner, 2026-10-20): the
estate publishes every 15 minutes, so a client reads at least every 7.5
minutes. The 30 s rate is retired — it gathered 2,880 times a day, ten
times what the rule needs, and every gather moved the whole tool
catalogue off the host. The interval is now DERIVED from the rule
(OPS_POLL = MAX_SAMPLING_INTERVAL_S), so the two cannot drift apart. - ONE CARD, ONE READ OF THE ESTATE. The answer a pass found is kept, and
a host whose answer is younger than the interval is not asked again — by
the pass, and not by the view either. The two pollers share one due rule,
so the client's second reader costs the estate one read and not two. - THE ANSWER CARRIES THE CLIENT'S OWN RATE. A client that samples
slower than half the publication interval says so where the verdict is
shown, on a line of its own (appended to the summary it was cut off by the
panel's width, which left the meaning off the end of the sentence). - THE CACHE KEEPS THE SERVER'S OWN WORDS. The raw answers are kept, and
the client parses them when it draws: an answer read back from the cache
is parsed by exactly the code that parses a live answer.
Measured, on the live estate (2026-10-21)
- One gather, in bytes the host sends:
.33128,455 B (of which the
tool catalogue is 63,749 B),.9075,574 B,.15840,988 B. - A day at the rule's rate (192 gathers a host):
.3324.7 MB,
.9014.5 MB,.1587.9 MB — 47.1 MB a day for the estate,
against 706 MB a day at the retired 30 s rate: a fifteenth. - The client: one pass over all four hosts is 0.55 s of wall time, so
1.8 min a day — against 26 min a day at the 30 s rate. - The estate's own time for one gather on
.33is 15 ms of the
server's own measurement (the audit log'smscolumn), so 2.9 s a day at
the rule's rate.
Verified
tools/gate.sh --fullgreen (14/14); the Rust suite (1,772 tests
across the workspace), the pytest suite and the coverage gate as the gate
of record ran them.- Each new behaviour was proven RED first: the interval, the warning on
the line, the due rule, the pass that gathers what is due, and the view
that shows the answer it already has each fail with an assertion when the
behaviour is taken away.
v0.60.0
v0.60.0 — the Ops view samples at TWICE the estate's publication rate (2026-10-20)
Changed — the Ops view's health section (DECISIONS D15)
- The owner's ruling, 2026-10-20: "don't we need to double the data
polls at twice of routine's polls to keep with the sampling lemma?"
Yes. The estate publishes its health verdict every 15 minutes, and a
client that samples at the PUBLICATION rate can ALIAS: its read can
land just before a publication and then wait a whole cycle, so its
verdict is two publications behind the truth. - THE OPS POLL IS NOT CHANGED. The owner has not ruled on this
client's interval, and E36 (the poll is not running) is not this
release's to reopen. The Ops poll already samples every 30 s — far
inside the rule — and the health section now carries the rule itself
(ops::MAX_SAMPLING_INTERVAL_S), so a client whose interval was ever
set too slow says so where the verdict is shown. - A client that samples TOO SLOWLY says so, in its answer — not in
silence. The age on the line cannot say it: a verdict that is two
publications behind looks exactly like one that is fresh.
Verified
tools/gate.sh --full14/14 green; the Rust suite, the pytest suite and
the coverage gate as the gate of record ran them.- The Ops view still never runs the suite — the same test as before.
- Each new behaviour was proven RED first: the sampling rule and the
warning it carries each fail with an assertion when the behaviour is
taken away.
v0.59.0
v0.59.0 — the Ops view's health verdict is the PUBLISHED one, and it is DATED (2026-10-20)
Changed — the Ops view's health section (servers ROADMAP E12, the client half)
- The verdict is READ, and it is the estate's own. The Ops view already
readhealth://latestand never ran the suite; what it did with the
answer was to count the checks. It now carries the whole verdict: the
machine's own freshness (oldest_age, in the SERVER's words —13 min ago), each check's own age, and the suite the verdict belongs to. - THE AGE IS ON THE LINE, and that is the point. The clients read the
PUBLISHED verdict instead of re-running the server's 50-check suite, so
what the section shows is as old as the estate's last cron run. A reader
that cannot date a verdict cannot tell a 3-minute-old answer from a
29-minute-old one — and those are different answers about the same
machine. So the age is on the line, in the server's own words (one
formatter dates a verdict everywhere it is shown), and a verdict past the
estate's interval says so and is NOT green. - THE VERDICT IS THE MACHINE'S PERIODIC SUITE. The published table
carries the DAILY checks' verdicts too, and those are a day old by
design, so a count that included them would be paired with an age that
does not describe them. The suite is the server's own answer
(health.list), and the client asks: it never guesses which checks count. - A machine that published NO verdict says so —
⚠ health no verdict published: nothing has run on this machine— and it is never an
all-clear: "could not ask is never fine", and an empty table is not a
green tick.
Verified
tools/gate.sh --full14/14 green; the Rust suite, the pytest suite and
the coverage gate as the gate of record ran them.- The Ops view still never runs the suite. A test drives
ops::gather
against a fakemcp_hostand asserts that no request named
health.run_allorhealth.runwas made: the verdict is read, never
run. - Each new behaviour was proven RED first — the age on the line, the
stale marking, the absent verdict and the narrowing to the suite each
fail with an assertion when the behaviour is taken away.