v0.0.11 — a host that is connected finally looks connected

A watcher whose whole job is telling you what your machines are doing was bad at telling you whether it could reach them. A connected-but-idle host looked much like a missing one, which is a good way to go restart a ComfyUI that was running fine.

Connected is now a lit state

The cause was different in each of the two card styles, so both were fixed:

• Rack card (1U Rack / Obsidian Glass / Obsidian Halo): an online-idle card and an offline card both painted the same slate jewel — identical, right down to the colour value. A reachable host now holds a jade standby colour, and a new LINK lamp with its own label sits beside the status word: dark lens means nothing is answering, jade means it is, amber blink means still connecting.
• Reactor panel (Control Room / Reactor Console): offline already read correctly here — what failed was the healthy case. A connected idle host lit nothing at all, which is also exactly what an app talking to nobody looks like.

The reactor panel's annunciators, rebuilt

• The bank drops from nine tiles to one row of four: Reactor Run, Cycle Done, Fault, Offline. The rest were long rectangles repeating facts the panel already stated — Queued is the QUEUE figure, Batch Run is the workflow bulb row that only exists on a batch job, No Step Data is STEPS LEFT already printing N/A. Tiles are capped and clustered left instead of stretched, so the panel's width goes to the instruments.
• LINK UP moves into the header as a lit pill, hard right: LINK UP / LINKING / OFFLINE, with exactly one always lit — an unlit header is no longer something a reachable host can produce.
• The status word does something. ONLINE never changed once painted; it now ticks on every update the collector pushes. The pill says a link exists; the tick says data is still arriving, so a link that dies without closing shows a frozen word instead of a confident green light.

Note: Relay Absent no longer has a lamp on the reactor panel. It is still reported per host in the (i) panel.

Also in this build

electron-builder is pinned to 26.15.7 (npm's latest tag still points at the older 26.15.3, so a caret range resolved backwards). That pin keeps the installer payload on a filter the install-time extractor can decode — see the v0.0.10 notes.

Verified before publishing

• npm test 5/5, npm audit 0 vulnerabilities.
• Checked against both live ComfyUI hosts, and the states a live rack cannot produce (offline, connecting) driven through the widget mock: the pill and the tick behave across a full drop-and-reconnect cycle, and a running module keeps its amber job colour while the link lamp stays jade.
• The node pack ships as a resource, so its contents were hashed inside the package and match the repo file for file; the installer payload was unpacked and contains no BCJ2 block; the packaged exe was launched (0.0.11.0, window up).

Installing the ComfyUI node pack is still the one manual step: copy resources/comfyui-relay/ into your ComfyUI custom_nodes/ and restart. The app's (i) panel prints the path and opens it for you.

v0.0.10 — all three installers at the same level

Windows, Linux AppImage and Linux deb are all cut from the same commit for the first time since 0.0.7. No new features — this release exists to close the gap.

What was out of step

• The 1s VRAM poll landed right after the 0.0.9 exe was built, so the relay bundled in that installer still refreshed the VRAM row every 2 seconds and visibly lagged nvidia-smi. It polls once a second here (still 2s on hosts without pynvml, where every read costs a subprocess).
• Linux stayed at 0.0.7. WSL2 stopped starting on this machine on 2026-08-16 and the AppImage/deb could not be built for 0.0.8 or 0.0.9. Virtualization is back, so both Linux artifacts skip straight to 0.0.10 and now carry everything from those two releases:
  • Electron 31 (end-of-life) → 43, electron-builder 25 → 26
  • VRAM read through NVML the way nvitop reports it, including the v2 memory struct so driver-reserved memory is not counted as used
  • the workflow-stage NODE line, with subgraph-expanded node ids collapsed to the node you actually see on the canvas
  • reactor dial figure clear of the arcs

Installer payload (why the .exe is ~8 MB bigger than it first was)

These artifacts were rebuilt on electron-builder 26.15.7. Under 26.15.3 the bundled 7za applies the BCJ2 filter to executable content, which the Nsis7z extractor inside the installer cannot decode — an install can silently drop the main .exe and every DLL and still report success (electron-builder #9983). This project's installers were not affected in practice, but 26.15.7 pins the payload to single-stream BCJ so it cannot happen; the archive compresses a little worse as a result.

Verified before publishing

• npm test 5/5. npm audit: 0 vulnerabilities.
• The ComfyUI node pack ships as a resource, so its contents were hashed inside each package: the exe, the AppImage and the deb all carry a relay byte-identical to the repo.
• The Windows installer payload was unpacked and inspected — no BCJ2 block remains.
• The packaged Windows build was launched (ProductVersion 0.0.10.0, window up), not just checked for a zero exit code.

Installing the ComfyUI node pack is still the one manual step: copy resources/comfyui-relay/ into your ComfyUI custom_nodes/ and restart. The app's (i) panel prints the path and opens it for you.

v0.0.9 — VRAM that matches nvitop, and a workflow-stage readout

VRAM now reads what nvitop reads

The canvas node's VRAM row was showing 3.7 GB on a card that nvitop, nvidia-smi and guiTOP all
showed at 9.5 GB. Six gigabytes off, and the watcher was the only tool on the box saying it.

Two separate causes, both fixed:

• ComfyUI's /system_stats counts PyTorch's idle allocator cache as free memory. That is
  correct from ComfyUI's point of view — it is memory it can reuse without asking the driver — but
  it is not what any monitoring tool means by "free".
• Even with that added back in, PyTorch and NVML disagree by about a gigabyte on the same card
  at the same instant. So the figures now come from NVML, the same source nvitop reads, through a
  new /watcher/vram route in the node pack.

Two traps that were only visible against a real box:

• The device index is not a join key. On a multi-GPU machine PyTorch orders devices by
  capability and NVML orders them by PCI bus — device 0 to PyTorch was device 2 to nvidia-smi.
  Matching index to index would have printed one card's memory on another card's row and looked
  entirely plausible. The cards are matched by UUID.
• NVML's memory query has two versions and only the second is nvidia-smi's. The older one
  counts driver-reserved memory as used (306 MiB on the test card). The node asks for v2.

If NVML is unavailable — an AMD box, or no pynvml — it falls back to ComfyUI's numbers with the
cache added back, which is close but reads a little high.

New: which part of the workflow is running

Every job layout now carries a NODE line: 4/21 KSampler.

STEP 7/20 tells you where you are inside one node. It does not tell you that the node is a
sampler rather than a VAE decode, or that there are fourteen more nodes after it. A graph spends
most of its wall clock loading, encoding and decoding either side of the sampler, and none of that
was visible before.

• The position is counted from the nodes actually seen executing.
• The total comes from the relay, which counts the nodes reachable from the outputs being run and
  subtracts the ones ComfyUI reports as cached. Without the relay you get the position on its own
  rather than a made-up total.
• Subgraphs are handled. ComfyUI executes a subgraph's contents under expanded ids
  (193:120, 193:119, ...) and all of them belong to one node on your canvas. They count once,
  and the line shows the name of the node you can actually see.
• Nothing has run yet → a dash. Never a zero.

Also in this release

• Electron 31 → 43. Electron 31 has been end-of-life since December 2024 and was no longer
  getting security fixes. Also updates electron-builder 25 → 26.
• Reactor panel, control-room idiom: the dial figure no longer overlaps the arcs, and it is set
  in near-black ink with a lit halo so the state colour still reads at a glance.

Note on this release

Windows only. The Linux AppImage and .deb are built under WSL2, which will not start on the build
machine right now (virtualization disabled in firmware). Linux users: 0.0.7 is the current build,
and the node pack changes above can be taken from the repo's comfyui-relay/ folder directly.

v0.0.8 - ComfyUI canvas node pack, draining ETA

The ComfyUI node pack is the headline

comfyui-relay/ was a one-file shim that rebroadcast progress messages so the widget could see
foreign jobs. It is now also a pack of ten display nodes you drop on the ComfyUI canvas — the
watcher's readouts, drawn on the graph itself, so you can see step position, rate, elapsed, ETA and
VRAM without leaving the tab you are working in.

• Four layouts — Four Wells, Plate, Bar, Trace — each with and without a VRAM strip,
  plus two standalone VRAM nodes (the cards your workflow selected, or every card on the box).
• Style is a widget on the node, not a node type. Switching between rack and glass never
  costs a node its wires or its position.
• The VRAM node shows only the cards the graph actually uses: a MultiGPU device widget naming
  cuda:1 is honoured, and with nothing pinned it falls back to ComfyUI's own device. The face
  prints which rule applied.
• Install: copy comfyui-relay/ into custom_nodes/, restart ComfyUI. It ships inside the
  installer too, under resources/comfyui-relay/ — the (i) button in the app tells you where.

A finished run keeps its numbers. The rate used to blank to N/A the moment a job ended, throwing
away the figure the node had just spent the whole run measuring while the step count and elapsed
time beside it stayed. Now steps, rate and elapsed all hold, and the ETA well — which has nothing
left to say — relabels itself to FINISHED, or FAILED / STOPPED. A node that has never run keeps
its ETA label; there is no end state to report.

The ETA counts down between steps

On a MiniMax-H3 sampler at 20-30 seconds per step, steps left / rate only changed when a step
landed: the figure sat frozen for half a minute and then jumped, which reads as a stuck widget
rather than a slow job. The step in flight is now counted apart from the ones after it, so the ETA
drains at one second per second. It stops at the end of the current step — if a job stalls, the
figure holds at the whole steps remaining instead of counting down to a finish that is not coming.

Fixed alongside it: the ETA was computed from the fresh rate estimate, which is null on the first
step of every batch item because the rate window has just been emptied. The held rate beside it kept
reading, so on a 28-image run the ETA blanked once per image while the rate did not.

Reactor panel: readable across a desk

• Everything went up about a quarter — header, annunciators, well captions, dial end words and
  figure, the build-info windows, recorder labels, bulb-bank captions, identity plate. The four
  middle-column boxes (STEPS LEFT / ETA / ELAPSED / QUEUE) were already sized and are unchanged.
• The dial figure is ink now, not lamp light. In the Control Room idiom it was drawn in the same
  amber the lamps use — pale orange on a cream face, and the hardest thing on the panel to read. It
  is mixed down to a dark tint that keeps the state colour (amber running, jade done, red fault) at
  printed-ink contrast. The Reactor Console is untouched: its face is glass, and the figure there
  really is light.

Also

• Relay verdict window 10s → 90s. A host whose relay was verifiably working was being reported
  absent, because an H3 sampler only speaks every 20-30 seconds and a 10-second window routinely
  held nothing.
• fmtSec printed "3m60s" at 239.6 seconds.

Downloads

Windows only this time. The Linux AppImage and .deb are built in WSL, and WSL2 will not start on
the build machine tonight (virtualisation is off at the firmware level, HypervisorPresent=False).
They will be attached to this release once that box is back — nothing in this build is
platform-specific, so the 0.0.7 Linux artifacts remain usable in the meantime.

v0.0.7 — video jobs get their own dial

Video jobs get a dial that fits them

The rate dial used one absolute scale for everything a ComfyUI host could be doing. That meant a
MiniMax-H3 video run at 15.75 s/it sat four degrees off the slow stop with the word SLOW printed
beside it — which is wrong. 15 s/it is a perfectly healthy video sampler; it is a catastrophic SDXL
one. No fixed scale can tell those apart, so the app now works out which kind of job is running and
prints the matching face.

What's new

• Video detection, from the running graph. Three signals, structural first: a node class that
  only exists in a video pipeline (WanImageToVideo, MiniMaxH3ReferenceToVideo, VHS_VideoCombine,
  AnimateDiff, SVD…), then any node asking for more than one frame under any of the five names the
  packs use for it, then a known video model family as a last resort. If there is no graph to judge,
  the app says nothing rather than guessing.
• A video face: 60 … 1 s/it, one unit, no SLOW/FAST wording at all, selectable at 30 / 60 /
  120 s/it. A 15 s/it run reads at 43% of the arc instead of pinned to the stop.
• A wider image face. Sampling ranges now carry their own fast end instead of being symmetric;
  the new default is 60 s/it — 5 it/s. The old 5 / 20 / 100 faces are still there.
• Three dial rows in the hosts panel: Images, Video, Training — each printing its own two ends.
• Cards switch face by reprinting it, never rebuilding, so a running job's needle swings to the
  new scale instead of dropping to its stop mid-run.

Fixes

• Size and frame count now show for Wan/HunyuanVideo image-to-video workflows, which build the latent
  inside the conditioning node and so had no latent node for the old lookup to find.
• The rate recorder labels its fast end 1 s/it instead of 1:1 on single-unit faces, where there
  is no crossover to name.
• The job's media kind was being computed and then dropped on the way to the UI — every card kept the
  image face regardless. Caught by running the packaged build, not by a test.

Notes

• Step counts for jobs queued from the ComfyUI web UI still need the relay node in
  resources/comfyui-relay — see the (i) panel on first launch.
• The video face is verified against real H3 graphs and end to end on the packaged build against a
  stub server; it has not yet been watched through a full live video run.

v0.0.6 — the reactor panel

Reactor panel — a second way to draw the rack

Pick it in the top-bar Look menu. The three existing looks (1U Rack, Obsidian Glass, Obsidian
Halo) are unchanged; two new entries draw each host as a control-panel module instead of a rack
card — Control Room (painted steel, ivory instrument faces, incandescent lamps, split-flap
counter) and Reactor Console (the same panel in glass and xenon, lit arcs, nixie tubes).

Everything on it is driven by something the watcher actually measures. Nothing is decoration:

• Annunciator bank — eight labelled lamps for eight real conditions: Reactor Run, Queued, Batch
  Run, No Step Data, Relay Absent, Cycle Done, Fault, Offline. A tile flashes when it first comes
  true, then holds. Unlit means not true.
• Twin rate dials — the rate meter split in two and mirrored, each movement owning one half of
  the same log scale, both reading up for faster. The exact figure sits between them. Only the half
  holding the reading lights.
• Cycle progress movement and a step counter (split-flap or nixie, by idiom). No step data
  still prints N/A — never a fabricated 0.
• LED bulb banks replace the progress bar: one bank across the current node's steps, and — when
  the relay reports a batch total — one bulb per image of the run.
• Rate recorder, 60 seconds of measured rate against the dials' own scale, labelled at both ends
  so it reads as a rate over time rather than a level.
• Host build info — ComfyUI, driver, PyTorch, and the GPU stack, which names itself: CUDA on
  an NVIDIA box, ROCm on an AMD one (detected from the PyTorch build tag, since ROCm otherwise looks
  identical to CUDA through ComfyUI's API), XPU or CPU where that is the build.

Also in this release

• Drag a module to reorder the rack, or use the ↑/↓ arrows in the hosts panel. The order is
  saved to hosts.json.
• Hide a host without deleting it — a hidden host stops being polled entirely, which is the
  point when the machine is switched off.
• Module size control (Compact / Normal / Large) now actually scales the modules, not just the
  column count.
• Selectable dial ranges (hosts panel → Dials), and a dedicated training dial face graduated
  60…1 s/it, because nothing a LoRA trainer does runs above 1 it/s.
• Batch runs read properly: "Image 3 / 28", a whole-batch ETA, and — the real fix — the rate
  estimator no longer goes blind for the first two thirds of every image in a dataset run.
• Obsidian Halo skin: obsidian glass and a floating xenon arc instead of a printed dial.
• The relay node gained a /watcher/host_info route (driver version, NVIDIA or AMD) and reports
  batch totals. If you use the relay, re-copy comfyui-relay/ into custom_nodes/ and restart
  ComfyUI — the (i) button in the app prints the path and opens the folder.
• Fixed: elapsed times printed like 3m60s.

Install

• Windows — comfyuiWATCHER Setup 0.0.6.exe
• Linux — comfyuiWATCHER-0.0.6.AppImage (chmod +x and run) or comfyuiwatcher_0.0.6_amd64.deb

ComfyUI hosts need the relay node for step counts; ai-toolkit hosts need nothing.

v0.0.5 — the relay ships with the installer

If you installed 0.0.4, install this one. That build shipped without the ComfyUI relay node,
so there was no way to enable step progress and nothing telling you it existed.

The relay now comes with the app

comfyui-relay/ installs alongside the app (in resources/comfyui-relay/) instead of being
sealed inside the application archive, so you can actually copy it where it needs to go.

On first launch the app now explains the one manual step, shows you the exact folder on your disk,
and gives you a button to open it. Afterwards it lives behind the (i) button in the top bar.

The step itself: copy that folder into <your ComfyUI>/custom_nodes/comfyui-watcher-relay/ and
restart ComfyUI. Its log prints comfyuiWATCHER progress relay installed. Repeat per ComfyUI
host. Training (ai-toolkit) hosts need none of this.

Why it is needed at all

ComfyUI sends step counts only to the client that submitted a job, so a passive watcher never sees
progress for anything queued from the ComfyUI web UI. The relay rebroadcasts those messages under
a watcher.-prefixed type that ComfyUI's own frontends ignore.

Without it a card still shows the host, queue, model, and whether a job is running — but no
step X/Y, rate or ETA. It shows an indeterminate sweep rather than inventing numbers.

It tells you whether it worked

The setup panel reports each ComfyUI host as relay active, no relay — steps unavailable, or
unknown until a job runs. Any watcher.* message is proof the relay is loaded, since nothing
else emits them. "No relay" is only claimed after a job has been running for ten seconds in
silence — an idle host stays unknown rather than being accused of a missing relay.

Downloads

• Windows — comfyuiWATCHER Setup 0.0.5.exe
• Linux — comfyuiWATCHER-0.0.5.AppImage or comfyuiwatcher_0.0.5_amd64.deb

v0.0.4 — LoRA training support (ai-toolkit)

comfyuiWATCHER now watches LoRA training, not just image generation. Add an
ai-toolkit host and its runs appear in the same rack as
your ComfyUI instances — step X/Y, s/it, ETA, loss, and what the run is doing right now.

Training hosts

Add a host, pick Training instead of Generation, and point it at ai-toolkit's UI server
(default http://host:8675). The top bar gains a Show filter — All / Generation / Training —
so a big rack can be narrowed to one kind. Existing hosts.json files keep working untouched; a
host with no kind is treated as ComfyUI.

A training module reads:

• STEPS LEFT and a progress bar, from the run's own config rather than the database column
• TRAINING RATE on a dial scaled for trainers: 60 … 1 s/it
• BASE MODEL / RESOLUTION / RANK, read from the job config — metadata only, never dataset
  contents or captions
• LOSS, and ELAPSED / ETA / RATE
• a phase readout in the top right — Model Loaded, Loading dataset, Training. A run is
  silent for minutes while it loads a model and caches a dataset, and without this the card is
  indistinguishable from a stall.

A dial that fits training speeds

Training gets its own face, 60 → 1 s/it. On the generation face (0.01–100 it/s) every real
training run piles up within a few degrees of centre, because LoRA training is far slower than
sampling — a 2 s/it run and a 30 s/it run were nearly indistinguishable. Both faces are graduated
in it/s underneath, so needle-right means faster on every module in the rack, whichever kind it
is.

The dial is also much larger — 176×81 to 371×171 pixels at the default window size.

Fixes

• ETA and rate were badly wrong on trainers, in three separate ways, all found by watching a
  real run: a model-load stall poisoned the average (45 s/it on a job 25 minutes from done), a 1s
  poll quantised the estimate, and a cached API response raced the live one to produce 0.005 s/it
  with a 3-second ETA. The estimator now takes a median of per-step intervals, discards pauses,
  and samples only the uncached read.
• A lone card no longer stretches across the window. With one host visible, the grid collapsed
  its empty tracks and smeared a single card to 863px.
• HTTP requests now time out instead of hanging a card on stale data forever.

Honesty rules, unchanged

Unknown is still shown as unknown. A run already in progress when the widget starts shows -- for
elapsed, because ai-toolkit records no training start time and time-since-noticed is not elapsed.
A stopped run clears the card rather than turning it red — stopping is a decision, not a fault.

Downloads

• Windows — comfyuiWATCHER Setup 0.0.4.exe
• Linux — comfyuiWATCHER-0.0.4.AppImage or comfyuiwatcher_0.0.4_amd64.deb

comfyuiWATCHER v0.0.3

Card rebuilt as a rack module, and the look is now switchable.

Instruments

• The arc gauge is replaced by a moving-coil rate meter with real needle ballistics. Log scale, because this watches both a 5 it/s image job and a 0.07 it/s video sampler; the face is split at the 1:1 mark — seconds-per-step on the left, iterations-per-second on the right, whole numbers on both — and the unit you are actually reading lights up.
• The dial is backlit and only lights when there is a rate, so a rack of idle hosts stays dark and the one that is working is the bright thing on screen.
• Steps remaining stays a seven-segment readout, still N/A rather than a fabricated 0 when a host has no relay node.

New on the card

• Job identity: model, latent size, and frames (video) or batch size (images), read from the graph the queue poll already fetches. Metadata only — prompt text is never read. Unknown values hide their row instead of guessing.
• Idle and offline hosts collapse to slim blanking panels.

Skins — pick from the top bar, remembered between launches:

• 1U Rack (default) — bolted faceplate, printed ivory dial, amber lamps.
• Obsidian Glass — the 0.0.2 look, kept.

Fixed

• Cards squashed to 83px each once their content exceeded the window height, clipping everything below the node row. Present in 0.0.2; anyone running more than about four hosts was hitting it.

Assets: Windows installer (NSIS), Linux AppImage, Debian package.

comfyuiWATCHER v0.0.2

First public build. Live multi-host ComfyUI job watcher: per-host cards with status, current node, step progress, speed, elapsed, ETA, queue depth and GPU load. Includes optional comfyui-relay custom node (copy comfyui-relay/ into ComfyUI custom_nodes/ and restart) for true step progress on jobs submitted by any client. Windows: NSIS installer. Linux: AppImage + deb.