progress: show hashing workers as idle when the queue starves

[martinus]

The bug

On a big NAS tree replayed from a huge hashfile, the worker lines froze on commit for the whole run:

  1  commit     /data/databases/immich-postgres/pg_stat_tmp/db_16385.stat  (size: 79.3 KiB)
  ...
⠏ scanning   ⠏ hashing   · dedupe   · done
hashing   8,685 files · 4 need hashing · 2.6 KiB/s

Those 4 files were hashed in milliseconds; the walk then ran for minutes with nothing left to hash.

A csum worker is persistent and keeps its progress line across files (deliberately — re-claiming per file made the display flash "idle" between small files), so between files the line keeps whatever the last file left behind: thread_committing, set for the final DB write. Invisible when hashing is the bottleneck, plainly wrong — and it reads like a hang in the commit path — when the walk is.

The fix

Split it where the knowledge actually is. Only the queue can tell "blocked with no work" from "blocked doing work", so the worker publishes when it started waiting — pscan_slot_waiting(slot, true/false), one atomic store bracketing the pop. Only the renderer should own display timing, so the renderer draws any line whose wait has outlasted a couple of redraws as idle (slot_is_idle()), against a now-named REDRAW_MS instead of two bare literals.

Nothing is polled and no timer is involved: the cost is one atomic store per file (plus one vDSO clock read), and the line flips within one redraw of the threshold. The slot's status is untouched while it waits, so it still looks claimed and no sibling worker takes over its line.

Also: pscan_register_thread() published a fresh slot into pscan.threads while it was still thread_idle — i.e. stealable by a concurrent claim — so it now sets the status before publishing, and is static (nothing outside progress.c used it).

The second commit is a /simplify pass over the first, which had done this from inside scan_workq_pop() with a bounded wait, a park call into the progress module and an owned flag. That worked but put display policy in a queue primitive; the rewrite deletes the timed wait, the park constant, owned, a near-duplicate pscan_slot_wait() and the extra scan_workq_pop() parameter (its signature and unit test are back to what they were). Happy to squash the two.

Verification

Reproduced with a 300k-file tree, cold cache, 3 files touched (--io-threads=1, captured through a pty):

• before — 1 commit .../hashme2 (size: 97.7 KiB) in every one of the 23 frames;
• after — commit for 2 frames (~200 ms), then 1 idle for the remaining 2 s.

No-flicker check on a hashing-bound run (94k-file cold scan, 46 frames): an idle worker line appears in exactly one frame, the drain at the end.

test_starved_worker_line_reads_idle pins both directions (a fresh wait still shows the file's status; a backdated one reads idle) plus the no-steal rule — with no sleep, so the C unit suite is back to sub-millisecond. scripts/verify.sh passes (clean build, 120 integration tests, valgrind smoke), as does the full suite under ThreadSanitizer.

🤖 Generated with Claude Code