perf: verify and cleanup hash over every core, and the race gate learns to notice

[donislawdev]

Hashing the files a manifest claims is what verify and cleanup are made of, and it is embarrassingly parallel. It ran on one goroutine.

Measured

tfg verify on 6.1 GB in 96 files of 64 MB, binary before and after, order alternated:

before  4302 / 4303 / 4280 ms      after  556 / 539 / 536 ms

7.9x, with a spread under one percent either side. The hashing on its own scales 1.92x / 3.56x / 6.63x / 9.33x at 2 / 4 / 8 / 16 workers, so the plateau sits at the hardware thread count and the width asks GOMAXPROCS rather than carrying a constant. On a corpus larger than memory, where the disk is the limit, it is 1.58x.

O116 turned this down on 2026-08-20 and the number it turned down was real: on 3000 files of 1 kB the whole of verify is about a second, so half a second was all there was to win. That reading describes small files. The owner reopened the decision with the measurement above in hand.

Nothing about the answers changes

Both binaries produce byte-identical output on a directory holding a missing file, a wrong size, a wrong hash and a stranger - JSON and plain, stdout and stderr, same exit code 7.

What makes the answers deterministic is that the workers cannot fail. Every path is resolved first, on one goroutine, in order, so the first entry that leaves the directory is the first one there is. If that refusal could arrive from a worker, stopping the others would mean a lower index never got asked and the same manifest would name a different file on different days.

Two guards, both proven by mutation

• the answers come back in the order the manifest lists them. cleanup prints that list to a person and then deletes from it, and a list assembled out of completion order holds the same files - so every other cleanup guard stays green while somebody reads one order and the tool acts in another.
• the race detector is run for every file that declares concurrency. The workflow claimed the two lists could not drift because the map guard reddens first. It does - but only while the file is missing from the map, and adding it there is what that guard tells you to do. Walking into it is how it was found.

Checks

Full suite green, preflight --quick 12 of 12, -race clean on 501 files with sixteen workers, five mutations all caught, three stale mutation patterns repointed after the refactor.

The depth ceiling asked for the worker loop to be flattened rather than raised, so the body of the goroutine is its own function. Two crowding ceilings came down.

⚠️ Touches .github/workflows/**, so gh pr merge will refuse - this one needs the browser.

🤖 Generated with Claude Code