Trace-Topology nightly has failed 60/60 runs since 2026-06-01 — tools/fixture-vm/flake.lock was never committed

[avrabe]

Summary

Trace-Topology Fixture Generation has failed 60 out of its last 60 runs — every run
back to its oldest recorded one, 2026-06-01. It has never produced a fixture.

$ gh run list --workflow "Trace-Topology Fixture Generation" --limit 60 \
    --json conclusion --jq '[.[]|.conclusion]|group_by(.)|map({(.[0]):length})'
[{"failure":60}]

Fuzz (nightly) is in the identical state, filed as #361 with a different root cause.

Cause: a one-time bootstrap that was never done

The workflow's first real step is a deliberate fail-closed guard
(.github/workflows/trace-fixtures.yml:58):

- name: Require committed flake.lock
  run: |
    if [ ! -f tools/fixture-vm/flake.lock ]; then
      echo "::error::tools/fixture-vm/flake.lock is missing."
      ...
      exit 1

And on disk:

tools/fixture-vm/
└── flake.nix          # committed 2026-06-11
                       # flake.lock — ABSENT, and not gitignored, just never added

The guard is correct and is working exactly as designed. The workflow header explains
why it must be this way:

Determinism: this job builds ONLY against the committed tools/fixture-vm/flake.lock —
it never runs nix flake update. The lock IS the pin (the whole reason for the NixOS
approach); re-resolving inputs per run would silently un-pin nixpkgs / the kernel.

So this is not a broken guard to relax — refusing to build against an unpinned toolchain is
the entire point, and doing it fail-closed is right. The gap is that the companion
bootstrap step was never carried out.

Fix (the procedure the error message already prescribes)

1. Dispatch .github/workflows/fixture-vm-flake-lock.yml (the fixture-vm flake.lock
   workflow) — it exists for precisely this.
2. Download its artifact.
3. Commit the resulting tools/fixture-vm/flake.lock in a PR, so the pin is reviewed
   rather than resolved silently.

Step 3 needs a human look at what nixpkgs/kernel revision is being pinned — that is a
supply-chain decision, not a mechanical one, so I have not auto-generated it.

The part that outlives this bug

Same meta-point as #361: neither of these nightlies is a required context, so 60
consecutive reds produced no signal anywhere a human looks. A workflow that has never once
succeeded is indistinguishable, on the PR page, from one that is quietly protecting you.
Either route these failures somewhere visible, or stop counting them as assurance.

There is also a narrower lesson: a fail-closed guard whose remediation is "a human must do
a one-time manual step" will sit red forever unless something tracks that step. The guard
did its job; nothing owned the follow-through.

Verified against spar at 457592f; file presence, gitignore status, and run history all
re-checked directly.

[avrabe]

Do not expect #364 to close this. It will clear the blocker named in the title
— the missing flake.lock — but the nightly will still be red afterwards, for a
different and previously-invisible reason.

Dispatching trace-fixtures on #364's branch got past trace-fixtures.yml:58 for
the first time in 60 runs, then died 15 seconds later in nix evaluation:
flake.nix never defines system.build.qcow2, the attribute the workflow
builds and the flake's own header documents. Filed as #365.

Two stacked blockers, the outer one fail-closed — which is why all 60 recorded
failures died at the guard and nobody ever saw the second one:

#	blocker	status
1	flake.lock never committed	#364 (open)
2	system.build.qcow2 undefined	#365 (open)

I've deliberately changed #364 to say Addresses rather than Closes so this
stays open until a trace-fixtures run actually reaches the end. And it may not
stop at two — the guest build, the /dev/kvm boot and fixture generation have
never executed either.

Worth stating plainly: this workflow has never once succeeded. It is not a
regression, it is infrastructure that was written, documented, wired into a
nightly schedule, and never run. It isn't a required context, so nothing ever
said so.

🤖 Generated with Claude Code