Skip to content

feat(orchestrator): dispatch builds per speculation path - #503

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-path-build-storefrom
preetam/speculation-build-dispatch
Open

feat(orchestrator): dispatch builds per speculation path#503
behinddwalls wants to merge 1 commit into
preetam/speculation-path-build-storefrom
preetam/speculation-build-dispatch

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The build stage triggered one build per batch, on the batch's full dependency list. Speculation needs one build per path, on the subset of dependencies the path assumes will succeed — that subset is what lets a batch be verified before the batches ahead of it resolve.

What?

The dispatch controller now reads the head's path set and starts a build for every pending path, on that path's own base. It only starts builds — stopping them belongs to the poll loop (next commit) — and it never writes the path set, which stays the speculate run's single-writer state; what this stage knows is the build it started, recorded in per-build rows of its own.

The write order is Trigger → Build record → write-once link (entity.PathBuild) → signal; each write makes the previous one reachable, and the link is the idempotency point for redeliveries. A lost link race publishes both builds' signals and acks — the poll loop keeps the one the link names. On redeliveries only, every live linked path gets its signal re-published, closing the crash window between link and signal without forking duplicate poll chains past the queue's dedup horizon.

The build signal carries only the runner's build ID and partitions on it, so one slow build cannot block a head's other paths. Known gap: a crash between Trigger and the link orphans that build; the fix is an idempotency key on BuildRunner.Trigger, marked TODO.

Also adds submitqueue/core/publish — the registry-lookup-and-send plumbing and the message-ID deduplication rule in one place.

Test Plan

bazel test //submitqueue/orchestrator/controller/build/... //submitqueue/core/publish/... — write order, the base holding only assumed-success dependencies, redelivery republish without rebuild, lost-race double signal, halted batch starting nothing, redelivery healing for live linked paths, and a pin that no path-set write of any kind happens.

make fmt, make gazelle, make mocks

Stack

  1. feat(speculation): generator contract and bestfirst impl #446
  2. feat(speculation): allocator contract and sticky impl #450
  3. feat(speculation): standard composed speculator #451
  4. feat(storage): speculation path set store #501
  5. feat(storage): path-build link store for per-path builds #502
  6. @ feat(orchestrator): dispatch builds per speculation path #503
  7. feat(orchestrator): poll builds and stop the ones nothing wants #504
  8. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  9. feat(orchestrator): re-plan the queue from the Speculator each run #506
  10. feat(orchestrator)!: finalize batches from their speculation paths #507
  11. feat(orchestrator): compose per-queue speculators and turn speculation on #508

## Summary

### Why?

The build stage triggered one build per batch, on the batch's full dependency list. Speculation needs one build per *path*, on the subset of dependencies the path assumes will succeed — that subset is what lets a batch be verified before the batches ahead of it resolve.

### What?

The dispatch controller now reads the head's path set and starts a build for every pending path, on that path's own base. It **only starts builds** — stopping them belongs to the poll loop (next commit) — and it **never writes the path set**, which stays the speculate run's single-writer state; what this stage knows is the build it started, recorded in per-build rows of its own.

The write order is Trigger → Build record → write-once link (`entity.PathBuild`) → signal; each write makes the previous one reachable, and the link is the idempotency point for redeliveries. A lost link race publishes both builds' signals and acks — the poll loop keeps the one the link names. On redeliveries only, every live linked path gets its signal re-published, closing the crash window between link and signal without forking duplicate poll chains past the queue's dedup horizon.

The build signal carries only the runner's build ID and partitions on it, so one slow build cannot block a head's other paths. Known gap: a crash between `Trigger` and the link orphans that build; the fix is an idempotency key on `BuildRunner.Trigger`, marked TODO.

Also adds `submitqueue/core/publish` — the registry-lookup-and-send plumbing and the message-ID deduplication rule in one place.

## Test Plan

✅ `bazel test //submitqueue/orchestrator/controller/build/... //submitqueue/core/publish/...` — write order, the base holding only assumed-success dependencies, redelivery republish without rebuild, lost-race double signal, halted batch starting nothing, redelivery healing for live linked paths, and a pin that no path-set write of any kind happens.

✅ `make fmt`, `make gazelle`, `make mocks`
@behinddwalls
behinddwalls force-pushed the preetam/speculation-build-dispatch branch from 7ff20d4 to 2e32d07 Compare August 4, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant