Skip to content

feat(stovepipe): migrate process and buildsignal waits to the hold primitive - #488

Draft
behinddwalls wants to merge 1 commit into
preetam/consumer-hold-implfrom
preetam/hold-migrate-stovepipe
Draft

feat(stovepipe): migrate process and buildsignal waits to the hold primitive#488
behinddwalls wants to merge 1 commit into
preetam/consumer-hold-implfrom
preetam/hold-migrate-stovepipe

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The process stage's build-slot wait and buildsignal's status poll loop were built on ack-and-republish: each wait tick acked the delivery and PublishAfter'd a fresh copy of the same message to the controller's own topic. That forced both controllers to be publishers to their own topics, required per-stage message-id minting (a wall-clock suffix in process, a deterministic generation counter in buildsignal) to dodge the queue's (topic, partition_key, id) dedup, wrote a new log row per tick, and hung each loop's liveness on a publish succeeding. The hold primitive (doc/rfc/consumer-hold.md) is the framework-owned replacement.

What?

process: the budget-full branch now records a hold for GateWaitDelayMs and returns success; the delivery is threaded through processAccepted/admitLatestHead, rescheduleProcess and its wall-clock message-id minting are deleted, and the positive-delay config guard moves into holdForBuildSlot. buildsignal: a non-terminal status holds the delivery for the per-status poll delay; nextPollMessageID, the /poll/ generation scheme, and publishBuildSignal are deleted, and publish loses its now-unused delay parameter. The held message is a partition barrier that redelivers in order without counting toward the retry limit, so the DLQ budget stays reserved for real failures, and a failed postpone write lapses into a normal visibility-timeout redelivery instead of killing the loop.

Docs updated to the decided mechanism: process.md's "Waiting for a slot" section replaces the previously-deferred Option 1/Option 2 comparison with hold (the primitive that deferral named), buildsignal.md's polling-primitive and algorithm sections describe hold, GateWaitDelayMs is re-documented as a redelivery delay, and the e2e slow-build test's obsolete dedup rationale is rewritten.

Test Plan

bazel test //stovepipe/... — reworked unit tests assert Hold(delay) is recorded on the gate-wait and poll paths and that no Hold happens on terminal/error paths. ✅ bazel test //test/e2e/stovepipe/... — the slow-build scenario reaches a terminal build through multiple hold-driven poll ticks and releases the build slot. ✅ make fmt.

Stack

  1. docs(rfc): consumer hold — postpone a delivery as a partition barrier #486
  2. feat(platform): hold/postpone — a fourth delivery outcome for backing off #487
  3. @ feat(stovepipe): migrate process and buildsignal waits to the hold primitive #488
  4. feat(orchestrator): migrate the buildsignal poll loop to the hold primitive #489
  5. feat(messagequeue)!: remove PublishAfter and the visible_after column #490
  6. feat(messagequeue)!: drop the requeue delay parameter from Nack #491
  7. feat(consumergate)!: rebuild the gate on the postpone primitive #492

…imitive

## Summary

### Why?

The process stage's build-slot wait and buildsignal's status poll loop were built on ack-and-republish: each wait tick acked the delivery and PublishAfter'd a fresh copy of the same message to the controller's own topic. That forced both controllers to be publishers to their own topics, required per-stage message-id minting (a wall-clock suffix in process, a deterministic generation counter in buildsignal) to dodge the queue's (topic, partition_key, id) dedup, wrote a new log row per tick, and hung each loop's liveness on a publish succeeding. The hold primitive (doc/rfc/consumer-hold.md) is the framework-owned replacement.

### What?

process: the budget-full branch now records a hold for GateWaitDelayMs and returns success; the delivery is threaded through processAccepted/admitLatestHead, rescheduleProcess and its wall-clock message-id minting are deleted, and the positive-delay config guard moves into holdForBuildSlot. buildsignal: a non-terminal status holds the delivery for the per-status poll delay; nextPollMessageID, the /poll/ generation scheme, and publishBuildSignal are deleted, and publish loses its now-unused delay parameter. The held message is a partition barrier that redelivers in order without counting toward the retry limit, so the DLQ budget stays reserved for real failures, and a failed postpone write lapses into a normal visibility-timeout redelivery instead of killing the loop.

Docs updated to the decided mechanism: process.md's "Waiting for a slot" section replaces the previously-deferred Option 1/Option 2 comparison with hold (the primitive that deferral named), buildsignal.md's polling-primitive and algorithm sections describe hold, GateWaitDelayMs is re-documented as a redelivery delay, and the e2e slow-build test's obsolete dedup rationale is rewritten.

## Test Plan

✅ `bazel test //stovepipe/...` — reworked unit tests assert Hold(delay) is recorded on the gate-wait and poll paths and that no Hold happens on terminal/error paths. ✅ `bazel test //test/e2e/stovepipe/...` — the slow-build scenario reaches a terminal build through multiple hold-driven poll ticks and releases the build slot. ✅ `make fmt`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant