Skip to content

feat(stovepipe): derive build strategy in process step - #358

Merged
mnoah1 merged 5 commits into
mainfrom
mnoah1/stovepipe-process-build-strategy
Jul 14, 2026
Merged

feat(stovepipe): derive build strategy in process step#358
mnoah1 merged 5 commits into
mainfrom
mnoah1/stovepipe-process-build-strategy

Conversation

@mnoah1

@mnoah1 mnoah1 commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

What?

  • Add source-control-backed build-strategy selection to Stovepipe’s process stage.
  • Pin admitted requests to an incremental last-green baseline when ancestry is valid; otherwise use a full build.
  • Fall back safely to full builds when the baseline is unavailable, and return unexpected source-control errors for normal consumer handling.
  • Add coverage for cold starts, rewrites, unavailable ancestry, failures, and queue CAS reloads.

Why?

Implementing proposed strategy selection from https://github.com/uber/submitqueue/blob/main/doc/rfc/stovepipe/steps/process.md

Test Plan

Incremental:

  • Seed an existing queue value in the DB matching the fake source control head
  • call uber.submitqueue.stovepipe.Stovepipe/Ingest
  • Check request in DB:
state:          processing
build_strategy: incremental_since_green
base_uri:       git://manual/incremental/HEAD

New Request:

  • Empty/fresh DB
  • call uber.submitqueue.stovepipe.Stovepipe/Ingest
  • Check request in DB
state:          processing
build_strategy: full
base_uri:      <blank>

Not an Ancestor

  • Seed an existing queue value in the DB that does not match the fake source control (so ancestor check returns false)
  • Empty/fresh DB
  • call uber.submitqueue.stovepipe.Stovepipe/Ingest
  • Check request in DB
state:          processing
build_strategy: full
base_uri:      <blank>
  • Planning some more integration tests around this flow

Pin each admitted request to the queue's last-green baseline, falling back safely to full builds when ancestry is unavailable.
@mnoah1 mnoah1 changed the title [draft/wip] feat(stovepipe): derive build strategy in process [draft/wip] feat(stovepipe): derive build strategy in process step Jul 14, 2026
@mnoah1 mnoah1 changed the title [draft/wip] feat(stovepipe): derive build strategy in process step feat(stovepipe): derive build strategy in process step Jul 14, 2026
@mnoah1
mnoah1 marked this pull request as ready for review July 14, 2026 17:33
@mnoah1
mnoah1 requested review from a team, behinddwalls and sbalabanov as code owners July 14, 2026 17:33
Reuse source-control resolution across queue CAS retries, record strategy outcomes, and keep the chosen strategy when a request CAS reloads an accepted row.
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-process-build-strategy branch from 14bfb30 to 17bbcb8 Compare July 14, 2026 18:27
@mnoah1
mnoah1 force-pushed the mnoah1/stovepipe-process-build-strategy branch from 4a8c221 to 9457616 Compare July 14, 2026 19:54

@behinddwalls behinddwalls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few non-blocking nits from review — the core derive→claim→persist ordering looks right (strategy is derived against the same queue-row version the claim CAS validates, so the persisted baseline can't be stale), and threading strategy/baseURI through markProcessing also fixes the latent reload-clobber on main. Nits inline.

Comment thread stovepipe/controller/process/process.go Outdated
Comment thread stovepipe/controller/process/process.go Outdated
Comment thread stovepipe/controller/process/process_test.go Outdated
Comment thread stovepipe/controller/process/process_test.go
mnoah1 and others added 2 commits July 14, 2026 22:16
Distinguish source-control resolution from ancestry failures and pin retry behavior when queue or request rows reload.
@mnoah1
mnoah1 enabled auto-merge July 14, 2026 22:22
@mnoah1
mnoah1 added this pull request to the merge queue Jul 14, 2026
Merged via the queue into main with commit 95126e6 Jul 14, 2026
15 checks passed
@mnoah1
mnoah1 deleted the mnoah1/stovepipe-process-build-strategy branch July 14, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants