Skip to content

Add BuildScheduler primitive (7a-1)#553

Merged
bdraco merged 1 commit into
mainfrom
7a-build-scheduler
May 10, 2026
Merged

Add BuildScheduler primitive (7a-1)#553
bdraco merged 1 commit into
mainfrom
7a-build-scheduler

Conversation

@bdraco
Copy link
Copy Markdown
Member

@bdraco bdraco commented May 10, 2026

What does this implement/fix?

First slice of the transparent install flow described in
issue #106 § Transparent install flow.

Pure decision function pick_build_path(...) that takes the
offloader's RAM-canonical state (paired receivers, open
peer-link sessions, per-peer queue snapshots) plus the
user-facing master switch, and returns a typed
BuildPathDecision telling the caller whether to spawn a local
FirmwareJob or dispatch to a specific paired receiver.

The decision function is intentionally side-effect-free: no
controller references, no event-bus interaction, no I/O. The
caller (eventually the firmware/install WS handler in 7a-3)
gathers the state and threads it in. Two reasons:

  • Unit-testability. The candidate-filter rules (master
    switch / PENDING / connected / idle / missing-snapshot) all
    flow through one function with simple inputs; the test suite
    covers them without standing up the controllers or the event
    bus.
  • Lifetime. State lives on RemoteBuildController
    (_pairings, _open_peer_links, _peer_queue_status).
    Passing it in keeps the helper callable from any future site
    that's already holding the slices.

What lands

helpers/build_scheduler.py:

  • BuildPath StrEnum (LOCAL / REMOTE) — string-valued so
    the eventual FirmwareJob.source field carries the
    discriminator on the wire without a custom encoder.
  • BuildPathDecision(path, pin_sha256) frozen dataclass +
    local() / remote(pin) factory classmethods.
    Structurally-uniform shape so callers pass
    decision.pin_sha256 through without an
    if path == REMOTE guard at every site.
  • pick_build_path(remote_builds_enabled, pairings, open_peer_links, peer_queue_status) -> BuildPathDecision.
    Walks pairings in dict-insertion order; first entry that's
    APPROVED + in open_peer_links + has a queue snapshot
    reporting idle: True wins. No-candidate cases
    silent-fallback to LOCAL (matches the design doc's stance
    for the master switch off / no idle peers cases).

14 new tests in tests/test_build_scheduler.py pin every
candidate-filter branch:

  • Master switch off → LOCAL even with idle remote.
  • Empty pairings → LOCAL.
  • PENDING / disconnected / busy / missing-queue-snapshot all
    skip; falling through to LOCAL when no candidate qualifies.
  • Multi-candidate: first eligible wins; busy / disconnected /
    PENDING candidates fall through to the next.
  • BuildPathDecision shape: local() has empty pin,
    remote(pin) round-trips, frozen prevents accidental field
    mutation.

Deliberately deferred to follow-up PRs

  • Caller integration. The firmware/install WS handler
    route-through lands in 7a-3 alongside the event-stream
    bridge that re-fires OFFLOADER_JOB_* as local-shaped
    JOB_*. Today this decision is unused — landing it as a
    standalone helper keeps the unit-test layer reviewable ahead
    of the integration's larger blast radius.
  • Version-compat check. StoredPairing doesn't currently
    carry the receiver's esphome_version (available in mDNS TXT
    for discovered peers but doesn't flow through the pair-time
    handshake into the persisted row). The design doc's
    version_compatible gate is documented inline as the
    version-cache home; needs separate wiring before it can
    fire. Until then every connected + idle APPROVED pairing is
    a candidate regardless of receiver version.
  • Richer pick policy. First-eligible is the first-cut.
    Round-robin / least-loaded / cache-hot-affinity are 7a-3+
    concerns; the design doc explicitly puts them beyond this
    first slice.

Related issue or feature:

Types of changes

  • Bugfix (non-breaking change which fixes an issue) — bugfix
  • New feature (non-breaking change which adds functionality) — new-feature
  • Enhancement to an existing feature — enhancement
  • Breaking change (fix or feature that would cause existing functionality to not work as expected) — breaking-change
  • Refactor (no behaviour change) — refactor
  • Documentation only — docs
  • Maintenance / chore — maintenance
  • CI / workflow change — ci
  • Dependencies bump — dependencies

Frontend coordination

No frontend change needed for this PR — the helper has zero
callers today. The frontend's eventual touch (the "Building on
{receiver_label}" sub-line on the install dialog) lands when
7a-3's integration routes firmware/install through this
scheduler.

  • No frontend change needed
  • Companion frontend PR: esphome/device-builder-dashboard-frontend#

Checklist

  • The code change is tested and works locally.
  • Pre-commit hooks pass (ruff, codespell, yaml/json/python checks).
  • Tests have been added or updated under tests/ where applicable.
  • components.json has not been hand-edited.
  • Architecture-level changes are reflected in docs/ARCHITECTURE.md and/or docs/API.md — N/A; this PR adds a helper with no caller, so the visible architecture doesn't change yet. The transparent-install architecture lives in issue Remote build offload: discover and delegate compiles to another dashboard on the LAN #106's § Transparent install flow until 7a-3 makes it real.

Copilot AI review requested due to automatic review settings May 10, 2026 22:37
@github-actions github-actions Bot added the new-feature New feature label May 10, 2026
@bdraco bdraco force-pushed the 7a-build-scheduler branch from 3f1d1bb to 6fc309c Compare May 10, 2026 22:38
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 10, 2026

Merging this PR will not alter performance

✅ 12 untouched benchmarks


Comparing 7a-build-scheduler (78ada85) with main (55d8904)

Open in CodSpeed

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 10, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.12%. Comparing base (ceace1b) to head (78ada85).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #553   +/-   ##
=======================================
  Coverage   99.11%   99.12%           
=======================================
  Files          76       77    +1     
  Lines        9993    10033   +40     
=======================================
+ Hits         9905     9945   +40     
  Misses         88       88           
Flag Coverage Δ
py3.12 99.09% <100.00%> (+<0.01%) ⬆️
py3.14 99.12% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
esphome_device_builder/helpers/build_scheduler.py 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@bdraco bdraco force-pushed the 7a-build-scheduler branch from 6fc309c to 88d4b67 Compare May 10, 2026 22:41
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces a new pure “build path” decision primitive that selects between local compilation and dispatching a firmware build to an eligible paired remote receiver, as the first backend slice of the transparent install flow (issue #106).

Changes:

  • Added helpers/build_scheduler.py with BuildPath, BuildPathDecision, and pick_build_path(...) (side-effect-free decision logic).
  • Added tests/test_build_scheduler.py with unit coverage for master-switch gating, candidate eligibility filters, and multi-candidate selection behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
esphome_device_builder/helpers/build_scheduler.py Adds the side-effect-free scheduler primitive and typed decision object for local vs remote build routing.
tests/test_build_scheduler.py Adds unit tests covering all current eligibility gates and selection policy for the scheduler.

Comment thread esphome_device_builder/helpers/build_scheduler.py Outdated
@bdraco bdraco force-pushed the 7a-build-scheduler branch 2 times, most recently from 35f404c to d107c73 Compare May 10, 2026 22:49
First slice of the transparent install flow described in issue
#106 § Transparent install flow. Pure decision function that
takes the offloader's RAM-canonical state (paired receivers,
open peer-link sessions, per-peer queue snapshots) plus the
user-facing master switch, and returns a typed
BuildPathDecision telling the caller whether to spawn a local
FirmwareJob or dispatch to a specific paired receiver.

New `helpers/build_scheduler.py`:

- `BuildPath` StrEnum (`LOCAL` / `REMOTE`) — string-valued so
  the eventual FirmwareJob.source field can carry the
  discriminator on the wire without a custom encoder.
- `BuildPathDecision(path, pin_sha256)` frozen dataclass +
  `local()` / `remote(pin)` factory classmethods.
  Structurally-uniform shape so caller sites pass
  ``decision.pin_sha256`` through without an
  ``if path == REMOTE`` guard at every site.
- `pick_build_path(remote_builds_enabled, pairings,
  open_peer_links, peer_queue_status) -> BuildPathDecision`.
  Walks pairings in dict-insertion order; first entry that's
  APPROVED + in open_peer_links + has a queue snapshot
  reporting idle wins. No-candidate cases (every gate)
  silent-fallback to LOCAL.

Deliberately deferred to follow-up PRs:

- Caller integration. The firmware/install WS handler
  route-through lands in 7a-3 alongside the event-stream
  bridge that re-fires OFFLOADER_JOB_* as local-shaped JOB_*.
  Today this decision is unused — landing it as a standalone
  helper keeps the unit-test layer reviewable ahead of the
  integration's larger blast radius.
- Version-compat check. StoredPairing doesn't currently carry
  the receiver's esphome_version (it's available in mDNS TXT
  for discovered peers but doesn't flow through the pair-time
  handshake into the persisted row). The design doc's
  version_compatible gate is documented inline as the
  version-cache home; needs separate wiring before it can
  fire. Until then every connected + idle APPROVED pairing is
  a candidate regardless of receiver version.
- Richer pick policy. First-eligible is the first-cut.
  Round-robin / least-loaded / cache-hot-affinity are 7a-3+
  concerns; the design doc explicitly puts them beyond this
  first slice.

14 new tests in tests/test_build_scheduler.py pin every
candidate-filter branch:

- Master switch off → LOCAL even with idle remote.
- Empty pairings → LOCAL.
- PENDING / disconnected / busy / missing-queue-snapshot all
  skip; falling through to LOCAL when no candidate qualifies.
- Multi-candidate: first eligible wins; busy / disconnected /
  PENDING candidates fall through to the next.
- BuildPathDecision shape: local() has empty pin, remote(pin)
  round-trips, frozen prevents accidental field mutation.

ruff + mypy clean. No behaviour change to any existing code
path; the new module has zero callers today.
@bdraco bdraco force-pushed the 7a-build-scheduler branch from d107c73 to 78ada85 Compare May 10, 2026 22:54
@bdraco bdraco merged commit 4650924 into main May 10, 2026
13 checks passed
@bdraco bdraco deleted the 7a-build-scheduler branch May 10, 2026 22:59
bdraco added a commit that referenced this pull request May 10, 2026
Two arch.md gaps accumulated across the last ~5 merges; close
them before 7a-2 lands so the next architectural slice has a
documented frame to refer back to.

**Endpoint rebind paragraph (post Auto-reconnect).** Covers the
two entry points that share `_probe_pairing_endpoint` /
`_commit_endpoint_rebind`:

- 4a-o part 7 automatic mDNS rebind (#539, merged).
- 8b user-driven `remote_build/edit_pairing_endpoint` WS command
  (#548, merged).

Both fire the same probe-before-mutate primitive — pin must
match the new endpoint or the edit refuses (substituting a fresh
pubkey under existing trust is what the 8a re-auth wizard
exists to gate).

**Transparent install flow sub-section (in the Remote build
section, before Persisted state).** Captures the load-bearing
"receiver only ever compiles; the offloader always installs"
policy that issue #106 § Transparent install flow elevated to
core architecture, plus the implementation slices:

- 7a-1 (`BuildScheduler` decision primitive, #553 merged) —
  pure `pick_build_path(BuildSchedulerInputs) ->
  BuildPathDecision`, frozen-view input wrapper with
  `Mapping` / `frozenset` typing, fail-closed-by-construction
  `is PeerStatus.APPROVED` gate, `pin_sha256: str | None`
  decision shape.
- 7a-2 (event-stream bridge, open) — new
  `FirmwareJob.source` discriminator + `OFFLOADER_JOB_*` →
  `JOB_*` re-fire so the install dialog renders remote builds
  without learning about remote event types.
- 7a-3 (`firmware/install` integration, open) — WS handler
  routes through scheduler, dispatches
  `submit_job{target: "compile"}`, kicks off the 7a-2 bridge.
- 7a-4 (cancel translation, open) — Stop button →
  `cancel_job` against the receiver when synthetic job's
  `source` is remote.

Plus a note on the deferred per-pairing `esphome_version`
wiring that the design-doc version-compat gate needs.

Points to issue #106 § Transparent install flow as the source
of truth for the full design.

No code change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

new-feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants