Add FirmwareJob.source / source_label fields (7a-2a)#556
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #556 +/- ##
=======================================
Coverage 99.12% 99.12%
=======================================
Files 77 77
Lines 10056 10062 +6
=======================================
+ Hits 9968 9974 +6
Misses 88 88
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a FirmwareJob “origin” discriminator so the backend can represent locally-built vs remotely-delegated firmware artifacts as the transparent install flow is layered in, while remaining backwards-compatible with pre-existing persisted job sidecars.
Changes:
- Introduce
JobSource(LOCAL/REMOTE) and addFirmwareJob.source+FirmwareJob.source_labelwith LOCAL-shaped defaults for backward-compatible deserialization. - Extend
FirmwareJob.reset()’s “preserves identity” contract to explicitly include dispatch-origin fields (source,source_label,remote_peer,remote_job_id). - Add unit tests covering defaults, JSON round-trip persistence, upgrade-from-older-sidecar behavior, and reset preservation.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
esphome_device_builder/models/firmware.py |
Adds JobSource enum and new FirmwareJob fields for local vs remote build provenance; updates reset identity contract docs. |
tests/models/test_firmware_job.py |
Adds tests that pin defaults, persistence/deserialization compatibility, and reset preservation for dispatch-origin fields. |
0561e5b to
7f7b973
Compare
This was referenced May 10, 2026
bdraco
added a commit
that referenced
this pull request
May 10, 2026
The arch.md description of the transparent install flow's implementation slices used a "synthetic offloader-side FirmwareJob + event-stream bridge" framing inherited from the original design doc. That shape was reverse-engineered from "what's the minimum change to avoid touching the install dialog's subscriber?" — a bad starting point. It created two FirmwareJobs per remote build (one on each side), a bridge that re-fired OFFLOADER_JOB_* events as JOB_* against a synthetic id, and a separate RAM-only id mapping. Extra ceremony for a problem that doesn't exist: lockstep frontend deployment means the install dialog can learn the new field (`job.source_label`) in the same coordinated PR that lands the source-routed runner. Replaced the 7a-2 / 7a-3 / 7a-4 section with the first-class shape: - One `FirmwareJob` per build, the runner branches on `source` to pick its pipeline (local subprocess vs peer-link `submit_job` dispatch). - Same `JOB_*` lifecycle events fire either way; the install dialog stays on its existing subscriber. - `OFFLOADER_JOB_*` events stay as the wire-layer fan-out (5c-3 keeps them for the explicit Send-builds dialog that hangs around as the power-user surface); the runner consumes them privately to update its single FirmwareJob. Renumbered slices to reflect the new shape: - 7a-2a: `FirmwareJob.source` / `source_pin_sha256` / `source_label` field additions (in flight as #556). - 7a-2b: source-routed runner (renamed from "event-stream bridge"; the bridge concept disappears entirely). - 7a-3: `firmware/install` integration unchanged in spirit but now constructs a regular FirmwareJob with source=REMOTE rather than a synthetic wrapper. - 7a-4: cancel translation also simplifies — the runner's cancel hook fires remote_build/cancel_job directly, no id translation between two FirmwareJobs. Added explanatory paragraph above the slices about why the synthetic framing was wrong, so future readers don't have to reconstruct the reasoning from PR history. Issue #106's § Transparent install flow gets the same drop in a separate edit (the issue body is the source of truth for the full design; this PR brings arch.md into alignment). No code change.
bdraco
added a commit
that referenced
this pull request
May 10, 2026
The arch.md description of the transparent install flow's implementation slices used a "synthetic offloader-side FirmwareJob + event-stream bridge" framing inherited from the original design doc. That shape was reverse-engineered from "what's the minimum change to avoid touching the install dialog's subscriber?" — a bad starting point. It created two FirmwareJobs per remote build (one on each side), a bridge that re-fired OFFLOADER_JOB_* events as JOB_* against a synthetic id, and a separate RAM-only id mapping. Extra ceremony for a problem that doesn't exist: lockstep frontend deployment means the install dialog can learn the new field (`job.source_label`) in the same coordinated PR that lands the source-routed runner. Replaced the 7a-2 / 7a-3 / 7a-4 section with the first-class shape: - One `FirmwareJob` per build, the runner branches on `source` to pick its pipeline (local subprocess vs peer-link `submit_job` dispatch). - Same `JOB_*` lifecycle events fire either way; the install dialog stays on its existing subscriber. - `OFFLOADER_JOB_*` events stay as the wire-layer fan-out (5c-3 keeps them for the explicit Send-builds dialog that hangs around as the power-user surface); the runner consumes them privately to update its single FirmwareJob. Renumbered slices to reflect the new shape: - 7a-2a: `FirmwareJob.source` / `source_pin_sha256` / `source_label` field additions (in flight as #556). - 7a-2b: source-routed runner (renamed from "event-stream bridge"; the bridge concept disappears entirely). - 7a-3: `firmware/install` integration unchanged in spirit but now constructs a regular FirmwareJob with source=REMOTE rather than a synthetic wrapper. - 7a-4: cancel translation also simplifies — the runner's cancel hook fires remote_build/cancel_job directly, no id translation between two FirmwareJobs. Added explanatory paragraph above the slices about why the synthetic framing was wrong, so future readers don't have to reconstruct the reasoning from PR history. Issue #106's § Transparent install flow gets the same drop in a separate edit (the issue body is the source of truth for the full design; this PR brings arch.md into alignment). No code change.
First slice of the transparent install flow described in issue #106 § Transparent install flow. Makes the local-vs-remote-build distinction a first-class field on FirmwareJob: the firmware queue's runner branches on `source` to choose its pipeline (local subprocess vs peer-link `submit_job` dispatch), and the install dialog reads `source_label` to decide whether to render the "Building on {receiver_label}" sub-line. No synthetic-job wrapper, no event-stream bridge — one FirmwareJob per build, the same lifecycle events fire either way, `source` is the only place LOCAL and REMOTE diverge from a caller's perspective. New `JobSource` StrEnum in `models/firmware.py`: - `LOCAL` — this dashboard's CPU compiled the artifacts (today's default; matches every job-row written before this PR). - `REMOTE` — a paired receiver compiled the artifacts and this dashboard fetched them via `download_artifacts`. New fields on `FirmwareJob`: - `source: JobSource = JobSource.LOCAL` — the discriminator. - `source_pin_sha256: str = ""` — machine-readable handle on the receiver that compiled this job, when `source == REMOTE`. Matches `StoredPairing.pin_sha256`. Load-bearing for restart recovery: a crash-recovered REMOTE job needs to know which receiver to route `download_artifacts` / `cancel_job` against, and the RAM-only `_open_peer_links` cache doesn't survive the restart. Same shape as 5c-2's receiver-side `remote_peer` (machine id, not display label) and for the same reason. - `source_label: str = ""` — display string for the receiver-label when `source == REMOTE`; empty for LOCAL. Mirrors `StoredPairing.label` at job-creation time; doesn't track later renames of the pairing label (snapshot semantics). The machine-readable handle lookups go through is `source_pin_sha256`; `source_label` is purely for rendering. All three default to the LOCAL shape so an offloader upgrading across this landing sees its prior on-disk job-rows deserialise correctly (mashumaro fills defaults for keys missing from older sidecars). Distinct from `remote_peer` / `remote_job_id` — those are *receiver*-side fields filled by 5c-2 when the receiver picks up an offloader's `submit_job`. `source` / `source_pin_sha256` / `source_label` are the *offloader*-side fields for the same delegation seen from the dispatching dashboard. Both pairs end up set on opposite-side FirmwareJob rows for a single transparent dispatch (offloader's job: source=REMOTE + source_pin_sha256=<receiver_pin> + source_label="<label>"; receiver's job: remote_peer="<offloader dashboard_id>" + remote_job_id="<offloader job_id>"). `FirmwareJob.reset()`'s "preserves identity" list extended to include `source` / `source_pin_sha256` / `source_label` / `remote_peer` / `remote_job_id` — all five describe the job's dispatch origin, not per-run state, so they have to survive a crash-recovery rebuild. The previous docstring listed only the local fields; the omission was load-bearing for 5c-2 receivers too (a crashed receiver-side job losing `remote_peer` would strand the rebuild from the offloader's correlation cache). Test additions: - test_source_defaults_to_local pins the default. - test_remote_source_round_trips_through_serialisation pins the persistence contract: REMOTE FirmwareJobs keep their source / source_pin_sha256 / source_label across a dashboard restart, otherwise the firmware queue's runner would lose the receiver handle it needs to route download_artifacts / cancel_job on the rebuild. - test_older_sidecar_without_source_field_loads_as_local — hand-crafted minimal JSON without the new keys deserialises with default values rather than crashing on the persistence-load path. - test_malformed_source_value_rejected_on_load — corrupt `source: "intergalactic"` raises on construction; doesn't silently land in code that branches on `is JobSource.REMOTE`. - test_reset_preserves_dispatch_origin_fields pins all five dispatch-origin fields surviving FirmwareJob.reset(). ruff + mypy clean. No caller integration in this PR; the runner's source-routing branch lands in 7a-2b alongside firmware/install integration in 7a-3.
7f7b973 to
6751574
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this implement/fix?
First slice of the transparent install flow described in
issue #106 § Transparent install flow.
Makes the local-vs-remote-build distinction a first-class
field on
FirmwareJob: the firmware queue's runner brancheson
sourceto choose its pipeline (local subprocess vspeer-link
submit_jobdispatch), and the install dialog readssource_labelto decide whether to render the"Building on {receiver_label}" sub-line.
No synthetic-job wrapper. No event-stream bridge. One
FirmwareJobper build, the sameJOB_*lifecycle events fireeither way,
sourceis the only place LOCAL and REMOTE divergefrom a caller's perspective. The original design doc proposed a
synthetic-FirmwareJob + bridge construction to avoid changing
the install dialog's subscriber; that was extra ceremony for a
problem that doesn't exist (frontend ships lockstep with
backend, so adding a small consumer in the dialog when this PR
lands is one coordinated change). Arch.md catches up in a
separate docs PR.
What lands
New
JobSourceStrEnum inmodels/firmware.py:LOCAL— this dashboard's CPU compiled the artifacts(today's default; matches every job-row written before this
PR).
REMOTE— a paired receiver compiled the artifacts andthis dashboard fetched them via
download_artifacts.New fields on
FirmwareJob:source: JobSource = JobSource.LOCAL— the discriminator.source_pin_sha256: str = ""— machine-readable handleon the receiver that compiled this job, when
source == REMOTE. MatchesStoredPairing.pin_sha256.Load-bearing for restart recovery: a crash-recovered REMOTE
job needs to know which receiver to route
download_artifacts/cancel_jobagainst, and the RAM-only_open_peer_linkscache doesn't survive the restart. Sameshape as 5c-2's receiver-side
remote_peer(machine id, notdisplay label) and for the same reason.
source_label: str = ""— display string for thereceiver-label when
source == REMOTE. MirrorsStoredPairing.labelat job-creation time; doesn't tracklater renames of the pairing label (snapshot semantics). The
machine-readable handle lookups go through is
source_pin_sha256;source_labelis purely for rendering.All three default to the LOCAL shape so an offloader upgrading
across this landing sees its prior on-disk job-rows
deserialise correctly.
Distinct from
remote_peer/remote_job_id— those arereceiver-side fields filled by 5c-2. The new fields are the
offloader-side for the same delegation seen from the
dispatching dashboard. Both pairs end up set on opposite-side
FirmwareJob rows for a single transparent dispatch.
FirmwareJob.reset()'s "preserves identity" list extendedto include
source/source_pin_sha256/source_label/remote_peer/remote_job_id. Theremote_*omission wasalready a latent bug for 5c-2 receivers (a crashed
receiver-side job losing
remote_peerwould strand therebuild from the offloader's correlation cache). Pinned with
a new test.
Test additions
test_source_defaults_to_local— default value.test_remote_source_round_trips_through_serialisation—persistence contract: all three new fields keep their values
across a dashboard restart.
test_older_sidecar_without_source_field_loads_as_local—upgrade-from-pre-7a-2a path: hand-crafted minimal JSON
without the new keys deserialises with default values
rather than crashing.
test_malformed_source_value_rejected_on_load— corruptsource: "intergalactic"raises on construction; doesn'tsilently land in code that branches on
is JobSource.REMOTE.test_reset_preserves_dispatch_origin_fields— all fivedispatch-origin fields survive
FirmwareJob.reset().ruff + mypy clean. 9 tests in
tests/models/test_firmware_job.py.What this PR does NOT do
branch lands in 7a-2b (rename of the previous "event-stream
bridge" slice; the synthetic framing is gone in the new
plan) alongside
firmware/installintegration in 7a-3.Today nothing reads the new field; landing it standalone
keeps the model / persistence layer reviewable ahead of
the runner's larger blast radius.
"Building on {receiver_label}" sub-line lands when 7a-3
wires
firmware/installthrough the scheduler; thefrontend
FirmwareJobmirror gets the new fields then.Lockstep deployment means this is one coordinated PR
rather than two waves of compatibility work.
Related issue or feature:
§ Transparent install flow.The 7a-2b (runner source-routing), 7a-3 (
firmware/installintegration), and 7a-4 (cancel translation) slices layer on
top of this model change.
bridge framing for the first-class shape this PR introduces.
Types of changes
bugfixnew-featureenhancementbreaking-changerefactordocsmaintenancecidependenciesFrontend coordination
Checklist
ruff,codespell, yaml/json/python checks).tests/where applicable.components.jsonhas not been hand-edited.docs/ARCHITECTURE.mdand/ordocs/API.md— follow-up docs PR drops the synthetic-job framing this PR replaces.