-
Notifications
You must be signed in to change notification settings - Fork 26
fwmanager: table-declared boot checkpoints replace BootMonitor #397
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
chrysh
wants to merge
13
commits into
OpenPRoT:main
Choose a base branch
from
9elements:add-boot-walk
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
5810d68
orchestrator: Replace BootMonitor with checkpoint-embedded evidence c…
chrysh e4b198a
orchestrator: Defunctionalize evidence checks into board-defined sign…
chrysh 0cf81e5
orchestrator: Let devices report failure and its retriability as evid…
chrysh a379115
orchestrator: Exercise message-path evidence in the reader tests
chrysh 68eb7a8
orchestrator: Carry the re-armed deadline in WalkVerdict::Retry
chrysh 4282840
orchestrator: Document wiring a concrete reader into EvidenceReader
chrysh 19f0021
orchestrator: Reject duplicate checkpoint names; pin max_retries=0 me…
chrysh a088e2e
orchestrator: Anchor the signal-id docs; show board-local validation
chrysh 7356e9b
orchestrator: Drop CommitPolicy from the device table
chrysh 4e1f031
orchestrator: Leave retry and terminal decisions to the orchestrator
chrysh b632715
orchestrator: Pin the orchestrator seams in the docs
chrysh 7d6bea8
orchestrator: Make invalid table entries unconstructible
chrysh 5d9974c
orchestrator: Fold BootStatus into the evidence module
chrysh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,148 @@ | ||
| // Licensed under the Apache-2.0 license | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| //! The orchestrator-facing seam of boot supervision. | ||
|
|
||
| /// One device's boot walk, pollable without knowing the device type. | ||
| /// | ||
| /// Everything device-specific — the driver type, its error type, the | ||
| /// checkpoint list — stays inside the concrete walk; the orchestrator's | ||
| /// fleet view is uniform. Object-safe so a heterogeneous fleet can sit | ||
| /// behind `&mut dyn BootWatch`; a board preferring static dispatch wraps | ||
| /// its walks in an enum and matches, without touching anything below the | ||
| /// seam. | ||
| pub trait BootWatch { | ||
| /// Judges the walk at `now_millis` (monotonic). Never sleeps — time is | ||
| /// injected, so every decision is host-testable. | ||
| fn poll(&mut self, now_millis: u64) -> WalkVerdict; | ||
| } | ||
|
|
||
| /// Everything the orchestrator needs to know about a boot walk. | ||
| /// | ||
| /// Observation only: the walk judges checkpoint windows, never lives. | ||
| /// Retry counts and terminal calls belong to the orchestrator state | ||
| /// machine (`ComponentStatus.retry`, the `Recovering` → `RecoveryFailed` | ||
| /// path) — a verdict that carried a retry budget would be a second owner | ||
| /// for the same decision, free to disagree with the first. | ||
| /// | ||
| /// Deliberately free of device and error types: the concrete detail is | ||
| /// logged by the walk while it is still in scope, not carried across the | ||
| /// seam. | ||
| /// | ||
| /// Intentionally exhaustive (not `#[non_exhaustive]`): adding a verdict is | ||
| /// a breaking change, so the compiler forces every consumer — in particular | ||
| /// the orchestrator's event mapping — to handle it explicitly. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] | ||
| pub enum WalkVerdict { | ||
| /// Nothing to decide yet; poll again by `deadline_millis`. | ||
| Waiting { | ||
| /// When the awaited checkpoint's window expires. | ||
| deadline_millis: u64, | ||
| }, | ||
| /// Every checkpoint passed — the device is up. Which state-machine | ||
| /// event this becomes is the shell's mapping, by component kind: | ||
| /// `ComponentReady` for an iRoT-backed device, `Booted` for a | ||
| /// symbiont. | ||
| Complete, | ||
| /// This boot attempt failed at `checkpoint`; the walk is over. | ||
| /// Whether to try again, recover, or give up is the orchestrator's | ||
| /// decision — a retry re-resets the device and starts a fresh walk. | ||
| Failed { | ||
| /// The checkpoint the attempt died at. | ||
| checkpoint: &'static str, | ||
| /// Why it died — the one input the retry decision needs. | ||
| cause: FailureCause, | ||
| }, | ||
| } | ||
|
|
||
| /// Why a boot attempt failed at a checkpoint. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] | ||
| pub enum FailureCause { | ||
| /// The window expired; the device reported nothing. | ||
| TimedOut, | ||
| /// The device reported a failure worth another attempt | ||
| /// ([`FailedRetriable`](crate::BootStatus::FailedRetriable)) — the | ||
| /// wait ended early. | ||
| DeviceRetriable, | ||
| /// The device reported a terminal failure | ||
| /// ([`FailedFatal`](crate::BootStatus::FailedFatal)) — re-running the | ||
| /// same image cannot change the verdict, whatever retry budget the | ||
| /// orchestrator has left. | ||
| DeviceFatal, | ||
| } | ||
|
|
||
| #[cfg(test)] | ||
| mod tests { | ||
| use super::*; | ||
|
|
||
| // A BootWatch implemented against no walker at all — the seam must be | ||
| // satisfiable by anything that can produce verdicts, and must stay | ||
| // object-safe (the fleet array below fails to compile otherwise). | ||
|
|
||
| struct ScriptedWalk { | ||
| verdicts: &'static [WalkVerdict], | ||
| next: usize, | ||
| } | ||
|
|
||
| impl BootWatch for ScriptedWalk { | ||
| fn poll(&mut self, _now_millis: u64) -> WalkVerdict { | ||
| let v = self.verdicts[self.next]; | ||
| self.next += 1; | ||
| v | ||
| } | ||
| } | ||
|
|
||
| #[test] | ||
| fn a_heterogeneous_fleet_pumps_through_the_erased_seam() { | ||
| let mut bmc = ScriptedWalk { | ||
| verdicts: &[ | ||
| WalkVerdict::Waiting { | ||
| deadline_millis: 90_000, | ||
| }, | ||
| WalkVerdict::Complete, | ||
| ], | ||
| next: 0, | ||
| }; | ||
| let mut nic = ScriptedWalk { | ||
| verdicts: &[ | ||
| WalkVerdict::Failed { | ||
| checkpoint: "heartbeat", | ||
| cause: FailureCause::TimedOut, | ||
| }, | ||
| WalkVerdict::Failed { | ||
| checkpoint: "heartbeat", | ||
| cause: FailureCause::DeviceFatal, | ||
| }, | ||
| ], | ||
| next: 0, | ||
| }; | ||
|
|
||
| let fleet: &mut [&mut dyn BootWatch] = &mut [&mut bmc, &mut nic]; | ||
|
|
||
| let first: [WalkVerdict; 2] = [fleet[0].poll(0), fleet[1].poll(0)]; | ||
| let second: [WalkVerdict; 2] = [fleet[0].poll(1), fleet[1].poll(1)]; | ||
|
|
||
| assert_eq!( | ||
| first, | ||
| [ | ||
| WalkVerdict::Waiting { | ||
| deadline_millis: 90_000 | ||
| }, | ||
| WalkVerdict::Failed { | ||
| checkpoint: "heartbeat", | ||
| cause: FailureCause::TimedOut | ||
| }, | ||
| ] | ||
| ); | ||
| assert_eq!( | ||
| second, | ||
| [ | ||
| WalkVerdict::Complete, | ||
| WalkVerdict::Failed { | ||
| checkpoint: "heartbeat", | ||
| cause: FailureCause::DeviceFatal | ||
| }, | ||
| ] | ||
| ); | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Retry & terminal decisions belong to the Orchestrator State Machine
WalkVerdict's retries_left, Retry, and Dead re-implement what orchestrator-sm already owns: ComponentStatus.retry/max_retry and the Recovering→RecoveryFailed path. uplicating the retry count means one side can think retries remain while the other says it's exhausted.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agreed, a retry re-resets the device, which restarts the whole walk — you can't retry "kernel" without re-passing bl1 and bl2 — so retry budgets are inherently per boot attempt, and boot attempts are owned by the orchestrator state machine. Fixed in e550a20: WalkVerdict is observation-only now (Waiting / Complete / Failed{checkpoint, cause}), retries_left/Retry/Dead are gone, and max_retries left the device table. The device's own judgment is still signalled by FailureCause::{TimedOut, DeviceRetriable, DeviceFatal}. DeviceFatal says no remaining budget can overturn the verdict.