Fix DPU firmware update health alert never clearing on DPF hosts

[wminckler]

DPF-ingested hosts never had their HostUpdateInProgress health alert removed after an automatic DPU firmware update finished. The machine would reprovision successfully, come back Ready, and then sit permanently with an AutomaticDpuFirmwareUpdate alert pinned to it. Non-DPF hosts were unaffected.

The cause is an asymmetry between how an update is started and how it is completed. DPF hosts are deliberately excluded from the version-based staleness check — find_outdated_dpus returns early for any host with dpf.used_for_ingestion, and find_outdated_dpus_dpf selects those hosts instead by comparing against the expected BFB from their DPUDeployment. The completion path had no such split: it only removed the update markers when the DPU's reported NIC firmware appeared in dpu_nic_firmware_update_versions, a config list that only ever contains NIC firmware version strings. Nothing in the DPF path constrains a DPU to land on a version from that list, so for a DPF host the comparison had no reason to ever match, and the marker was never removed.

The consequences went beyond a stale alert. The alert carries prevent_allocations, so affected hosts stayed unallocatable indefinitely. The lingering alert also makes is_available_for_updates return false, which excluded those hosts from every subsequent firmware update. And each manager pass emitted a spurious WrongVersionAfterUpdate failure metric for a host that had in fact updated correctly.

The fix carries the host's DPF-ingestion flag through DpuMachineUpdate so the completion path can mirror the split the start path already has, skipping the NIC-version comparison for DPF hosts. The rest of the completion gate is untouched: the host must still be holding the update marker, be in ManagedHostState::Ready, and have every one of its DPUs' reprovisioning_requested flags cleared before anything is removed.

Related issues

None.

Type of Change

• Add - New feature or capability
• Change - Changes in existing functionality
• Fix - Bug fixes
• Remove - Removed features or deprecated functionality
• Internal - Internal changes (refactoring, tests, docs, etc.)

Breaking Changes

• This PR contains breaking changes

Testing

• Unit tests added/updated
• Integration tests added/updated
• Manual testing performed
• No testing required (docs, internal refactor, etc.)

Added test_clear_completed_updates_dpf_host_off_list_version, which marks a host DPF-ingested, lands its DPU on a firmware version asserted to be absent from dpu_nic_firmware_update_versions, clears the reprovisioning flag, and leaves the update marker in place. It asserts the health report is removed, aggregate health is clean, and no wrong_version_after_update failure is counted. The test fails against the previous behaviour.

Ran the dpu_nic_firmware (7 tests), dpu_machine_update, and machine_update_manager (18 tests) suites — all passing. Not yet exercised against a live DPF-managed cluster.

Additional Notes

DpuMachineUpdate::dpf_managed is marked #[sqlx(default)] because the struct is also loaded via FromRow in get_reprovisioning_machines, whose query does not select the column; that path only consumes host_machine_id, so defaulting to false there is inert.

Worth a reviewer's opinion: when a DPF host's reprovision completes but the DPU is still mismatched against a DPUDeployment that changed mid-update, this clears the marker rather than retaining it. That is deliberate — run_single_iteration calls clear_completed_updates and then start_updates in the same pass, so the host is re-detected by find_outdated_dpus_dpf and the update is re-triggered immediately, with the marker re-applied. Retaining the marker instead would make is_available_for_updates false forever and recreate exactly the permanent-pin failure this PR fixes.

🤖 Generated with Claude Code

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f79dd42b-3014-4126-8f67-972c08604660

📥 Commits

Reviewing files that changed from the base of the PR and between 858c158 and 7ba222e.

📒 Files selected for processing (7)

• crates/api-core/src/machine_update_manager/dpu_nic_firmware.rs
• crates/api-core/src/tests/dpu_machine_update.rs
• crates/api-core/src/tests/dpu_nic_firmware.rs
• crates/api-core/src/tests/instance.rs
• crates/api-core/src/tests/machine_update_manager.rs
• crates/api-db/src/dpu_machine_update.rs
• crates/api-model/src/dpu_machine_update.rs

---

Summary by CodeRabbit

• Bug Fixes

  • DPF-managed machine updates are now correctly marked complete after reprovisioning, even when the resulting NIC firmware version is outside the configured allowlist.
  • Prevented completed DPF updates from being incorrectly reported as wrong-version failures.
  • Ensured update markers and related alerts are cleared when DPF reprovisioning finishes.

• Tests

  • Added coverage validating successful completion handling for DPF-managed machines.

Walkthrough

The change adds DPF provenance to DpuMachineUpdate, propagates it from DPF-ingested hosts, and treats completed DPF updates as successful regardless of firmware allowlist membership. Tests verify marker cleanup, metric behavior, and explicit non-DPF initialization.

Changes

DPF firmware completion

Layer / File(s)	Summary
Update provenance and discovery crates/api-model/src/dpu_machine_update.rs	Adds dpf_managed to DpuMachineUpdate and sets it for DPF and non-DPF discovery paths.
Completed update propagation crates/api-db/src/dpu_machine_update.rs	Derives DPF-management status from host ingestion configuration when constructing completed updates.
Completion handling and validation crates/api-core/src/machine_update_manager/dpu_nic_firmware.rs, crates/api-core/src/tests/*	Routes DPF-managed completions through success handling, removes update markers, avoids wrong-version failure metrics, and updates test fixtures with explicit provenance values.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant DPFDiscovery
  participant CompletedUpdates
  participant ClearCompletedUpdates
  participant UpdateHealth
  participant FirmwareMetrics
  DPFDiscovery->>CompletedUpdates: Set dpf_managed provenance
  CompletedUpdates->>ClearCompletedUpdates: Return completed DpuMachineUpdate
  ClearCompletedUpdates->>UpdateHealth: Remove update marker
  ClearCompletedUpdates->>FirmwareMetrics: Skip wrong-version failure metric

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the bug fix and identifies the affected DPF-host DPU firmware alert.
Description check	✅ Passed	The description directly matches the code changes and tests, so it is clearly relevant to the changeset.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}