Skip to content

fix(promote): build channel-aware delta when promoting across skipped versions#84

Merged
peters merged 1 commit intomainfrom
fix/promote-channel-aware-delta
Apr 12, 2026
Merged

fix(promote): build channel-aware delta when promoting across skipped versions#84
peters merged 1 commit intomainfrom
fix/promote-channel-aware-delta

Conversation

@peters
Copy link
Copy Markdown
Contributor

@peters peters commented Apr 12, 2026

Summary

surge pack always builds the release delta against the immediately-previous version overall, regardless of channel. When intermediate releases live only on test between two production releases, the production-side delta basis no longer matches what production nodes have on disk. The first per-file PatchFile basis-hash check during apply fails:

```
Sparse delta file hash mismatch for 'camera-tuner.deps.json':
expected , got
```

The supervisor then never activates the delta. The only operational workaround has been `surge compact ` after every promotion, which deletes the broken delta and forces every fleet node into a full reinstall. This pattern shows up in every production promotion log of the youpark fleet (Apr 10, Apr 11, Apr 12).

What changes

  • commands/promote.rs — after `ensure_release_full_artifact`:
    1. Look up the previous release on the target channel.
    2. If the release does not already carry a delta with `from_version` matching that predecessor, restore both full archives, build a sparse delta, upload it as `---from--delta.tar.zst`, and `upsert_delta` it onto the release entry.
    3. Original test-channel delta is preserved (multi-delta), so test chains keep working.
    4. Re-running `surge promote` on an already-promoted release also rebuilds missing channel deltas, so releases promoted before this fix can recover without a demote/repromote cycle.
  • update/manager/release_index.rs — chain walker resolves each apply step by `delta_from_source(prev_step.version)`. Releases pushed before `from_version` was tracked carry one delta with an empty `from_version`; the resolver accepts those as a legacy fallback so existing chains keep working. Returns `None` (forcing full install) when no compatible delta exists, instead of letting the apply path crash with the basis-hash mismatch.
  • releases/manifest.rs — adds `ReleaseEntry::delta_from_source` and `ReleaseEntry::upsert_delta` helpers.

Test plan

  • `./scripts/sync-surge-core-vendor.sh --check`
  • `./scripts/check-version-sync.sh`
  • `cargo fmt --all -- --check`
  • `RUSTFLAGS="-D warnings" cargo test --workspace` (358 tests, +4 new)
  • `cargo clippy --workspace --all-targets --all-features -- -D warnings`
  • `cargo clippy --workspace --lib --bins --examples -- -D warnings -D clippy::unwrap_used -D clippy::expect_used`
  • `cargo clippy --workspace --all-targets --all-features -- -D warnings -W clippy::pedantic`

New tests

  • `commands::promote::tests::execute_rebuilds_channel_delta_when_promoting_across_skipped_versions` — constructs real archives via `ArchivePacker`, sets up the `v1.0.0(prod)` / `v1.1.0(test)` / `v1.2.0(test)` layout from the field bug, promotes `v1.2.0` to production, and verifies the rebuilt delta correctly transforms a v1.0.0 archive into v1.2.0 with the new `camera-tuner.deps.json` content.
  • `update::manager::release_index::tests::resolver_picks_delta_matching_current_install_version`
  • `update::manager::release_index::tests::resolver_falls_back_to_legacy_delta_with_empty_from_version`
  • `update::manager::release_index::tests::resolver_returns_none_when_no_delta_matches_or_is_legacy`

… versions

Surge `pack` builds the release delta against the immediately-previous version
overall, regardless of channel. When intermediate releases live only on `test`
between two `production` releases, the production-side delta basis no longer
matches what production nodes have on disk. The first per-file `PatchFile`
basis-hash check during apply throws

  Sparse delta file hash mismatch for 'camera-tuner.deps.json':
    expected <previous-test hash>, got <previous-production hash>

and the supervisor never activates the delta. The only operational workaround
has been `surge compact <prod>` after every promotion, which deletes the broken
delta and forces every fleet node into a full reinstall.

Make `surge promote` channel-aware. After ensuring the new release's full
artifact exists in storage:

* Look up the previous release on the target channel.
* If the new release does not already carry a delta with `from_version` matching
  that predecessor, restore both full archives, build a sparse delta, upload it
  as `<app>-<version>-<rid>-from-<prev>-delta.tar.zst`, and `upsert_delta` it
  onto the release entry without removing the original test-channel delta.

The chain walker in `update::manager::release_index` now resolves each apply
step by `delta_from_source(prev_step.version)` so production and test nodes
each pick the delta whose basis matches their installed version. Releases
pushed before `from_version` was tracked carry one delta with an empty
`from_version`; the resolver still accepts those as a legacy fallback so
existing chains keep working.

The early no-op return only fires when the release is already on the channel
**and** the channel-specific delta is already present, so re-running
`surge promote` recovers releases that were promoted before this fix shipped
without a demote/repromote cycle.

`ReleaseEntry::delta_from_source` and `ReleaseEntry::upsert_delta` are the
small primitives the new logic builds on. The regression test for promote
constructs real archives via `ArchivePacker`, sets up the v1.0.0(prod) /
v1.1.0(test) / v1.2.0(test) layout from the field bug, promotes v1.2.0 to
production, and verifies the rebuilt delta correctly transforms a v1.0.0
archive into v1.2.0 with the new `camera-tuner.deps.json` content.

Validation:
- ./scripts/sync-surge-core-vendor.sh --check
- ./scripts/check-version-sync.sh
- cargo fmt --all -- --check
- RUSTFLAGS="-D warnings" cargo test --workspace
- cargo clippy --workspace --all-targets --all-features -- -D warnings
- cargo clippy --workspace --lib --bins --examples -- -D warnings -D clippy::unwrap_used -D clippy::expect_used
- cargo clippy --workspace --all-targets --all-features -- -D warnings -W clippy::pedantic
@peters peters merged commit bb9caa8 into main Apr 12, 2026
12 checks passed
@peters peters deleted the fix/promote-channel-aware-delta branch April 12, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant