fix(gallery): keep auto-upgrade off non-dev backends when -development is installed#9736
Merged
Merged
Conversation
…t is installed
A `-development` backend variant (e.g. `cuda12-llama-cpp-development`)
shares its `alias` with the stable counterpart and is meant to be a
drop-in replacement via ListSystemBackends alias resolution. Two paths
in the auto-upgrade flow let the stable variant slip back in on top of
the user's explicit dev pick:
1. ListSystemBackends emits a synthetic alias row keyed by the alias
name that re-uses the chosen concrete's metadata pointer. In
distributed mode, the worker's handleBackendList serialised that
row over NATS as `{Name: <alias>, URI: <dev URI>, Digest: <dev>}`
— the frontend can't reconstruct the alias relationship, and the
wire-rebuilt row then carried `Metadata.Name = <alias>` and
resolved against an unrelated gallery entry on the next upgrade
check.
2. CheckUpgradesAgainst happily iterated the synthetic row in
single-node too. Today the duplicate gallery lookup is harmless
because both rows share the same `Metadata.Name`, but any gallery
change that gives a meta backend a version, or any concrete
sharing its alias with a dev counterpart, would surface a phantom
non-dev upgrade and auto-upgrade would install it — shadowing the
dev one through alias-token preference.
Two layered fixes:
- `core/services/worker/lifecycle.go` (`handleBackendList`): drop
rows where the map key differs from `b.Metadata.Name`. Concrete
and meta entries always have `key == Metadata.Name`; only synthetic
aliases violate it. Workers now report only what's actually on disk;
the per-node UI listing and CheckUpgrades both stop seeing phantoms.
- `core/gallery/upgrade.go` (`CheckUpgradesAgainst`): iterate by key,
skip rows where `key != Metadata.Name` (belt-and-suspenders for any
caller-supplied installed set), and apply the dev-aware rule —
build a set of installed `Metadata.Name`s and drop any non-dev
candidate `X` whose `X-<devSuffix>` counterpart is installed. Uses
the configured dev suffix from `getFallbackTagValues(systemState)`.
Manual `POST /api/backends/upgrade/<name>` is unaffected: it goes
straight through `bm.UpgradeBackend(name)` without consulting the
suppression list, so users who genuinely want the stable variant
upgraded can still trigger it explicitly.
Tests in core/gallery/upgrade_test.go cover three cases under
"CheckUpgradesAgainst (distributed)": dev-only installed → only the
dev surfaces; both variants installed → dev still wins; synthetic
alias row is ignored. Generic backend names are used to avoid the
capability filter dropping cuda-prefixed entries on a CPU-only host.
Assisted-by: Claude:claude-opus-4-7
Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A
-developmentbackend variant (e.g.cuda12-llama-cpp-development) shares itsaliaswith the stable counterpart and is meant to be a drop-in replacement via ListSystemBackends alias resolution. Two paths in the auto-upgrade flow let the stable variant slip back in on top of the user's explicit dev pick:{Name: <alias>, URI: <dev URI>, Digest: <dev>}— the frontend can't reconstruct the alias relationship, and the wire-rebuilt row then carriedMetadata.Name = <alias>and resolved against an unrelated gallery entry on the next upgrade check.Metadata.Name, but any gallery change that gives a meta backend a version, or any concrete sharing its alias with a dev counterpart, would surface a phantom non-dev upgrade and auto-upgrade would install it — shadowing the dev one through alias-token preference.Two layered fixes:
core/services/worker/lifecycle.go(handleBackendList): drop rows where the map key differs fromb.Metadata.Name. Concrete and meta entries always havekey == Metadata.Name; only synthetic aliases violate it. Workers now report only what's actually on disk; the per-node UI listing and CheckUpgrades both stop seeing phantoms.core/gallery/upgrade.go(CheckUpgradesAgainst): iterate by key, skip rows wherekey != Metadata.Name(belt-and-suspenders for any caller-supplied installed set), and apply the dev-aware rule — build a set of installedMetadata.Names and drop any non-dev candidateXwhoseX-<devSuffix>counterpart is installed. Uses the configured dev suffix fromgetFallbackTagValues(systemState).Manual
POST /api/backends/upgrade/<name>is unaffected: it goes straight throughbm.UpgradeBackend(name)without consulting the suppression list, so users who genuinely want the stable variant upgraded can still trigger it explicitly.Tests in core/gallery/upgrade_test.go cover three cases under "CheckUpgradesAgainst (distributed)": dev-only installed → only the dev surfaces; both variants installed → dev still wins; synthetic alias row is ignored. Generic backend names are used to avoid the capability filter dropping cuda-prefixed entries on a CPU-only host.
Assisted-by: Claude:claude-opus-4-7