Skip to content

fix(model-artifacts): persist companion artifacts so remote workers get the base_model option#11075

Merged
mudler merged 1 commit into
masterfrom
fix/distributed-companion-path
Jul 23, 2026
Merged

fix(model-artifacts): persist companion artifacts so remote workers get the base_model option#11075
mudler merged 1 commit into
masterfrom
fix/distributed-companion-path

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

The bug

Distributed loads of a model with a companion artifact (e.g. longcat-video-avatar-1.5, whose base_model companion points at meituan-longcat/LongCat-Video) fail on the remote worker with:

rpc error: code = InvalidArgument desc = base_model must point to a LongCat-Video checkpoint

and the backend log shows it fetching the base repo itself instead of using the staged companion:

Downloading required files for meituan-longcat/LongCat-Video

Root cause

preloadOne resolves the full artifact set (primary + every companion) in memory, but the binding written back to disk carried only the primary:

// persistArtifactBinding, before
encoded, err := yaml.Marshal([]modelartifacts.Spec{result.Spec})
...
setMappingValue(target, "artifacts", ...)   // replaces the WHOLE artifacts: list

setMappingValue overwrites the entire artifacts: list, so every target: companion entry was silently dropped from the config file.

In a single process this is invisible (the in-memory config still holds the companion). It bites on the next controller restart: the config reloads from the mangled file with the primary alone, so withCompanionArtifactOptions finds no resolved companion and synthesizes no base_model option. The remote longcat-video backend never receives base_model, falls back to BASE_MODEL_ID, and downloads the repo itself — failing validation.

This also explains the discriminating observation from live testing: adding an explicit base_model:<path> to the model's options: works, while the managed companion does not. Explicit options live in options:, which the binding never rewrites; the managed companion lives in artifacts:, which the binding overwrote.

What this is NOT

The companion path nesting (bare /models/.artifacts/... vs per-model staged /models/longcat-video-avatar-1.5/.artifacts/...) is already handled by #10949: staging derives the worker ModelPath as the nested per-model staged root, and the relative companion option resolves under it (router_companionstage_test.go pins this). Rewriting the option with a per-model prefix would double the nesting and break it, so staging is left untouched. Once the option is generated again (this fix), the existing staging/resolution path works.

The fix

  • persistArtifactBinding now writes the full resolved artifact set (primary + all companions).
  • bindingNeedsPersistence compares the whole artifact list, so a companion resolving for the first time still triggers a write.

Single-node is unaffected (the in-memory config already carried the companion). The gallery-install path (bindPrimaryArtifact) already persisted the full list; only the preload write-back was lossy.

Tests (TDD, Ginkgo/Gomega)

  • New failing-first spec keeps every resolved artifact in the persisted file across a reload (models a controller restart with a fresh loader): red before, green after.
  • New persistArtifactBinding unit spec proving both primary and companion survive the write.

Verification (exit codes read separately)

  • go build ./core/... ./pkg/... → 0
  • go test ./core/backend/ → 0
  • go test ./core/services/nodes/ → 0 (196s)
  • go test ./core/config/ → 0
  • golangci-lint run --new-from-merge-base=origin/master ./core/config/ → 0

This is the last blocker in the distributed video-model bring-up (after #10986, #11029, #11030, #11053, #11071): the backend now receives the managed base_model companion and resolves it from the staged snapshot instead of downloading it.

🤖 Generated with Claude Code

https://claude.ai/code/session_0156CbKpDh8qbAYzJZJDZ2yk

A managed model can declare companion artifacts (LongCat-Video-Avatar-1.5
pulls its tokenizer, text encoder and VAE from the separate LongCat-Video
base repo via a target: companion artifact). preloadOne resolves the whole
set in memory, but the binding written back to disk carried only the
primary: persistArtifactBinding marshalled []Spec{result.Spec} and replaced
the entire artifacts: list with it, silently dropping every companion.

In a single process the loss is invisible because the in-memory config keeps
the companion. It bites on the next controller restart: the config reloads
from the mangled file with the primary alone, so withCompanionArtifactOptions
finds no resolved companion and synthesizes no base_model option. The remote
longcat-video backend then never receives base_model, falls back to
BASE_MODEL_ID and downloads the repo itself ("Downloading required files for
meituan-longcat/LongCat-Video"), failing the load with "base_model must point
to a LongCat-Video checkpoint".

This is why an explicit base_model:<path> added to the config options works
where the managed companion does not: an explicit option lives in options:,
which is never rewritten, while the managed companion lives in artifacts:,
which the binding overwrote.

Persist the full resolved set (primary + every companion), and widen
bindingNeedsPersistence to compare the whole artifact list so a companion
resolving for the first time still triggers a write. The single-node path is
unaffected: there the in-memory config already carried the companion, and the
staging/ModelPath resolution for a remote worker (nested per-model staged
root, #10949) is unchanged and already correct once the option is generated.

Assisted-by: Claude Code:claude-opus-4-8[1m] [Read] [Edit] [Bash]

Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
@mudler
mudler merged commit 2fe10c3 into master Jul 23, 2026
65 of 67 checks passed
@mudler
mudler deleted the fix/distributed-companion-path branch July 23, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants