feat(agents): multi-device support — CPU, GPU, NPU per-agent selection#1252
Conversation
#1220) Agents now declare which (device, model, recipe, backend) tuples they support via DeviceConfig. Users pick a device per-agent in the Agent UI dropdown or via `--device {cpu,gpu,npu}` on CLI. GPU is the default. NPU uses the FLM backend on Ryzen AI XDNA2 hardware; CPU falls back with a latency warning. Backend: DeviceConfig dataclass, GaiaConfig persistence (~/.gaia/config.json), LemonadeClient.install_backend/uninstall_backend/get_recipe_status, LemonadeManager device parameter, session-level device column + migration. CLI: `gaia init --profile npu` (NPU detection, FLM backend install, recipe- aware model download), `gaia chat --device npu`, `gaia eval agent --device npu`. Frontend: DeviceConfig type, activeDevice/detectedDevices store state, per-agent device dropdown with verified/unverified badges. Eval-verified on Ryzen AI MAX+ PRO 395: NPU matches or exceeds GPU quality (personality 3/3 9.5avg, context_retention 4/4 9.8avg) at ~24 tok/s.
github-actions
Bot
added
documentation
kovtcharov-amd
modified the milestones:
v0.19 — Test & CI Hardening [OSS],
v0.20 Email Agent & Platform Foundations
- Black: reformat cli.py - Pylint: remove reimport of LemonadeClient (already at module level) - Flake8: remove unused imports (tempfile, Path) in test file
|
@itomek-amd, this feature requires manual testing. |
itomek
left a comment
itomek
left a comment
There was a problem hiding this comment.
Solid, well-layered feature: the device abstraction threads cleanly from DeviceConfig/DEFAULT_DEVICE_CONFIGS in registry.py through the CLI --device, the sessions DB migration, the agents API response, and the UI dropdown, with docs (npu.mdx + docs.json) and a 382-line test file. GPU stays the default and explicit-device requests fail loudly when hardware is missing — the right shape. One process note: this is an LLM-affecting change (it swaps the model per device), and CLAUDE.md asks for a gaia eval agent run compared to the committed baseline. The description cites strong NPU numbers but no baseline scorecard / --compare output is included — worth attaching so a reviewer can see GPU-vs-NPU parity rather than take the numbers on faith, especially since the NPU path runs at a much smaller ctx (4096 vs 32768). Two small inline notes on the device-probe except: pass and the _DEVICE_TO_MIN cpu omission. Approving on the understanding the eval evidence gets attached.
Generated by Claude Code
…untime Multi-device support (#1252) shipped non-functional: the Agent UI device dropdown and CLI --device flag never changed the model, and an unavailable device degraded silently. Fixes v0.20.0 release-review blockers B1/B2/H1 plus a GPU-tier validation bug. B1: resolve the device's DeviceConfig.model (and ctx_size) at every UI agent-build site instead of always using the session/GPU model; rewrite the session model on a device switch so eviction/recreation picks it up. A guard keeps an agent's own pinned model from being clobbered on the default GPU. B2: add a device field to ChatAgentConfig, thread it through the base Agent into LemonadeManager.ensure_ready(device=...), and fail loudly with an actionable HardwareRequirementError when the requested device is absent. H1: narrow the CLI device-probe except clause to only swallow connection/ timeout errors; a reachable-but-broken Lemonade now surfaces. Make the GPU->CPU fallback state its reason. Medium: map the gpu selector to amd_dgpu (lowest GPU tier) so a discrete- Radeon-only host satisfies an explicit GPU requirement.
2 participants
GAIA agents couldn't leverage AMD Ryzen AI NPU hardware — inference defaulted to GPU via llamacpp with no way to select an alternative device. Users with XDNA2 NPUs had no path to power-efficient local inference, and there was no framework for per-agent device selection across CPU, GPU, and NPU.
Now each agent declares which devices it supports via
DeviceConfigtuples. Users pick a device per-agent — dropdown in Agent UI,--device {cpu,gpu,npu}on CLI. GPU remains the default. NPU uses the FLM backend (gemma4-it-e2b-FLM); CPU falls back automatically with a latency warning.gaia init --profile npuhandles NPU detection, FLM backend installation, and model download. Eval-verified on Ryzen AI MAX+ PRO 395: NPU matches or exceeds GPU quality (personality 3/3 @ 9.5/10, context_retention 4/4 @ 9.8/10) at ~24 tok/s.Test plan
python -m pytest tests/unit/test_npu_device_support.py -xvs— 25 testsgaia chat --device gpu— announces device, loads correct modelgaia chat --device npu— loads FLM model on NPU hardwaregaia chat --device cpu— warns about slow response timesgaia init --profile npu— detects NPU, installs FLM backend, downloads modelgaia init --profile npuon non-NPU hardware — fails loudlygaia eval agent --device npu --category personality— all scenarios pass