Skip to content

Failed to get model definition wit /compact + Docker Model Runner #2800

@k33g

Description

@k33g

When running docker-agent against a model served by Docker Model Runner (provider dmr), the manual /compact command always fails with the error: Failed to get model definition

Additionally, automatic compaction never runs.

Environment

  • Provider: dmr (Docker Model Runner, local inference via llama-server)
  • Example model: huggingface.co/unsloth/qwen3.5-4b-gguf:Q4_K_M
  • provider_opts.context_size: 32768 is set in the agent YAML
  • base_url: http://localhost:12434/engines/v1

Steps to reproduce

  1. Configure an agent that uses a local DMR model (see agent.yaml snippet below).
  2. Start a conversation and let it grow.
  3. Run /compact at any point.
  models:
    qwen-local:
      provider: dmr
      model: huggingface.co/unsloth/qwen3.5-4b-gguf:Q4_K_M
      base_url: http://localhost:12434/engines/v1
      max_tokens: 24576
      provider_opts:
        context_size: 32768

Expected behavior

  • /compact should produce a summary of the conversation and free up context.
  • When the conversation approaches the configured context_size (32768), proactive compaction should trigger automatically.

Actual behavior

  • /compact emits Failed to get model definition and exits without compacting anything.
  • The conversation continues to grow until the context window overflows; automatic compaction never fires.

Version of docker-agent: 1.50.0

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions