What you'd like to see
Better support for DMR's feature-set, similar to the support compose provides (e.g. for runtime parameters and context length)
Why you'd like to see it
To better take advantage of local models
Workarounds?
Creating the local model runtimes with a compose file, then pointing the base url in the agentfile to the appropriate urls exposed by compose (which might even need to be proxied to localhost, not sure atm)
Additional context
I'd want to be able to:
models:
localboi:
provider: dmr
model: ai/qwen3:14B-Q6_K
ctx_length (or max_tokens): 32000
runtime_opts: |
<llama.cpp_args_here>
...
...
...
What you'd like to see
Better support for DMR's feature-set, similar to the support compose provides (e.g. for runtime parameters and context length)
Why you'd like to see it
To better take advantage of local models
Workarounds?
Creating the local model runtimes with a compose file, then pointing the base url in the agentfile to the appropriate urls exposed by compose (which might even need to be proxied to localhost, not sure atm)
Additional context
I'd want to be able to: