Proposal: Add --engine-args parameter for engine/podman parameters

[eduncan911]

We have --runtime-args to pass arbitrary parameters into llama-serve and other commands.

We need the same for the underlying podman/docker engine container runtime. This will allow us to pass additional parameters into Podman for things like mounting additional files, like mmproj files needed. Something like:

--engine-args : [Default=""] Used to pass additional parameters into the underlying engine such as Docker or Podman.

Example use:

ramalama \
  --engine-args "--mount=type=bind,src=/models/qwen3.6-mmproj/mmproj.gguf,destination=/mnt/models/mmproj.gguf,ro" \
  serve \
  --temp 1.0 \
  --ngl 99 \
  --runtime-args "--mmproj=/mnt/models/mmproj.gguf" \
  ...

Original post below:

I may be missing something; but, I am not able to get mmproj files working with Ramalama.

ramalama serve looks to bind mount the model passed as a parameter as /mnt/models/<filename>.gguf. However, passing --mmproj into the --runtime-args doesn't work as the mmproj files are not mounted.

I searched the codebase here for some type of file glob pattern that I can rename my mmproj file to or something, for some kind of auto-detect, but I came up short.

I see there has been a PR merged that allows for mmproj files to be specified in the .container files when --generate quadlet is used. This may be a fallback approach; but right now, I just want to mount and test various models - not setup each and every model as a full blown Systemd service.

---

If Ramalama is going to limit what can be mounted then we may need some type of auto-detection based on a naming convention for additional models. Otherwise, there has to be additional parameters to control new bindings. One or the other.

➜  ls -la ~/.local/share/ramalama/store/huggingface/unsloth/gemma-4-E4B-it-GGUF/
total 0
drwxr-xr-x. 1 eric eric 270 Apr 29 10:32 .
drwxr-xr-x. 1 eric eric 888 Apr 26 09:47 ..
drwxr-xr-x. 1 eric eric  36 Apr 26 00:46 gemma-4-E4B-it-UD-Q4_K_XL.gguf
drwxr-xr-x. 1 eric eric  36 Apr 26 00:24 gemma-4-E4B-it-UD-Q5_K_XL.gguf
drwxr-xr-x. 1 eric eric  36 Apr 26 00:00 gemma-4-E4B-it-UD-Q6_K_XL.gguf
drwxr-xr-x. 1 eric eric  36 Apr 25 23:23 gemma-4-E4B-it-UD-Q8_K_XL.gguf
drwxr-xr-x. 1 eric eric  36 Apr 29 10:32 mmproj-F16.gguf

[eduncan911]

The good news: I was able to use --debug and grab the exec_cmd of the entire podman command. This allowed me to add an additional --mount to point specifically to the mmproj.gguf file and add the additional parameters to llama-serve!

The bad news: i don't think there's a point in using Ramalama if I am using podman to start the container with its --label init sequence?

---

In the end, I think Ramalama should expose a parameter to allow us to add additional Docker/Podman engine attributes, much like how --runtime-args with adding raw commands to llama-serve. Something like --engine-args that could look like this:

ramalama --engine-args="--mount=..." serve hf://path/to/model --runtime-args="..." ...

I think being before the Action command makes it generic across all actions (serve, bench, run, etc) .

[rhatdan]

Agree on adding engine-args option.