Should --name be propagated amongst the stack?

[BansheeHero]

Hi,

I have a strong expectation that every container following a 1:n relationship should spawn under the name provided and not ramalama. Are my expectations wrong?

• Version: Fedora 44 package of 0.21.0 (I really wish Fedora packaging would not be 2 versions behind)
• The same happens on 0.23.0 in python venv
• Considerations:
  1. You cannot spawn multiple instances of the --run --rag as they will fail due to hard coded host.containers.internal instead of network. (Both will likely fail instead of one succeeding: Address already in use)
  2. It will not re-use the existing containers making it 1:n relationship and not n:m
  3. When crashing or killing the main instance this leaves orphans running under generic names.
  4. ramalama already contains a procedure for generating a unique suffix.
  5. Running multiple instances of RAG is well withing the scope of current HW.
• Motivation: Spawning containers is expected behavior, but ramalama does not clean them up after a crash. Crashes are still common with Llama.cpp and when using Rag particularly.

CONTAINER ID  IMAGE                               COMMAND               CREATED             STATUS             PORTS                   NAMES
be77de6fc593  quay.io/ramalama/ramalama:0.21      llama-server --ho...  About a minute ago  Up About a minute  0.0.0.0:8082->8082/tcp  ramalama-OJl4wDfo0T
3bb864a34b86  quay.io/ramalama/ramalama-rag:0.21  rag_framework ser...  About a minute ago  Up About a minute  0.0.0.0:8080->8080/tcp  ragtest
f15cc7a75f4f  quay.io/ramalama/ramalama:0.21      llama-server --ho...  About a minute ago  Up About a minute  0.0.0.0:8081->8081/tcp  ramalama-8q1MHC2W1M

A smaller follow-up:
Is this a good place to have these constrained topics to answer? I believe so, but maintainers do have their preferences. Perhaps offloading work to the community on Discord.

[0xshubhs]

Your expectations aren't wrong — it's a gap, and it's visible in one function.

ramalama/plugins/runtimes/inference/rag/handler.py:151, _build_serve_args, is what constructs the args for each internal llama.cpp server. It builds a fresh Namespace rather than deriving from yours, and two fields are hardcoded:

network=None,      # :170
...
name=None,         # :179

That's both of your problems in one place.

name=None means every child falls through to genname() (common.py:298):

return "ramalama-" + "".join(random.choices(string.ascii_letters + string.digits, k=10))

which is exactly the ramalama-OJl4wDfo0T you're seeing. So to your point 4 — the unique-suffix machinery is there, it's just seeded from nothing instead of from args.name. Something like name=f"{args.name}-docling" / -embed / -caption would give you both the grouping and the uniqueness, and would make orphans after a crash identifiable rather than anonymous.

network=None is the cause of your point 1. With no shared network the children can only be reached over host.containers.internal on fixed ports, so a second stack collides rather than getting its own namespace. Propagating args.network — or creating a per-stack network named after --name — is what makes n:m possible. Note the ports are already allocated dynamically via compute_serving_port(..., exclude=allocated_ports) at :43-51, so the port side is solved; it's only the network that isn't.

Worth filing as an issue with those two lines quoted. They look independent but they're the same fix: _build_serve_args should inherit from args instead of resetting. The orphan cleanup you mention is downstream of this — you can't reliably reap containers you can't identify.

[BansheeHero]

I have been burned by an agent, are you human? The analysis is not bad even if it misses the point of asking the question.

[BansheeHero]

I decided to raise it as a feature request: #2906

I believe that while RAG cannot overlap and/or run on --internal networks there is a need keep builds tracked.