feat: make deploy-ollama script generic

[zdtsw]

change description:

• for security:
  • olllma: create SA, Role, RoleBinding, not set annotation in deployment
  • vllm: do not create SA, Role, RoleBinding, but set restricted-v2 SCC in deployment
  • both set securitycontext in deployment but based on different provider
    - rename old deploy-ollama.sh to deploy-quickstart.sh
    - introduce a utils.sh to host common functions
    - make deploy-quickstart.sh works for both ollama (default provider) and vllm
    - make new flag --model for user to pass in different model than default llama3.2:1b
    - make new flag --runtime-args --runtime-env for user to provide customized config
    vllm :https://github.com/vllm-project/vllm/blame/main/docs/deployment/k8s.md#L77
    ollama: keep old behavior https://github.com/llamastack/llama-stack-k8s-operator/blob/main/hack/deploy-ollama.sh#L86 make --keepalive as env variable
    - rename old ollama-scc.sh to quickstart-scc.sh to support vllm if called via deploy-quickstart.sh
    - update documents to reflect changes
    - default config for vllm is on CPU only, if run on GPU should set env or args
    - add readinessprob
    ollama: https://github.com/ollama/ollama/blob/main/docs/api.md#version
    vllm: https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/openai/api_server.py#L414

to fix: #76

[rhdedgar]

Very cool to see support for anther provider! I believe vLLM should work without the scc logic, so maybe moving that call to quickstart-scc.sh into a if [ "${PROVIDER}" = "ollama" ]; then block would be enough.

Thank you!

[rhdedgar]

Some good news here: I only know this SecurityContextConstraint to be required for ollama container deployments in particular.

It would be worth using the default restricted-v2 SCC (no extra ServiceAccount config needed) unless the chosen provider deployment type requires it.

[zdtsw]

oh, nice
for the ollama: we need SA, SCC, Role, RB, and set security context in Deployment
for vllm: we can skip all ^ these. set annotation to restricted-v2
is this correct understanding?

[rhdedgar]

Yep! The annotation was just to meant to convey which SecurityContextConstraint the ollama-sa ServiceAccount uses, but shouldn't be necessary for others. It can be omitted for vllm-sa for example, because a newly created SA will use the restricted-v2 SCC by default.

So for just the ollama provider we can run hack/quickstart-scc.sh, and I expect we can safely skip it for other providers.

[zdtsw]

Very cool to see support for anther provider! I believe vLLM should work without the scc logic, so maybe moving that call to quickstart-scc.sh into a if [ "${PROVIDER}" = "ollama" ]; then block would be enough.

Thank you!

You are right 👍
unless needed (in ollama case) should not grant scc on uid 0, which can be a security concern, esp. vllm does not run on root. I will fix this

[rhdedgar]

As we add more providers in the future, we may want to organize the yaml resources in plain .yaml files in config/samples to make the examples easy to understand, but this approach may be ok for now.

I'll see if @leseb and @VaishnaviHire have a preference on if we should look into this approach now for vLLM, or in a later refactor.

[rhdedgar]

It might be good to mention the format that the BASH_REMATCH commands match in utils.sh, or maybe prompt for the value if the secret doesn't already exist.

And a slight formatting change:
namesapce ->
namespace

[zdtsw]

did some updates to include the typo also add the "error out" on the missing secret case.

[rhdedgar]

I think that covers all the questions I had. Thanks!