Reduce GPU Requirements for Getting Started Guide

[danehans]

Currently, the vLLM deployment requires 3 replicas. We should consider using 1 replica to reduce GPU requirements. With 1 replica, the guide can still demonstrate LoRA-based load balancing.

[danehans]

Thoughts @ahg-g @kfswain?

[ahg-g]

yes, and also use max-lora 2 instead of 4 so that we allow using smaller gpus

[kfswain]

Maybe we make some callouts.

Reducing replica count to 1 may defeat the purpose of demoing a routing tool should someone choose to experiment/benchmark.

But agreed that accelerators are expensive and highly sought after. So will make some callouts to adjust the vLLM deployment based on what your have

[anshuman-agarwala]

/assign

[ahg-g]

Good point @kfswain ; other option is to use a kustomize template.