[CLI] llama-stack-client CLI for querying server distro

[yanxi0830]

CLI Reference

llama-stack-client

$ llama-stack-client -h

usage: llama-stack-client [-h] {models,memory_banks,shields} ...

Welcome to the LlamaStackClient CLI

options:
  -h, --help            show this help message and exit

subcommands:
  {models,memory_banks,shields}

llama-stack-client configure

$ llama-stack-client configure
> Enter the host name of the Llama Stack distribution server: localhost
> Enter the port number of the Llama Stack distribution server: 5000
Done! You can now use the Llama Stack Client CLI with endpoint http://localhost:5000

llama-stack-client models list

llama-stack-client models list

llama-stack-client models get

llama-stack-client models get Llama3.1-8B-Instruct

$ llama-stack-client models get Random-Model 

Model RandomModel is not found at distribution endpoint host:port. Please ensure endpoint is serving specified model.

llama-stack-client memory_banks list

llama-stack-client memory_banks list

llama-stack-client shields list

llama-stack-client shields list

[raghotham]

I should be able to just configure the client cli to point to a host by providing an API key if needed.

$ llama-stack-client configure
host: ...
port: ...
api_key: ...

After that, I should be able to do:

$ llama-stack-client <models|memory_banks|shields> <list|get>

without having to specify host/port each time.

[ashwinb]

Requesting changes mainly for Raghu's comment on doing a configuration first and then reusing those values for contacting the distribution server.

A small nit -- can the cli directory be placed until src/llama_stack_client instead of inside src/llama_stack_client/lib/

[yanxi0830]

Requesting changes mainly for Raghu's comment on doing a configuration first and then reusing those values for contacting the distribution server.

A small nit -- can the cli directory be placed until src/llama_stack_client instead of inside src/llama_stack_client/lib/

For purely additive helper or utility code, consider adding it to the package's lib directory. Stainless will never modify the contents of these directories.

I put it to lib for now to be compatible w/ Stainless's custom code suggestion, and think may us save some further trouble when syncing. We can do further refactor when API is more stable.

[ashwinb]

This looks pretty reasonable to me, but I want to have @raghotham take one look at it since he has had a lot of thoughts on how precisely this needs to work.

[ashwinb]

I don't think we should have this one if we have separate host and port arguments. It feels confusing.

[yanxi0830]

I was thinking that if we have an endpoint like https://llama-stack.together.ai, we don't need to specify separate host:port args.

[ashwinb]

So I think @raghotham wanted things to work like the aws CLI. i.e., there's a notion of a "profile"... its default value is "default". And you can update the default. And then each command gets an optional "profile" name which it uses to look up the corresponding YAML and gets all its values from there.

Specifically, we should never have an endpoint here (and in all commands) but only a "name" which represents a link to the YAML.

[yanxi0830]

Hmm, my understanding is that for now we only have endpoint to store in "profile", but I could remove this --endpoint s.t. we read from "profile" config only.

[yanxi0830]

currently saves endpoint only, could introduce a map later if we want to store different endpoints.

[ashwinb]

Let's get this in!