How to have LLM see raspberry pi endpoint as a model?

[thejeangenie18]

Hiya,

Working on a personal project using a small distributed setup with llm and I'm trying to register a remote TinyLlama instance running on a Raspberry pi 3b+ (yes, that little guy can work!). The pi exposes an OpenAI-compatible API at http://:11435/v1
Its job is to run background tasks and call Python tools (@llm.tool) for DB writes, vendor‑finder scans, cron‑style jobs, etc.
Since it's the tool agent, I need it to appear in llm models list and tried the following plugins: llm-ollama, llm-llama-cpp, llm-llama-server, llm-openai. None register the Pi endpoint as a model.

What’s the correct way to register a remote OpenAI‑compatible endpoint so that it shows up in llm models list and can be targeted for tool‑calling?

Thank you much.

jg

[gyanu2507]

You don't need those plugins for a remote OpenAI-compatible server. The built-in OpenAI plugin already covers it via a config file.

Find the config dir:

dirname "$(llm logs path)"

On Linux that's often ~/.config/io.datasette.llm. On a Mac it's usually under ~/Library/Application Support/io.datasette.llm.

Drop an extra-openai-models.yaml in there. api_base should be the Pi's /v1 root, and model_name has to match whatever /v1/models actually returns (might not be literally tinyllama):

- model_id: pi-tinyllama
  model_name: tinyllama
  api_base: "http://<pi-ip>:11435/v1"
  supports_tools: true
  can_stream: true

If the Pi wants a key:

llm keys set pi

Then add api_key_name: pi to that YAML entry. Important: once api_base is set, LLM won't send your normal OpenAI key by default.

Sanity check:

llm openai endpoint http://<pi-ip>:11435/v1 --models
llm models --options -m pi-tinyllama
llm -m pi-tinyllama 'ping'

For tools, with supports_tools: true and a server that actually implements function calling:

llm -m pi-tinyllama -T some_tool 'do the background job'

If that fails, hit the endpoint once without registering anything. That helps you tell "Pi doesn't do tools" apart from "YAML is wrong":

llm openai endpoint http://<pi-ip>:11435/v1 -m tinyllama -T some_tool '...'

Docs are here: OpenAI-compatible models and Adding more OpenAI models.