Add Models Endpoint

[chand1012]

Adds /models endpoint to make the API compatible with some UIs that use the endpoint to get a list of usable models. An example that I've been using heavily is Open WebUI.

[thadius83]

Silly question, how do we get this to work with Open WebUI..

Have defined the api endpoint, I can see the request in cloudflare dashboard logs.

I've attempted to create a model file within Open WebUI that references a model and such but end up with a result code of 400 in the WebUI interface, and code 200 within the Cloudflare dashboard

I've verified via CLI and postman that a request to the v1/models works when setting the bearer access token

I'm assuming I'm doing something wrong in regards to the webui model integration as I can't seem to find a method to include the bearer token?

[chand1012]

@thadius83 I wrote a guide on how to add external models with Open WebUI, see here. Scroll down to "How To Add External AIs".

[thadius83]

@thadius83 I wrote a guide on how to add external models with Open WebUI, see here. Scroll down to "How To Add External AIs".

That's actually how I how found your project. Brilliant write up.

The issue I'm facing, and I'm pretty sure it's something I'm doing wrong.

via python on the cli, and postman, i can pull the model list down.

When adding it into Open WebUI via your instructions, I assume I need to create a model and reference it as below?

From the cloudflare log interface

Which appears similar to the output from python/postman when doing a GET on /models

I believe it's just one step or understanding that I'm missing?

[chand1012]

You can add your model by going to Settings -> Connections then adding them like so.

However this will not persist between restarts. If you want that you should it via environment variables as described here in the Open WebUI docs and here in my original article.

docker run -d -p 3000:8080 \
  -v open-webui:/app/backend/data \
  -e OPENAI_API_BASE_URLS="https://api.openai.com/v1;https://api.groq.com/openai/v1;https://openai-cf.yourusername.workers.dev/v1" \
  -e OPENAI_API_KEYS="sk-proj-ABCDEFGHIJK1234567890abcdef;gsk_1234567890abcdefabcdefghij;0123456789abcdef0123456789abcdef" \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

If you already have the container running you'll have to run docker stop open-webui before you can continue. If you aren't using Groq or OpenAI you can simply remove them from the above command.

[thadius83]

Sigh. I'm an idiot.

I'd been having issues with the env.CLOUDFLARE_API_TOKEN

I'd modified it as such, and was attempting to add it via the model option rather than refreshing and checking my model list..

All there /facepalm

Thanks mate.