how to load a model from tray UI?

[mwmcode]

0.25.0 • llama.cpp b8088

llamabarn.mov

Before, I was able to load a model by clicking it (its icon). Now, clicking a model only opens up context length list. Doesn't seem to load the model even when a context length is clicked.

[mwmcode]

can only do it via web UI

[mwmcode]

but it doesn't answer how to load a model from tray icon UI?

[mdstaff]

Ah, true. What you expect to happen after you load it from the UI aside from the icon turning blue? Are you considering using the newly loaded model from the web ui or via the llama server api?

[mdstaff]

I just took a peek and it doesn't seem like loading the model via the tray is supported right now:
https://github.com/ggml-org/LlamaBarn/blob/main/LlamaBarn/Menu/ModelItemView.swift#L171

We might need to repurpose this post to discuss how "loading via the tray" should work considering how the UI currently allows you to change the context size via the Model row dropdown.
...
That said, I think it would be easy to use the existing loadModel method or use the Llama Server API directly:
https://github.com/ggml-org/LlamaBarn/blob/main/LlamaBarn/System/LlamaServerAPI.swift#L18

[mwmcode]

but that defeats the purpose, no? It's a menu bar app after all. "LlamaBarn is a macOS menu bar app for running local LLMs."

It used to work fine. you click the tray icon, you get a list of supported models and all you needed to do is just a click to download/load the desired model.

a recent updated took away (broke?) this feature

From there you'd either use the built-in web server (chat UI) to interact with the loaded model or use a 3rd party app that is openAI complaint.

but even this doesn't work like before. It used to auto-select the loaded model but now you have to select it manually.

If there is a model that has been loaded already, doesn't it make sense to auto-select it? (again, it used to before).

[mdstaff]

@mwmcode Download still works 😅 I'd bet it's a quick fix to enable loading in the tray, idk about the web ui stuff I think that's a separate system baked into Llama.cpp.

If it was possible to load a model from the tray maybe we should file a bug ticket?