Create Bluefin Ollama + WebUI Boxkit

[mecattaf]

Hey @castrojo I saw the new local LLM justfile in bluefin
https://github.com/ublue-os/bluefin/blob/6b58f9836bffb7ec0250296d2809e30e2c9325b6/just/bluefin-tools.just

I was wondering if it would make sense to use the Ollama + WebUI combo within a single distrobox based on bluefin-dx? I had a bad experience with justfiles and think that this feature would be a great use case for the Boxkit logic.

@p5 do you think that Boxkit is a better way to get ollama on ublue devices?

Many thanks!

[castrojo]

How about a dedicated container we could put in ublue-os/toolboxes with what you want in there? ublue-os/ollama or something, we could build it off of wolfi so it's nice and small and then just build it every time there's an update? If you've got time to help out I think this would be great, I don't have time to work on it for a while tho. And we could do rocm6 and cuda versions, etc too?

…

On Fri, May 3, 2024 at 12:10 PM Thomas Mecattaf ***@***.***> wrote: Hey @castrojo <https://github.com/castrojo> I saw the new local LLM justfile in bluefin https://github.com/ublue-os/bluefin/blob/6b58f9836bffb7ec0250296d2809e30e2c9325b6/just/bluefin-tools.just I was wondering if it would make sense to use the Ollama + WebUI combo within a single distrobox based on bluefin-dx? I had a bad experience with justfiles and think that this feature would be a great use case for the Boxkit logic. @p5 <https://github.com/p5> do you think that Boxkit is a better way to get ollama on ublue devices? Many thanks! — Reply to this email directly, view it on GitHub <#72>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJUT3IB3GZZYNOHP2CQCRTZAOZIZAVCNFSM6AAAAABHFYYXIGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI3TOOJZG4YTMNA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[mecattaf]

Thanks for the quick reply @castrojo. This could live in ublue-os/toolboxes. Submitted a first PR over there. I chose to build the container off bluefin-cli.

In later iterations we can add a nvidia-specific image, but for now let's do without.

Finally we could set up the systemd units by drawing inspiration from https://github.com/wayblueorg/wayblue/tree/live/config/systemd/system and the systemd module from bluebuild instead of having a justfile that writes the quadlets?