Prebuilt `ollama-linux-amd64.tgz` without cuda libs, please?

[sevaseva]

I occasionally update ollama on a linux box by downloading URLs like https://github.com/ollama/ollama/releases/download/v0.3.7-rc6/ollama-linux-amd64.tgz and extracting/overwriting files into a local directory (not into /usr as a root mind you, just into a local directory as a non-privileged user; that is how I prefer to use it).

I have necessary cuda libs installed in the system.

I don't care to use the libs distributed with ollama to begin with (and if bin/ollama defaults to searching libs in ../lib first I don't love that but that's fine).

But I certainly don't care to download the same 1GB of libs every time I update.

(I wonder how many users are like me).

I can haz a version of linux-amd64 without cuda libs included in https://github.com/ollama/ollama/releases prebuilt assets?

...or should I instead just git pull and build the binary from source whenever I want to update (which would be fine with me) or what would you guys recommend?

[robert-mcdermott]

I have a similar issue. I run Ollama as a non-privileged user on an HPC cluster node. I need to run Ollama in my home folder and don't have access to add the library files to /usr/lib.

[kth8]

I have been using this one liner to extract the latest ollama binary into my local bin folder:

curl -L https://github.com/ollama/ollama/releases/download/$(curl -s https://api.github.com/repos/ollama/ollama/releases/latest | jq -r .tag_name)/ollama-linux-amd64.tgz | tar -xz --strip 2 ./bin/ollama -C ~/.local/bin 

[dhiltgen]

We're continuing to refine how we package our official builds. PR #6547 will bring some additional improvements to the container image to optimize CUDA and ROCm sizes and startup times for container users. Due to the size of ROCm, we've had to split the dependencies out in a separate tar file as combined it exceeds github's artifact size limit. It's possible we could consider splitting the cuda dependencies out in a similar fashion and then add smarts into the install script to make sure it grabs the correct bundles for the GPUs we detect.

[lowlyocean]

+1, there are such frequent releases that it's a bit of a waste of bandwidth to be re-downloading CUDA so often (especially if the machine already has CUDA installed system-wide). Have APT / YUM package management (with proper dependency management of shared libs) been decided against?