Pulling models resets the download on raspberry pi 5

[nexon33]

What is the issue?

The raspberry pi 5 is connected over wifi but this shouldn't be a problem since my other device pulls the models correctly on the same wifi.

Suggested fix: I currently have to constantly cancel the downloads with CTRL + C, this saves the download state and then continues where it left off, functioning as some kind of save/checkpoint. Maybe its possible to backup the download state once in a while so that it doesn't stop downloading on a crash?

Feel free to request more info or let me know if you want me to test!

Edit: sometimes the error "Error: context canceled" shows up and I think it might be related

OS

Linux

GPU

Other

CPU

Other

Ollama version

0.5.4

[rick-github]

It's not clear to me what problem you are having. If a download (ollama pull or ollama run) is interrupted, it should restart from the point of interruption the next time you run ollama pull. The exception is if you reboot the server - when it starts, the server does some housecleaning which includes clearing failed downloads. So if you interrupt a download but want to continue it later, don't restart the server.

[nexon33]

The problem is that the state is saved only when interrupted manually, not when it crashes during the download.

[rick-github]

Pulling a model shouldn't crash the server. Can you provide more info?

[rick-github]

Server logs might help.

[nexon33]

The server is not crashing but there are errors in the backend, I will try once more when I get home this evening

[cjuerg67]

I have the same issue on a RPi4 (4 GB), connected via LAN. The download (after run or pull) gets to about 10%, then stops, and after some seconds starts again with 0%. Looking now how to copy (via WinSCP) the same model from the Windows PC, where it works flawless.

[ACertainArchangel]

This control C thing you came up with is brilliant! Exactly what I needed. Thanks.

[rick-github]

The "control C thing" is a fine workaround but doesn't fix the problem. for which server logs might be helpful.

[cjuerg67]

Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.299Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 15 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connect>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.300Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 4 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connecti>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.300Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 13 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connect>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.299Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 0 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connecti>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.300Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 2 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connecti>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.300Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 14 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connect>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.301Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 12 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connect>
Jan 15 08:24:10 raspihailo1 ollama[1083]: time=2025-01-15T08:24:10.301Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 10 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connect>
Jan 15 08:24:22 raspihailo1 ollama[1083]: [GIN] 2025/01/15 - 08:24:22 | 200 |          4m7s |       127.0.0.1 | POST     "/api/pull"

This repeats again and again. What else could help to solve the problem?

[cjuerg67]

Jan 15 08:38:42 raspihailo1 ollama[3741]: [GIN-debug] HEAD   /api/tags                 --> github.com/ollama/ollama/server.(*Server).ListHandler-fm (5 handlers)
Jan 15 08:38:42 raspihailo1 ollama[3741]: [GIN-debug] HEAD   /api/version              --> github.com/ollama/ollama/server.(*Server).GenerateRoutes.func2 (5 handlers)
Jan 15 08:38:42 raspihailo1 ollama[3741]: time=2025-01-15T08:38:42.736Z level=INFO source=routes.go:1310 msg="Listening on 127.0.0.1:11434 (version 0.5.4)"
Jan 15 08:38:42 raspihailo1 ollama[3741]: time=2025-01-15T08:38:42.737Z level=INFO source=routes.go:1339 msg="Dynamic LLM libraries" runners="[cpu cuda_v11 cuda_v12]"
Jan 15 08:38:42 raspihailo1 ollama[3741]: time=2025-01-15T08:38:42.737Z level=INFO source=gpu.go:226 msg="looking for compatible GPUs"
Jan 15 08:38:42 raspihailo1 ollama[3741]: time=2025-01-15T08:38:42.757Z level=INFO source=gpu.go:392 msg="no compatible GPUs were discovered"
Jan 15 08:38:42 raspihailo1 ollama[3741]: time=2025-01-15T08:38:42.758Z level=INFO source=types.go:131 msg="inference compute" id=0 library=cpu variant="no vector extensions" compute="" driver=0.0 name="" total="3.7 GiB" available="3.2 GiB"
Jan 15 08:38:58 raspihailo1 ollama[3741]: [GIN] 2025/01/15 - 08:38:58 | 200 |     335.162µs |       127.0.0.1 | HEAD     "/"
Jan 15 08:38:58 raspihailo1 ollama[3741]: time=2025-01-15T08:38:58.941Z level=INFO source=download.go:175 msg="downloading dde5aa3fc5ff in 16 126 MB part(s)"
Jan 15 08:39:19 raspihailo1 ollama[3741]: time=2025-01-15T08:39:19.068Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 4 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."
Jan 15 08:39:19 raspihailo1 ollama[3741]: time=2025-01-15T08:39:19.068Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 1 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."
Jan 15 08:39:19 raspihailo1 ollama[3741]: time=2025-01-15T08:39:19.069Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 8 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."
Jan 15 08:39:19 raspihailo1 ollama[3741]: time=2025-01-15T08:39:19.068Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 15 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."
Jan 15 08:39:19 raspihailo1 ollama[3741]: time=2025-01-15T08:39:19.070Z level=INFO source=download.go:370 msg="dde5aa3fc5ff part 9 stalled; retrying. If this persists, press ctrl-c to exit, then 'ollama pull' to find a faster connection."

Switched on debug mode and restarted ollama, but I guess no new information...

[rick-github]

Background: #8330 (comment)

What's the load on the pi5 during download? Can you get a measure of bandwidth saturation on the network link? If other machines on your LAN can download models, it could be that the pi5 is choking it's own download either due to network congestion, CPU overload or disk bandwidth. The parellelization of the download might just be too much. Unfortunately there are currently no knobs to control the download, the only workarounds at the moment are the "control C thing" or copying the models off another machine.

[rick-github]

Something else to try: downgrade to an older version and try the pull. I haven't checked, but something may have changed in the download code to make it more sensitive to stalls on low power machines.

curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.4.0 sh
ollama pull llama3.1
curl -fsSL https://ollama.com/install.sh | OLLAMA_VERSION=0.5.5 sh

[rick-github]

Are specific models causing a problem?

[cjuerg67]

Same problem with qwen:0.5b.
I guess it's related to the slow SD card writing. As I said, I'm using a pi4. Today I get a pi5 with nvme SSD :-)
Now that llama3.2 works, I see that I need a more powerful system...

And using ctrl-c is really a workaround. But you have to use it immediately after observing a decreasing download rate, otherwise you fall back to the last successful point - which is 0% at the beginning. After a ctr-c I wait till the green LED turns off (as it seems to show sd-card access?), and then start the pull again.
In that way I was able to download llama3.2:3b with about 13x ctrl-c.