Download of cuda12-ace-step backend consistently resets mid-transfer (~53%), while all other backends download fine

[Pinucci0]

I'm running into a reproducible issue when installing the cuda12-ace-step backend (quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-ace-step). The download always progresses to roughly 53% (~2.9 GiB out of 5.4 GiB) and then abruptly resets to a few KB, restarting from scratch. This has happened consistently across multiple attempts, on different days.

All other backends I've tried download without any problem — this seems specific to cuda12-ace-step.

Log excerpt:

Sep 12 09:47:34 INFO Downloading fileName="Downloading 1/1 quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-ace-step" current="2.9 GiB" total="5.4 GiB" percentage=53.59620969745899 eta=40m13.118203712s
Sep 12 09:47:47 INFO Downloading fileName="Downloading 1/1 quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-ace-step" current="3.3 KiB" total="5.4 GiB" percentage=5.819562581902235e-05 eta=1336886h49m59.649681408s

Is anyone else seeing this specifically with cuda12-ace-step, or is it isolated to my setup/network?
Could this be related to a specific layer/blob on the quay.io registry for this image being unusually large or misbehaving with resumable downloads (Range header support)?

Thanks!

[helloe365]

The fact that it always dies at the same point (~2.9 GiB) suggests one specific layer in that image, and the behavior after the reset is the tell: LocalAI's image puller restarts the layer from scratch — it doesn't resume from the last received byte with a Range request — so any connection hiccup means starting the whole ~2.6 GiB blob over. Your other backends are smaller and finish before the connection has a chance to drop, which is why they look fine while this one never completes. A ~40 minute ETA just gives the flaky link more chances to reset.

Two ways around it:

1. Pull the image with Docker instead — docker pull quay.io/go-skynet/local-ai-backends:latest-gpu-nvidia-cuda-12-ace-step. Docker retries layers and validates digests, so it survives resets much better. Then extract the backend binary (docker create + docker cp) and register it directly with EXTERNAL_GRPC_BACKENDS="ace-step:/path/to/backend", or the --external-grpc-backends name:path flag — that skips LocalAI's downloader entirely for this backend.

2. If you just want music generation working, take a look at the acestep-cpp backend instead — it's the C++/GGML implementation of ACE-Step 1.5 and is a much smaller image than the CUDA Python one you're pulling (see the compatibility table in the docs), so the download problem mostly evaporates.

Worth reporting either way — the logs showing the reset would make a decent feature request for resumable pulls, since "large backend on a flaky link can never finish" will bite other people too.

If this gets the install through, marking the answer would help the next person fighting the same blob.

[Pinucci0]

Thank you for the answer!