App on local machine, but models running on remote machine?

[iandouglas]

I have a few AMD Strix Halo systems, and soon to have an AMD Gorgon Halo system arriving, all running ROCm.

Can I run the app locally on my Mac (with limited RAM) and have that access the models running on my Halo systems? I could set them up with llama.cpp, vllm, lemonade, etc, and access it over an API.

Has that already been done, or is there a guide on how to do this? My Halo systems are all server-only, no GUI, and my Mac doesn't have enough RAM to do everything else I need to do and also load these models into memory.

[rudycelekli]

Yes — the current docs cover running the VoiceStudio backend on the GPU machine and using the Mac app as the client: remote backend guide.

On the Mac, the setting is Settings → Sharing → Remote backend. It takes the VoiceStudio service URL and key. A llama.cpp/vLLM chat endpoint is a separate LLM-provider configuration; the desktop's backend setting expects the VoiceStudio API, including its voice and project endpoints.

For the Halo boxes, I'd verify the chosen voice engine works with your ROCm setup separately — connecting the Mac doesn't establish GPU compatibility. The Linux guide documents the ROCm option. The remote-backend path moves the projects/backend to the server; remote workers are the option to investigate if you want projects to stay on the Mac and offload individual jobs.