Skip to content

Concurrency and Progress

aivrar edited this page Nov 18, 2026 · 1 revision

Concurrency & Progress

One LocalSoundsAPI instance can run different engines at the same time, for example Kokoro narrating while Stable Audio renders a thunderstorm on another GPU, and every panel shows its own live progress.

Kokoro and Stable Audio generating side by side

The rules

Situation What happens
Two jobs on different engines Both run, if there's VRAM
A second job on the same engine Refused right away: 503 Busy + Retry-After: 5
Load / Unload an engine that's busy Refused: 503 Busy
Unload / move Whisper while a job verifies with it Refused: 503 ("busy_with": "whisper")
Two engines writing to the same save path The second gets 409
Stop Cancels only the job you started (499 Cancelled)

There's no queue: a refused request isn't stored for later. The web UI tells you what's busy; scripts decide whether to wait and retry, use another engine, or use another instance.

Shared models are protected. Whisper (used for verification and subtitles) can't be unloaded or moved while any job relies on it. CLAP (used by both music engines) stays loaded until neither needs it.

Strict mode for small GPUs

To allow only one job at a time across all engines, set:

# config.py
JOB_CONCURRENCY = "single"

or start the app with the environment variable LSAPI_JOB_CONCURRENCY=single. In this mode, the other panels' Generate buttons are disabled while any job runs.

Spreading engines over GPUs

Every panel has its own device selector, so you can place models where they fit. Example from a two-GPU machine, with all engines loaded at once:

GPU Engines VRAM used
RTX 3090 (24 GB) Stable Audio + ACE-Step + CLAP ≈ 12 GB
RTX 3060 (12 GB) XTTS + Fish Speech + Kokoro + Whisper ≈ 7 GB

Kokoro and Whisper also run on cpu.

What the progress bar shows

Stage Meaning
Preparing Loading the model, splitting the text
Generating Real percentage: Chunk 3/9 (speech), Diffusion step 42/100 (Stable Audio), variant progress (ACE-Step)
Cleaning up Joining chunks, post-processing, saving

Plus elapsed time and an ETA based on progress so far. While a panel's engine is busy its controls are locked, except Stop. Jobs started from another browser tab or by a script appear as started elsewhere, with Stop disabled.

For scripts: /job_status

GET /job_status returns every running job:

{
  "mode": "per_engine",
  "busy": true,
  "jobs": [
    {"engine": "kokoro", "kind": "generate", "stage": "generating",
     "message": "Chunk 3/9", "current": 2, "total": 9, "unit": "chunk",
     "percent": 22.2, "elapsed_sec": 14.1, "eta_sec": 49.0}
  ]
}

A busy reply looks like:

HTTP 503   Retry-After: 5
{"error": "busy", "busy_with": "kokoro", "mode": "per_engine",
 "message": "...", "retry_after_sec": 5, "job": { ...the running job... }}

A simple retry loop in Python:

import time, requests

def post(url, payload):
    while True:
        r = requests.post(url, json=payload, timeout=3600)
        if r.status_code != 503:
            return r
        time.sleep(int(r.headers.get("Retry-After", 5)))

[stand-alone-app]-API_client.py in the repo uses this pattern. To cancel only your own job, send any id in an X-Job-Client-Id header with both the generate and the cancel request. See API Reference.

Clone this wiki locally