v0.6.0 - GPU inference ⚡

What's new

GPU inference via the ONNX Runtime CUDA execution provider. Parakeet can now offload transcription to an NVIDIA GPU instead of running on CPU. GPU support ships as a dedicated Docker image with the -cuda tag suffix, bundling the GPU build of ONNX Runtime and the fp32 models on a CUDA base image. CPU stays the default; nothing changes unless you ask for cuda.

How to use it

Docker (GPU enabled by default):

docker run -d \
  --name parakeet \
  --gpus all \
  -p 5092:5092 \
  ghcr.io/achetronic/parakeet:latest-cuda

Binary, or to select a specific device:

./parakeet -gpu cuda -gpu-device 0
# or, equivalently, via environment:
PARAKEET_GPU=cuda PARAKEET_GPU_DEVICE=0 ./parakeet

New CLI flags

Flag	Default	Description
-gpu	cpu	Execution provider: cpu or cuda (env: PARAKEET_GPU)
-gpu-device	0	GPU device index for cuda (env: PARAKEET_GPU_DEVICE)

An explicit flag always overrides its environment variable.

Docker

• New -cuda image at ghcr.io/achetronic/parakeet, tagged latest-cuda and per-version (v0.6.0-cuda). linux/amd64 only, since ONNX Runtime has no standard arm64 GPU build.
• New Makefile targets docker-build-cuda and docker-run-cuda.

Prerequisites

An NVIDIA GPU with up-to-date drivers and the NVIDIA Container Toolkit installed on the host.

Notes

• If the requested provider cannot be initialized (missing CUDA libraries, driver or version mismatch), the server fails at startup instead of falling back to CPU, so misconfiguration is visible immediately.
• fp32 is the precision used by the CUDA image.

Contributors

@achetronic @hadsie