GPU and CUDA

GPU & CUDA

TypeWhisper ships optional NVIDIA CUDA acceleration for its bundled local transcription engines. The GPU runtime is fetched on demand the first time you select CUDA — nothing GPU-related ships inside the packages — and cached locally. This page covers what it requires, which engines it applies to, and how to switch backends.

Requirements

To use the CUDA backend you need just one thing:

• An NVIDIA GPU and driver — this is the only hard requirement.

The CUDA 12 runtime libraries the engines need (libcudart.so.12, libcublas.so.12, cuDNN, and friends) are downloaded on demand the first time you select NVIDIA CUDA, verified by SHA-256, and cached under ~/.local/share/TypeWhisper/Runtimes/. Only the libraries you are actually missing are downloaded, so a host with a partial CUDA install fetches just the gaps and a host that already has a full CUDA toolkit on its library path downloads nothing. You do not need to install a CUDA toolkit yourself, and nothing GPU-related is bundled into the packages.

If an NVIDIA GPU is present but the runtime isn't cached yet, the Dictation section shows a Download CUDA runtime button (see below). On a host with no NVIDIA GPU, CUDA is unavailable and the engine stays on CPU — see Troubleshooting.

Choosing the backend

The Dictation section has an acceleration selector with three choices:

Option	Behavior
Auto	Use CUDA when available, otherwise CPU.
CPU	Force CPU.
NVIDIA CUDA	Force the CUDA backend. Selectable whenever an NVIDIA GPU is present; if the runtime libraries aren't cached yet, TypeWhisper offers to download them.

Restart to switch

The process is pinned to the first backend it loads. Switching between CPU and CUDA therefore takes effect only after an app restart: when the active backend differs from your selection, the status text reads "Restart TypeWhisper to apply."

Download CUDA runtime

When an NVIDIA GPU is present but the CUDA runtime isn't cached yet, the Dictation page shows a Download CUDA runtime button. Clicking it fetches only the libraries TypeWhisper is missing — with a download-progress indicator — into ~/.local/share/TypeWhisper/Runtimes/, then asks you to restart TypeWhisper to load CUDA. The download only starts on a host that actually has an NVIDIA GPU, so a GPU-less machine never pulls the large runtime.

Fix CUDA path

When the CUDA 12 libraries already exist on disk (for example a system toolkit) but aren't visible to the running process, TypeWhisper shows a Fix CUDA path button instead. Clicking it preloads libcudart.so.12 and libcublas.so.12 into the process so the engine can find them without LD_LIBRARY_PATH. Restart TypeWhisper afterward to load CUDA.

Scope

CUDA applies to the bundled whisper.cpp and sherpa-onnx (Parakeet, Canary) engines. Other bundled local engines stay on CPU.

Related pages

• Transcription engines — the engines available, including bundled whisper.cpp.
• Dictation — the acceleration selector and model settings.
• Requirements — full list of Linux requirements and optional helpers.

---

Changelog

Date	Change
2026-06-17	Initial version.
2026-06-17	Added acceleration selector, restart-to-switch behavior, Fix CUDA path button, and clarified only two libs are required.
2026-06-23	v0.10.0: CUDA runtime is now fetched on demand (no longer bundled) and cached under Runtimes/; added the Download CUDA runtime action; extended CUDA to the sherpa-onnx engine (Parakeet, Canary).