OpenAI-compatible transcription API backed by nvidia/parakeet-tdt-0.6b-v3.
The service exposes POST /v1/audio/transcriptions so existing Whisper/OpenAI speech-to-text clients can be pointed at a local CUDA-backed container instead of OpenAI's hosted endpoint.
The dependency workflow uses uv, and the container installs official nemo_toolkit[asr] on top of CUDA with PyTorch CUDA 12.8 wheels.
POST /v1/audio/transcriptionsGET /v1/modelsGET /healthz- OpenAI-style request fields:
filemodellanguageresponse_formatstreamtimestamp_granularities[]include[]=logprobstemperatureis accepted for compatibilitypromptis accepted by clients but ignored by Parakeet
- Response formats:
jsontextverbose_jsonsrtvtt
- Minimal SSE mode for
stream=true - Long-form audio chunking with
ffmpegbefore inference - Native NeMo local attention enabled by default with
[128,128] - Optional bearer auth via
PARAKEET_API_KEY
OpenAI's transcription API does not expose VAD as a public request parameter. If we add VAD later, it should remain an internal server-side chunking optimization, not part of the external Whisper/OpenAI-compatible contract.
The runtime enables native NeMo local attention by default:
asr_model.change_attention_model(
self_attention_model="rel_pos_local_attn",
att_context_size=[128, 128],
)That matches the long-form inference pattern documented on the official Parakeet v3 model card and the NeMo ASR docs, and matches your stated preference for the best speed/accuracy tradeoff.
diarized_jsonis not implemented.gpt-4o-transcribe-diarizeand speaker-reference fields are rejected.include[]=logprobsreturns an empty array because Parakeet/NeMo does not expose OpenAI-style token logprobs.languageis treated as a compatibility hint and validation check. Parakeet still auto-detects internally./v1/audio/translationsis intentionally unsupported because this model transcribes in the source language.
Build:
docker build -t parakeet-api .Run with NVIDIA Container Toolkit:
docker run --rm \
--gpus all \
-p 8000:8000 \
-e PARAKEET_API_KEY=local-dev-key \
-v parakeet-model-cache:/models \
parakeet-apiOr with Compose:
docker compose up --builduv sync
uv run python -m unittest discover -s tests -v
uv run uvicorn parakeet_api.main:app --reloadPARAKEET_MODEL_NAME- Default:
nvidia/parakeet-tdt-0.6b-v3
- Default:
PARAKEET_MODEL_ALIASES- Default:
whisper-1,gpt-4o-transcribe,gpt-4o-mini-transcribe,parakeet-tdt-0.6b-v3,nvidia/parakeet-tdt-0.6b-v3
- Default:
PARAKEET_API_KEY- Optional bearer token required for all endpoints except
/healthz
- Optional bearer token required for all endpoints except
PARAKEET_PORT- Default:
8000
- Default:
PARAKEET_CHUNK_DURATION_SECONDS- Default:
900 - Local attention stays enabled regardless. This chunking limit is still a server-side guardrail around NeMo's
transcribe()path, which is more reliable on shorter fragments than on single very long files.
- Default:
PARAKEET_MAX_CONCURRENT_REQUESTS- Default:
1
- Default:
PARAKEET_ALLOW_CPU_FALLBACK- Default:
true
- Default:
PARAKEET_USE_LOCAL_ATTENTION- Default:
true
- Default:
PARAKEET_ATTENTION_MODEL- Default:
rel_pos_local_attn
- Default:
PARAKEET_ATT_CONTEXT_SIZE- Default:
128,128
- Default:
PARAKEET_UPLOAD_DIR- Default:
/tmp/parakeet-api/uploads
- Default:
If you want to rely more heavily on NeMo local attention instead of server-side chunking for long files, set PARAKEET_CHUNK_DURATION_SECONDS=0.
curl http://localhost:8000/v1/audio/transcriptions \
-H "Authorization: Bearer local-dev-key" \
-F file=@sample.wav \
-F model=whisper-1 \
-F response_format=verbose_json \
-F "timestamp_granularities[]=word"Point the client at this API instead of OpenAI:
from openai import OpenAI
client = OpenAI(
api_key="local-dev-key",
base_url="http://localhost:8000/v1",
)
with open("sample.wav", "rb") as audio_file:
transcript = client.audio.transcriptions.create(
model="whisper-1",
file=audio_file,
response_format="verbose_json",
)
print(transcript)This project is intentionally smaller than tulas75/parakeet-api. The older repo has accumulated a large amount of runtime tuning and heuristics in one Flask file. This version keeps the compatibility surface focused on the current OpenAI transcription endpoint shape, isolates the model runtime, and uses a CUDA-first FastAPI container that is easier to reason about and extend.
- OpenAI transcription reference:
- NVIDIA Parakeet model card:
- NVIDIA NeMo ASR docs:
- NVIDIA NeMo software component versions:
- uv docs:
- Package releases: