asr-transcribe

1. Native installation and usage (for CUDA)

Requirements

• Python >= 3.10
• UV (tested)
• ffmpeg
• Cairo (Legacy. Probably not needed anymore)
• If CUDA is available (depending on you GPU): CUDA Toolkit
• direnv --> You need this to fix the "missing libcudnn_ops_infer.so.8_fix" bug, see details

1. Clone repository

2. Install dependencies:

uv sync

3. You need to reinstall llama-cpp-python with CUDA support

The default PyPI wheel is CPU-only. Rebuild from source with CUDA enabled:

CMAKE_ARGS="-DGGML_CUDA=on" uv sync --reinstall-package llama-cpp-python

4. Usefull shell scripts for troubleshooting

4.1. "missing libcudnn_ops_infer.so.8_fix" bug

sh ./help/setup_cudnn.sh

direnv allow # direnv needs to be installed and set up.

4.2. "Weights_only" bug (fix probably not needed anymore if using WhisperX =< 3.8.5)

sh ./help/patch_lightning_fabric.sh

5. Create the configuration file.

cp config.example.toml config.toml

6. Usage

Run the workflow script.

uv run asr_workflow.py

2. Alternative: Docker installation (experimental)

Note: Works only with Cuda 13.1 or higher on your system

Requirements

• Docker, e.g. Docker Desktop
• NVIDIA Container Toolkit

1. Clone repository

2. Build Docker image

docker compose build

3. Set paths in config.toml

In the config.toml set the container paths for the input and output paths as well as the model paths

[system]
input_path = "/app/data/_input"
output_path = "/app/data/_output"

[llm]
model_path = "/app/models/your-model.gguf"

The container paths are mapped to actual paths as following:

• "/app/data/_input" --> "./data/_input"
• "/app/data/_output" --> "./data/_output"
• "/app/models" --> ./models

4. Run container for single transcription job

docker-compose run --rm asr-transcribe

Configuration

The config.toml file is used to configure the application. You can copy the config.example.toml to create your own config.toml.

System Options ([system])

• input_path / output_path: Source and destination folders the workflow watches and populates.
• email_notifications: Enables success/failure/warning emails via the settings in [email].
• zip_bags: When true, each generated bag directory is also written as a .zip archive in the same output folder.

Whisper Options ([whisper])

• model / device / compute_type / beam_size / batch_size: Core WhisperX transcription settings.
• language: Force a language or omit the key for auto-detection (remove the entry entirely to let Whisper detect automatically).
• translation_enabled / translation_target_language: Toggle Whisper’s translate task and pick the output language (defaults to English). Keep this disabled for pure transcription.
• translation_model: When translation is enabled and the configured model is not multilingual (e.g., the turbo variants), this fallback model is loaded automatically. By default the workflow uses large-v3, which yields the best translation quality.
• use_speaker_diarization, min_speakers, max_speakers: Control diarization; when enabled you must supply hf_token so WhisperX can download the diarization model from Hugging Face.
• pause_marker_threshold: Minimum gap in seconds before inserting pause markers into speaker-aware exports (e.g., _speaker.csv, MAXQDA variants); defaults to 2.0s.
• use_initial_prompt, initial_prompt, max_sentence_length: Fine-tune segmentation and prompt injection.
• no_repeat_ngram_size / repetition_penalty: Anti-hallucination guards against repetition loops ("äh äh äh…"), applied only to external/fine-tuned models loaded by a filesystem path (ignored for built-in names like large-v3). no_repeat_ngram_size is the primary guard and defaults to 10 (on for external models): a hard cap that breaks runaway loops while leaving genuine speech untouched and not garbling repeated compounds. repetition_penalty is an optional soft penalty, off by default (1.0) — being an always-on global bias it also suppresses genuine repeated interjections (äh/ähm), so prefer no_repeat_ngram_size. Set 0 / 1.0 to disable.

Email Options ([email])

• smtp_server / smtp_port / username / password: SMTP host and credentials used for notifications.
• from / to: Sender and recipient list for success, warning, and failure emails triggered by the workflow.

LLM Options ([llm])

• use_summarization: Enables LLM-generated summaries after transcription finishes.
• use_toc: Enables LLM-generated table of contents with hierarchical chapter markers (H1/H2/H3).
• model_path: Filesystem path to a llama-cpp-compatible GGUF model (e.g., stored under models/).
• n_gpu_layers: GPU offloading depth for llama-cpp; adjust based on your hardware. The model is loaded inside llm_subprocess.py, so no additional services are required.
• llm_languages: List of language codes (e.g., ["de", "en"]) for summaries and TOC. Remove or limit entries to skip specific languages.
• verbose: Enables verbose output from llama-cpp; useful for debugging model loading issues.
• debug_file: Path to a JSON file with WhisperX segments for testing LLM workflows without running transcription.
• output_debug: Output directory for debug runs via python -m llm_workflows.llm_debug.

BagIt Options ([bag])

The application uses the BagIt specification to package the output files. The following options are available to add metadata to the bag-info.txt file.

• group_identifier: A persistent, globally unique identifier for a logical set of bags.
• bag_count: The number of bags in a set (e.g., "1 of 3").
• internal_sender_identifier: An identifier for the creator of the bag.
• internal_sender_description: A description of the creator of the bag.

Summaries

If llm.use_summarization is enabled, the workflow runs an LLM subprocess that produces per-language summaries according to llm.summary_languages.

For example, with the default ["de", "en"] configuration:

• _summary_de.txt contains the German abstract.
• _summary_en.txt contains the English abstract.

Both files are stored inside each bag's data/content_extraction/ directory. The prompts favour concise, third-person prose and silently correct minor ASR issues. When the LLM step fails, transcription continues without summaries.

Table of Contents (experimental)

If llm.use_toc is enabled, the workflow generates a structured table of contents from the transcript. The LLM analyses topic shifts and creates a hierarchical outline with timestamps.

For example, with the default ["de", "en"] configuration:

• _toc_de.vtt contains the German table of contents.
• _toc_en.vtt contains the English table of contents.

The output is a VTT file with chapter markers (H1/H2/H3 levels) that can be used for navigation in media players. Each entry includes a title and precise time range derived from the transcript. When the LLM step fails, transcription continues without the table of contents.

Output

For every processed file a timestamped bag directory is created under the configured output path. Each bag contains:

• data/transcripts/: All transcript formats (TXT, RTF, CSV, VTT, SRT, JSON, ODT, PDF/A, etc.), including speaker CSVs with pause markers and a _speaker_nopause.csv variant without pause markers.
• data/translations/: Mirrors the transcript formats but contains the translated text when translation is enabled.
• data/abstracts/: Language-specific summaries (currently _summary_de.txt and _summary_en.txt).
• data/ohd_import/: Copies of the speaker CSV exports (with and without pause markers) for downstream ingestion.
• documentation/: Reference material copied from doc_files/ (export formats, citation text, upload instructions).
• bagit.txt, bag-info.txt, manifest-sha512.txt, tagmanifest-sha512.txt: Files required by the BagIt specification.

If zip_bags is true, the complete bag directory is additionally written as <bag-name>.zip alongside the folder.

Tests

• Run automated tests with pytest.

pytest

Additional information about features?

• Avoids misinterpreting titles and dates as sentence endings.
• Merges segments that do not have punctuation with the following segments.
• Customisable segment splitting: Splits segments longer than specified number of characters at the next comma (default = 120 characters).
• Configurable pause markers: Inserts <pN> pause tags in speaker-aware exports when word-level gaps exceed pause_marker_threshold (default 2s).
• Changes the lowercase letter of the first word of the segment to an uppercase letter (only in cases where the previous segment ends without a comma).
• Enables word lists for names, special terms or filler words (initial prompt).
• Embeds the code into an automated input/output folder workflow.
• Calculates audio duration, workflow duration and real time factor for transcribing an audio file.
• To speed up processing, it is possible to change the calculation type to "int8" and the beam size to 4 or less (default = 5), but there is a risk of quality loss.
• Sends an success or failed email if the workflow is (not) completed successfully.
• Captures the stdout/terminal output and sends an email if the word "failed" is found in the output.
• Automatically summarizes transcripts via LLMs