TranscribeJob

TranscribeJob

Speech to text with Whisper-family models: plain text, or text with timed segments.

dib.TranscribeJob(
    model,               # spec or URI
    audio_col,           # column with a path (str) or the file's bytes
    language=None,       # e.g. "en", "pl"; None = auto-detect
    prompt=None,         # initial prompt: vocabulary, names, style
    timestamps=False,    # True -> {"text", "language", "segments": [...]}
    translate=False,     # Whisper's translate-to-English task
    engine="auto",
    engine_options={},
)

Audio input

Each cell of audio_col is one of:

• a path: local, /Volumes/... (FUSE-mounted on Databricks workers) or dbfs:/... (read via /dbfs/...);
• bytes: e.g. the content column of spark.read.format("binaryFile").

Null, empty, or other types give row errors (InvalidInputError). So do files that can't be read or decoded.

audio = spark.read.format("binaryFile").option("pathGlobFilter", "*.wav").load("/Volumes/main/raw/calls/")
job = dib.TranscribeJob(model=..., audio_col="content")          # bytes
job = dib.TranscribeJob(model=..., audio_col="path")             # or let workers read paths

Paths keep shuffled data small, while bytes avoid a second read from storage. For long recordings prefer paths with binaryFile and select only path.

Output

• timestamps=False: the transcript as a string.
• timestamps=True: {"text": str, "language": str | None, "segments": [{"start": float, "end": float, "text": str}]} (a struct on Spark).

<out>_meta.audio_seconds holds the audio duration, so dib.summarize() reports audio hours.

Engines

• faster_whisper (default when installed): CTranslate2 Whisper, the fastest on GPU and CPU. Hugging Face Whisper weights, e.g. from Unity Catalog, are converted once per node. Use engine_options for compute_type, beam_size, vad_filter, batched and num_workers.
• transformers: the ASR pipeline. Audio longer than chunk_length_s (30 s) is transcribed in chunks.
• openai: audio/transcriptions (or audio/translations). Files are uploaded as they are. Bytes get a file name from their format signature. Files larger than max_upload_mb (25) are row errors.

Example

asr = dib.TranscribeJob(
    model=dib.models.uc("main.audio.whisper_large_v3", alias="prod", stage_dir="/Volumes/main/audio/stage"),
    audio_col="path",
    language="pl",
    prompt="Rozmowa z infolinią banku.",
    timestamps=True,
    engine_options={"compute_type": "float16", "vad_filter": True},
)
calls = spark.read.format("binaryFile").load("/Volumes/main/raw/calls/").select("path")
out = asr.run(calls, backend="spark", batch_size=4, checkpoint="/Volumes/main/audio/ckpt/calls", id_col="path")