interviewr-ai: a Linux interview and meeting copilot using whisper.cpp on Vulkan for live transcription

[PSYEONE]

Hi all, I've been building an open source interview and meeting copilot for Linux and whisper.cpp is the core of it, so I wanted to share it here along with a few things I learned that might help others.

It captures system audio through PipeWire, transcribes it live with whisper.cpp (through pywhispercpp) on the Vulkan backend, and when you press a hotkey it answers the last question with a local model through Ollama, using your CV and the job description.

How the live transcription works

• A simple RMS gate decides what's speech. 0.7 s of silence ends an utterance, and long speech gets flushed at 15 s.
• While someone is talking it re-transcribes the buffer every 0.4 s and only commits words once two passes in a row agree on them, holding back the newest few words since whisper often revises those.
• Trailing silence gets trimmed to 0.2 s before each pass, because long silent tails were where whisper made up text like "thank you".
• It runs with no_context and suppress_nst, strips [BLANK_AUDIO] style notes, and collapses runs of repeated words.
• small.en is the default after running an eval on LibriSpeech test-clean.

Things that bit me, in case they help someone

• PyPI's cmake is now 4.x, and pip's isolated build for pywhispercpp picks it up over the system one. The bundled CMakeLists then fails with a misleading "Could NOT find Vulkan". Pinning cmake<4 through PIP_CONSTRAINT fixed it.
• pywhispercpp's setup.py passes environment variables through to cmake, so an unrelated PREFIX variable in my Docker image broke the configure step.
• For a .deb that runs on any x86-64 machine I build with GGML_NATIVE=OFF and check libggml-cpu for AVX-512 instructions before packaging.
• whisper.cpp tends to crash rather than fail cleanly when VRAM runs out, so I check free VRAM first (the LLM shares the card) and fall back to the CPU if the model won't fit with some margin.
• A dummy transcription at startup gets the Vulkan shader compile out of the way, so the first real utterance isn't slow.

It ships as a .deb for Ubuntu 24.04+ with its own Python and the Vulkan build, so nothing compiles on the user's machine. I've only properly tested on an RTX 3070 so far, so if anyone tries it on AMD or Intel through Vulkan I'd really like to hear how it goes.

Thanks :)

https://github.com/PSYEONE/interviewr-ai