[Feature] Voice input — transcribe Slack audio messages

[davesleal]

Context

Slack mobile supports voice messages (audio clips). When sent to Shellack, the event arrives with empty text and the audio as a files attachment. The bot currently sends the empty text to the API, causing a 400 error: user messages must have non-empty content.

Error

anthropic.BadRequestError: Error code: 400 - messages.0: user messages must have non-empty content

Proposed fix

1. Detect audio files in the Slack event (event.get("files", []) where filetype is mp4/webm/ogg/m4a)
2. Download the audio via Slack API (files.info + authenticated URL)
3. Transcribe using one of:
   • OpenAI Whisper API — $0.006/min, best accuracy, requires OpenAI key
   • Local Whisper — whisper CLI or faster-whisper Python, free, needs model download (~1.5GB)
   • Deepgram — real-time transcription API, pay-per-use
4. Use transcript as prompt — feed to the agent as if the user typed it
5. Post transcript as a thread reply so the user can verify what was understood

Immediate fix (before transcription)

At minimum, handle the empty-text case gracefully:

if not prompt.strip():
    if event.get("files"):
        app.client.chat_postMessage(
            channel=channel_id, thread_ts=thread_ts,
            text="Voice messages aren't supported yet — type your message or try again with text."
        )
        return

Priority

Medium — nice UX improvement for mobile users. The immediate graceful error should ship now.

[m13v]

we handle voice message transcription in a macOS agent and the pipeline you're describing is solid. a few notes:

for the audio format handling, Slack voice messages from mobile are m4a (AAC encoded). whisper.cpp and faster-whisper both need 16kHz mono PCM/WAV input. the simplest path is ffmpeg but if you want to avoid the system dependency, the pydub library can do the conversion with just the audioop module from the Python stdlib (no ffmpeg needed for m4a to wav).

on model choice, Whisper base.en is the sweet spot for chat message transcription. voice messages are typically short (5-30 seconds) and conversational, so the larger models don't improve accuracy much but add noticeable latency. base.en processes a 15 second clip in about 400ms on a modern CPU.

the graceful error handling for empty text is essential. one thing to add: check for files where mimetype starts with 'audio/' rather than checking specific extensions. Slack sometimes sends voice messages with unexpected container formats depending on the mobile OS version and recording app.

worth caching the transcription result alongside the original audio URL in case you need to re-process. whisper is deterministic for the same input, but if you switch models later you might want to re-transcribe without re-downloading from Slack (URLs expire after a while).

[m13v]

our voice transcription service with model selection, audio preprocessing, and the chunked processing pipeline: https://github.com/m13v/fazm/blob/main/Desktop/Sources/TranscriptionService.swift

covers the 16kHz mono conversion, model lifecycle, and confidence-based transcript filtering.