Repository navigation
AssemblyAI Dictation as a speech-to-text provider #18045
dan-ince-aai
started this conversation in
Feature Requests & Suggestions
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
What: AssemblyAI as a speech-to-text provider, next to OpenAI and Azure OpenAI, using AssemblyAI's Dictation API.
Why: Speech-to-text engines return the words as spoken, so when you talk to a model, the filler, false starts and self-corrections go straight into the prompt ("um, can you, no wait, summarize the second doc"). Cleaning that up means editing the text by hand or a second LLM call before sending. Dictation returns cleaned-up text in the same request.
How:
POST https://dictation.assemblyai.com/v1/transcribewith the recording as a multipartaudiopart (WAV, WebM, MP3, M4A, Ogg/Opus, FLAC; 80 ms to 20 minutes) and anAuthorization: <api key>header. No upload or polling.text(verbatim) andllm_response(filler words removed, punctuation, capitalization and self-corrections fixed). Usellm_response, falling back totextwhen it's empty.configpart takeskeyterms_prompt(names and jargon to spell right) andlanguage_codes. It covers 32 languages.Areas affected:
api/server/services/Files/Audio/STTService.js(a new provider inSTTProvidersalongsideopenAIProviderandazureOpenAIProvider), thespeech.sttsection oflibrechat.yaml, and its docs. It would follow the OpenAI provider's shape: a request built from the uploaded audio, and the text pulled out of the response.I'm happy to take this once it's approved.
Disclosure: I work at AssemblyAI. If you have questions or want a hand with anything on our side, comment here or reach me on X at @dan_aai.
All reactions