Skip to content

Scribe 2.3.7 feat: add AssemblyAI LLM support with model selection and configuration

Choose a tag to compare

@mikealicea mikealicea released this 12 Jul 20:31
· 11 commits to main since this release
d2aff65

Scribe 2.3.7 feat: add AssemblyAI LLM support with model selection and configuration

Add AssemblyAI as an LLM summarization provider

Summary

Adds AssemblyAI's LLM Gateway as a summarization provider, so a user can run both transcription and LLM summarization through AssemblyAI — keeping data within a single vendor's privacy envelope (the motivation behind this ZDR / no-training branch) instead of sending transcripts to a second provider.

Because the gateway is OpenAI-compatible, it routes through the existing createOpenAiCompatibleLlmAdapter (LangChain ChatOpenAI + obsidianFetch + a baseURL override) — the same mechanism already used for Gemini and OpenRouter. No new transcription/LLM call logic was required; this is almost entirely wiring a new provider through the existing data-driven plumbing.

What changed

New provider wiring

  • src/util/consts.ts — added assemblyAi to the PROCESS_PLATFORM enum.
  • src/aiProviders/llm/openAiCompatibleLlm.ts — added the gateway base URL, default model (claude-sonnet-4-6), and a seed list of common model IDs.
  • src/aiProviders/llm/llmAdapter.ts — resolveLlmConfig case mapping the provider to assemblyAiApiKey + assemblyAiLlmModel + the gateway base URL. createLlmAdapter needed no change (non-Anthropic already routes to the OpenAI-compat adapter).
  • src/aiProviders/providerMetadata.ts — LLM_PROVIDERS entry (drives the settings dropdown, privacy card, and missing-key checks).
  • src/settings/settings.tsx — new assemblyAiLlmModel setting, defaulted to claude-sonnet-4-6.

UI (free-text model combobox, mirroring OpenRouter)

  • src/settings/components/ProviderSettingsSections.tsx — AssemblyAI section reusing the existing API-key input + a combobox seeded with known model IDs (any gateway model ID allowed).
  • src/modal/components/options/ModalAiModelOptions.tsx — matching combobox branch in the per-run modal picker.

Bug fixes (gateway-specific)

Summarization initially failed only with AssemblyAI. Two gateway behaviors differ from OpenAI/Gemini/OpenRouter, both fixed in openAiCompatibleLlm.ts:

  1. Output truncation — the gateway defaults max_tokens to 1000 when omitted (unlike OpenAI/Gemini, which use the model's high default). LangChain doesn't send max_tokens, so full structured notes were truncated mid-JSON and failed to parse. Fixed by sending an explicit maxTokens: 8192 for the gateway.
  2. Dropped transcript — the transcript lives in the first system message, and we always send ≥2 system messages (the language instruction defaults to en). When the gateway collapses OpenAI's multiple system messages into Anthropic's single system field for Claude models, the transcript-bearing message could be lost. Fixed by merging the main prompt + extra instructions into a single system message (behavior-equivalent for the other OpenAI-compat providers).

ZDR note

No special request headers are needed — Zero Data Retention on the LLM Gateway is an account-level arrangement (executed BAA + training opt-out + optional TTL) and applies
only when routing to Anthropic or Google models, not GPT/Qwen/Kimi. The de ZDR-eligible; users who select a gpt-* model in the combobox fall outsidethe ZDR envelope.

Testing

  • npm run format:write, npm run lint:write, npm run build:prod all pass cleanly (incl. the mobile-safety check).
  • Manual verification still needed in Obsidian: with an AssemblyAI key ander selected, confirm the note gets its summary sections + LLM-suggestedtitle. If it fails, the gateway's error body surfaces in the console via [obsidianFetch] error body:.

(Note: the branch also switches the AssemblyAI transcription default speech model to universal-3-5-pro in assemblyAiTranscriber.ts.)

What's Changed

  • Scribe 2.3.7 feat: add AssemblyAI LLM support with model selection and configuration by @Mikodin in #109

Full Changelog: 2.3.6...2.3.7