Skip to content

Scribe 2.3.5

Choose a tag to compare

@mikealicea mikealicea released this 11 Jul 23:19
· 14 commits to main since this release
a2f5b39

Scribe 2.3.5 Adds a ton of providers for transcription + summarizing

Multi-provider AI refactor: transcription + summarization

Closes #54 @skorphil

Tested every provider on mobile and desktop

Summary

Replaces the single-LLM (OpenAI only) / two-transcriber (OpenAI + AssemblyAI) setup with a provider-adapter architecture supporting 6 transcription providers and 5 LLM providers, plus inline privacy/data-retention info for each and a searchable model picker for OpenRouter.

  • LLM (summarization + mermaid-fix): OpenAI, Anthropic, Google (Gemini), OpenRouter, custom OpenAI-compatible endpoint
  • Transcription: OpenAI Whisper, AssemblyAI, ElevenLabs Scribe, Deepgram Nova-3, Mistral Voxtral, Google (Gemini), custom OpenAI-compatible endpoint

Why this shape

PR #101 attempted Google support via @langchain/google-genai and was reverted (#102): that package transitively pulls in node:async_hooks, which crashes Obsidian mobile (WebView, no Node runtime). This PR avoids that class of bug entirely:

  • Anthropic uses the native @anthropic-ai/sdk (browser-safe, dangerouslyAllowBrowser: true) — the only new LangChain-adjacent dependency added.
  • Google, OpenRouter, and custom endpoints all route through the existing LangChain ChatOpenAI client via baseURL, since Gemini and OpenRouter both expose OpenAI-compatible chat completions endpoints. No Google SDK, no new provider packages.
  • ElevenLabs, Deepgram, Mistral, and Gemini transcription are raw fetch calls through the existing obsidianFetch CORS wrapper — no SDKs at all.
  • A new scripts/checkMobileSafe.mjs build step greps build/main.js for async_hooks/AsyncLocalStorage and fails the production build if either appears — thithat would have caught the #101 bug before merge.

Architecture

New src/aiProviders/ module:

src/aiProviders/
  providerMetadata.ts          # registry: diseld, keyConsoleUrl,
                                #   models?, supportsDiarization?, privacy info
  prompts.ts                   # shared promptd from the old
                                #   openAiUtils.ts, byte-identical prompt text)
  llm/
    llmAdapter.ts               # LlmAdapter interface + resolveLlmConfig() + createLlmAdapter()
    openAiCompatibleLlm.ts      # LangChain Ch google / openRouter / customOpenAi
    anthropicLlm.ts             # native @anthropic-ai/sdk adapter
    openRouterModels.ts         # fetches + cacatalog
  transcription/
    transcriptionAdapter.ts     # dispatches oPlatform
    openAiTranscriber.ts        # moved from src/util/openAiUtils.ts
    assemblyAiTranscriber.ts    # moved from s unchanged
    elevenLabsTranscriber.ts
    deepgramTranscriber.ts
    mistralTranscriber.ts
    geminiTranscriber.ts
    diarizationFormat.ts        # shared "**Speaker N**: text" formatter

Deleted: src/util/openAiUtils.ts, src/util/split),src/settings/components/AiModelSettings.tsx` (replaced by the registry-driven settings UI).

src/util/obsidianFetch.ts wraps Obsidian's requestUrl() as a CORS-bypassing fetch implementation — used by every
HTTP-based provider client (OpenAI SDK, Anthroetch calls). Since requestUrl() buffers the fullresponse, no provider call streams — all LLM calls use non-streaming structured output.

Provider details

LLM

Provider Transport Models Notes
OpenAI LangChain ChatOpenAI GPT-5.6/5.5/5.1/5/4.1/4o family unchanged from before
Anthropic @anthropic-ai/sdk, client.mestFormat claude-opus-4-8, claude-sonnet-5(default), claude-sonnet-4-6, claude-haiku-4-5 never sends temperature (current Claude models reject non-default
values); surfaces refusal/max_tokens stop
Google (Gemini) LangChain ChatOpenAI via Gemini's OpenAI-compat endpoint
(`generativelanguage.googleapis.com/v1beta/opedefault, GA), gemini-3.1-pro-preview,gemini-3-flash-preview, gemini-3.1-flash-lite, gemini-2.5-pro, gemini-2.5-flash no Google SDK — sidesteps the async_hooks
issue entirely
OpenRouter LangChain ChatOpenAI via openrouter.ai/api/v1 any model id, searchable combobox (see below)
temperature omitted — OpenRouter routes to arbs/models with their own constraints
Custom OpenAI-compatible LangChain ChatOpenAI via user-supplied baseURL free text unchanged from before

Transcription

Provider Transport Diarization Notes
OpenAI Whisper OpenAI SDK, chunked WAV upload No unchanged from before
AssemblyAI assemblyai SDK Yes unchand through obsidianFetch)
ElevenLabs Scribe raw fetch, multipart Yes (word-level → merged into speaker runs)
Deepgram Nova-3 raw fetch, raw audio byt)
Mistral Voxtral raw fetch, multipart No
Google (Gemini) raw fetch to `generateCoo Yes (prompt-driven) no dedicated STT endpoint — audio is chunked to WAV (reusing the existing chunker) to stay under the 20MB inline-request limit, sent with a
transcription prompt

All diarized output converges on the same `**Son regardless of provider.

Settings UI

  • AI Providers tab: separate "Transcription" as, each with a provider dropdown, that provider'sAPI key + model fields inline, and a privacy card — retention summary, whether the provider trains on your API data,
    and a policy link. Facts are July-2026 snapshopolicies drift.
  • OpenRouter model field is now a searchable combobox (native <input list> + <datalist>, no new dependency) — it
    fetches OpenRouter's public model catalog (no ers as you type, while still accepting arbitraryfree text. Same picker is used in the modal's per-run model override, which was previously a disabled read-only input for
    OpenRouter.
  • Settings migrate automatically: the old useCustomOpenAiBaseUrl boolean maps to processPlatform/transcriptPlatform = customOpenAi; everyone else is unaffected. "R all API keys (now 7 of them).

Bug fix (behavior change)

handleTranscription previously read this.se, ignoring the modal's per-runscribeOptions.transcriptPlatform` override — so switching providers in the modal had no effect on the actual request. This
is now fixed; modal overrides take effect. Worince it changes observable behavior for anyone who was relying on (or unknowingly affected by) the old ignored-override behavior.

Build safety

  • Fixed a broken esbuild config: builtin-modules@5 is ESM-only, so its default import silently resolved to undefined
    and Node builtins were being bundled unnoticedle's builtinModules(bare +node:`-prefixedvariants).
  • Added scripts/checkMobileSafe.mjs, wired ifails the build if build/main.js containsasync_hooks or AsyncLocalStorage.

Verification

  • npm run format:write — clean
  • npm run build:prod (tsc + esbuild + mobile
  • Confirmed all provider endpoints present in build/main.js (ElevenLabs, Deepgram, Mistral, Gemini generateContent +
    OpenAI-compat, OpenRouter, Anthropic)
  • Confirmed extracted summary prompt text is unchanged (byte-identical modulo trailing whitespace) vs. the original
    openAiUtils.ts

Not covered by this PR (needs real API keyse)

  • Live provider calls per transcription providses for ElevenLabs/Deepgram/Mistral/Gemini)
  • Anthropic and Gemini structured-output round-trips against real audio/transcripts
  • OpenRouter combobox behavior against the liv
  • Settings migration against a hand-edited data.json with useCustomOpenAiBaseUrl: true
  • Mobile sideload smoke test
  • Mistral Voxtral's file-size limit is unverified (chunking fallback noted in code but untested)

What's Changed

  • Feat: Add support for all major providers by @Mikodin in #108

Full Changelog: 2.3.4...2.3.5