Plugin SDK

Plugin SDK

The Plugin SDK (TypeWhisper.PluginSDK) is the contract layer for extending TypeWhisper. A plugin is a .NET 10 class library that references the SDK, ships a manifest.json, and implements one or more of the capability interfaces below. The host discovers it, reads the manifest, instantiates the plugin class, and calls into it through these contracts.

For where plugins live on disk, how bundled plugins are deployed, and the known gaps, see Plugins.

Plugin lifecycle

Every plugin implements the base interface ITypeWhisperPlugin (which is IDisposable):

Member	Purpose
string PluginId	Unique id, e.g. com.example.my-plugin.
string PluginName	Human-readable display name.
string PluginVersion	Semantic version, e.g. 1.0.0.
Task ActivateAsync(IPluginHostServices host)	Called once after construction. Receives the host services object (below).
Task DeactivateAsync()	Called before Dispose.

Neither ActivateAsync nor DeactivateAsync may block — kick off long-running work on a background task. ActivateAsync is where you capture the IPluginHostServices handle you'll use for storage, secrets, logging, and events.

The manifest

The host loads plugin metadata from a manifest.json in the plugin's directory. A real example (the bundled OpenAI plugin):

{
  "id": "com.typewhisper.openai",
  "name": "OpenAI / ChatGPT",
  "version": "1.2.0",
  "author": "TypeWhisper",
  "description": "OpenAI transcription, ChatGPT/OpenAI prompt processing, and text-to-speech.",
  "category": "transcription",
  "assemblyName": "TypeWhisper.Plugin.OpenAi.dll",
  "pluginClass": "TypeWhisper.Plugin.OpenAi.OpenAiPlugin"
}

Field	Required	Notes
id	✔	Unique plugin identifier.
name	✔	Human-readable name.
version	✔	Semantic version.
assemblyName	✔	The DLL containing the plugin type.
pluginClass	✔	Fully-qualified class implementing ITypeWhisperPlugin.
author	—	Author name.
description	—	Short description.
category	—	UI grouping: transcription, llm, memory, action, utility, …
isLocal	—	true for on-device plugins, false for cloud.
minHostVersion	—	Minimum host version, or omit for any.

Capability interfaces (roles)

A plugin class implements one or more of these roles. A single class can fill several — the bundled OpenAI plugin implements ITranscriptionEnginePlugin, ILlmProviderPlugin, ITtsProviderPlugin, and IPluginSettingsProvider at once.

Interface	Role	Key members
ITranscriptionEnginePlugin	Speech-to-text engine	ProviderId, IsConfigured, TranscriptionModels, SelectModel, TranscribeAsync(wav, language, translate, prompt, ct)
ILlmProviderPlugin	LLM for prompts/cleanup	ProviderName, IsAvailable, SupportedModels, ProcessAsync(systemPrompt, userText, model, ct), ProcessStreamingAsync(...)
IPostProcessorPlugin	Text transform after transcription	ProcessorName, Priority (lower runs first), ProcessAsync(text, context, ct)
IActionPlugin	Action on transcribed/selected text	ActionId, ActionName, ExecuteAsync(input, context, ct) → ActionResult
IMemoryStoragePlugin	Persist/recall extracted facts	StoreAsync, SearchAsync, GetAllAsync, DeleteAsync, ClearAllAsync, CountAsync
ITtsProviderPlugin	Spoken-feedback voices	ProviderId, AvailableVoices, SelectVoice, SpeakAsync(request, ct) → playback session

Transcription-engine extras

ITranscriptionEnginePlugin has rich optional (default-implemented) members so simple engines stay small while advanced ones opt in:

• Model management — SupportsModelDownload, IsModelDownloaded, DownloadModelAsync (with IProgress<double>), LoadModelAsync (a LoadModelAsync(modelId, IProgress<double>, ct) overload reports provisioning progress, which the host surfaces as a downloading state), UnloadModelAsync, DeleteModelAsync.
• Acceleration — SupportedAccelerationBackends, AccelerationStatus, ConfigureComputeBackendAsync("cpu"|"cuda"). The host resolves Auto to Cpu/NvidiaCuda before calling. An engine that downloads its own GPU runtime opts in with ProvisionsCudaRuntimeOnDemand = true, reports readiness via IsCudaRuntimeProvisioned, and fetches the missing libraries in EnsureCudaRuntimeReadyAsync(progress, ct); for an explicit CUDA load on such an engine the host skips its system-CUDA preflight and lets the engine provision and fall back itself. See GPU & CUDA.
• Streaming — set SupportsStreaming = true and implement StartStreamingAsync(...) returning an IStreamingSession; the host feeds it PCM16 audio for real-time partials. See Transcription engines.
• Translation — SupportsTranslation, SupportedLanguages.

Host services

ActivateAsync receives IPluginHostServices — the plugin's window into the app:

Member	Purpose
PluginDataDirectory / PluginAssetDirectory	Where to write config vs. large model/runtime assets (the asset dir may be redirected to a user-chosen drive).
StoreSecretAsync / LoadSecretAsync / DeleteSecretAsync	Per-plugin secret storage via the platform secret store (use this for API keys).
GetSetting<T> / SetSetting<T>	Per-plugin JSON-serialized settings.
Localization	The plugin's IPluginLocalization (see below).
EventBus	Publish/subscribe IPluginEventBus.
ActiveAppProcessName / ActiveAppName	The focused app (for context-aware behavior).
AvailableProfileNames	The dictation Profiles.
Log(level, message)	Logs through the host (surfaces in the About Error Log / app logs).
NotifyCapabilitiesChanged()	Tell the host to rebuild capability indices and refresh the UI (e.g. after fetching new models).
SetStreamingDisplayActive(bool)	Suppress the host overlay while the plugin renders its own streaming UI.

Use StoreSecretAsync for API keys, not SetSetting — secrets go to the platform secret store, settings to plain JSON.

Optional capability interfaces

Implement these alongside a role to opt into extra host integration:

Interface	What it adds
IPluginSettingsProvider	A host-rendered settings UI (see below).
IPluginCollectionSettingsProvider	An ordered, user-editable list (e.g. custom prompts, word replacements) the host edits in the settings UI.
IModelCatalogProvider	For providers whose model list is fetched at runtime from a remote endpoint (e.g. an OpenAI-compatible server) and can change.
IStreamingSession	An active real-time streaming session (e.g. a WebSocket) created by StartStreamingAsync.
IPluginDataLocationAware	Receive your on-disk data directory before ActivateAsync.
IPluginSettingsActivity	Report progress/status messages into your settings view.
IAdditionalLlmProvidersProvider / IAdditionalTranscriptionEnginesProvider	Expose multiple provider/engine roles from one plugin (e.g. extra OpenAI-compatible endpoints as their own providers).

Settings UI

Implement IPluginSettingsProvider and the host renders a generic settings panel from your definitions — no UI code required. Values are stored and retrieved as strings; you parse them.

IReadOnlyList<PluginSettingDefinition> GetSettingDefinitions();
Task<string?> GetSettingValueAsync(string key, CancellationToken ct = default);
Task SetSettingValueAsync(string key, string? value, CancellationToken ct = default);
Task<PluginSettingsValidationResult?> ValidateAsync(CancellationToken ct = default); // optional

Each PluginSettingDefinition has a Key, Label, optional Placeholder/Description/Options, an IsSecret flag, and a Kind:

PluginSettingKind	Rendered as
Auto	Inferred from the other properties
Text	Single-line text field
Secret	Masked field, stored securely
Dropdown	Picker from Options (PluginSettingOption(value, label))
Boolean	Toggle
Multiline	Text area

ValidateAsync lets you run a connectivity or key-format check and return a PluginSettingsValidationResult(isSuccess, message) the host shows inline (e.g. the "Test" button on cloud providers).

Localization

Plugins localize their own UI. IPluginLocalization (from host.Localization) loads flat key-value JSON from a Localization/<lang>.json subdirectory (en.json, de.json, es.json, ru.json, …):

{
  "Manifest.Name": "OpenAI / ChatGPT",
  "Manifest.Description": "OpenAI transcription, ChatGPT prompt processing, and TTS.",
  "Settings.ApiKey": "API Key",
  "Settings.Test": "Test"
}

GetString(key) (and GetString(key, args)) returns the string for the current language, falling back to English, then to the key itself. The host renders your setting definitions through this layer, so the same definitions display localized labels. Manifest.Name / Manifest.Description localize the plugin's name and description in the Plugins list. The loc.L(key) / loc.L(key, args) extension helpers wrap GetString with a null-safe fallback so plugin code can resolve strings even before a catalog is attached.

Implement the optional IPluginLocalizationAware interface (a single SetLocalization(IPluginLocalization) method) to receive the catalog at load time, before — and independent of — activation. This is what lets a disabled plugin's settings panel render localized labels instead of raw keys like Settings.ApiKey; without it, localization would only be available after activation, which only happens for enabled plugins. When the user switches interface language, the host re-resolves these definitions, so plugin settings follow the rest of the app.

Built-in helpers

For OpenAI-compatible providers, the SDK ships shared helpers so you don't reimplement request shaping and error handling:

Helper	Use
OpenAiChatHelper	OpenAI-compatible chat-completion calls (for ILlmProviderPlugin).
OpenAiTranscriptionHelper	Whisper-compatible audio transcription calls (for ITranscriptionEnginePlugin).
OpenAiApiHelper	Shared HTTP send + error handling for the above.

Key model types

Type	Used by
PluginManifest	The parsed manifest.json.
PluginModelInfo(Id, DisplayName)	A model in TranscriptionModels / SupportedModels; carries SizeDescription, EstimatedSizeMB, IsRecommended, LanguageCount.
PluginTranscriptionResult(Text, DetectedLanguage, DurationSeconds, NoSpeechProbability?)	Returned by TranscribeAsync; optional Segments (timed PluginTranscriptionSegments) for SRT/VTT.
ActionContext(AppName, ProcessName, Url, Language, OriginalText)	Passed to IActionPlugin.ExecuteAsync.
ActionResult(Success, Message?, Url?, Icon?, DisplayDuration)	Returned by an action.
PostProcessingContext	SourceLanguage, ActiveAppName, ActiveAppProcessName, ProfileName, AudioDurationSeconds.
PluginVoiceInfo / TtsSpeakRequest	TTS voices and speak requests.
PluginEvent	Base type for events published on the IPluginEventBus (in Models/PluginEvents.cs).

Minimal example

A complete post-processor plugin that upper-cases transcribed text:

using TypeWhisper.PluginSDK;
using TypeWhisper.PluginSDK.Models;

public sealed class UpperCasePlugin : ITypeWhisperPlugin, IPostProcessorPlugin
{
    public string PluginId => "com.example.uppercase";
    public string PluginName => "Upper Case";
    public string PluginVersion => "1.0.0";

    public string ProcessorName => "Upper Case";
    public int Priority => 100; // lower runs first

    public Task ActivateAsync(IPluginHostServices host) => Task.CompletedTask;
    public Task DeactivateAsync() => Task.CompletedTask;
    public void Dispose() { }

    public Task<string> ProcessAsync(string text, PostProcessingContext context, CancellationToken ct)
        => Task.FromResult(text.ToUpperInvariant());
}

with a manifest.json beside the built DLL:

{
  "id": "com.example.uppercase",
  "name": "Upper Case",
  "version": "1.0.0",
  "category": "utility",
  "assemblyName": "MyPlugin.dll",
  "pluginClass": "MyPlugin.UpperCasePlugin"
}

Project layout and deployment

• A plugin is a .NET 10 class library referencing TypeWhisper.PluginSDK, with manifest.json (and any Localization/*.json) marked as content copied to output.
• The build/publish output — the DLL, its dependencies, manifest.json, and Localization/ — is what the host loads. Bundled plugin sources live under plugins/; release builds run scripts/deploy-linux-plugins.sh to publish and stage them.
• At runtime, plugins are loaded from the user plugin directory (~/.local/share/TypeWhisper/Plugins/ on typical setups). See Plugins and Data and file paths.

Related pages

• Plugins — plugin categories, loading, deployment, and known gaps.
• Transcription engines, LLM providers, Other plugins — the bundled plugins built on these contracts.
• Contributing — building and contributing to the project.

---

Changelog

Date	Change
2026-06-17	Initial version.
2026-06-17	Expanded: lifecycle, manifest, capability and optional interfaces, host services, settings UI, localization, helpers, model types, and a minimal example.
2026-06-17	Corrected events base type name to PluginEvent (singular).
2026-06-19	v0.9.0: documented IPluginLocalizationAware (load-time catalog injection) and the loc.L(...) helpers; added es.json to the localization example.
2026-06-23	v0.10.0: documented the on-demand CUDA provisioning members (ProvisionsCudaRuntimeOnDemand, IsCudaRuntimeProvisioned, EnsureCudaRuntimeReadyAsync) and the progress-reporting LoadModelAsync overload.