Release Notes - 1.2.0

Release Date: March 2026
Code Name: "Pulsar"

🎙️ Overview

"Pulsar" introduces the Wyoming Voice Platform — a full local speech pipeline that turns Lucia into a Wyoming protocol-compatible voice satellite for Home Assistant. This release delivers streaming speech-to-text, speaker verification, wake word detection, speech enhancement, a multi-engine model management system, and a new dashboard Voice Platform page for configuring everything from one surface. In addition, we introduce the Conversation Command Parser — a fast-path processing pipeline that pattern-matches common voice commands (lights, climate, scenes) and executes them directly against Home Assistant, bypassing the LLM entirely for sub-50ms response times. We also introduce user-facing personalization to Lucia's orchestration layer, letting users define a personality prompt that rewrites all agent responses through an LLM before delivery, and introduces Pluggable Data Providers — an agnostic persistence layer that enables mono-container deployment as a Home Assistant add-on with InMemory caching and SQLite storage, eliminating the need for Redis and MongoDB on resource-constrained devices.

🚀 Highlights

Wyoming Protocol Server — Native TCP server implementing the Home Assistant Wyoming satellite protocol for real-time voice processing.
Multi-Engine STT Pipeline — Hybrid streaming STT with progressive re-transcription, Sherpa streaming, Sherpa offline (Parakeet TDT/CTC), and IBM Granite 4.0 1B Speech ONNX engines.
Speaker Verification & Profiling — Cosine-similarity speaker identification with enrolled profiles, provisional auto-discovery, adaptive profile updates, and guided voice enrollment onboarding.
ONNX Auto-Detection — Automatic GPU/accelerator selection (CUDA, ROCm, OpenVINO, DirectML, CoreML) across all six inference engines — no manual configuration required.
GTCRN Speech Enhancement — Real-time streaming noise reduction for cleaner audio in noisy environments.
Voice Platform Dashboard — New unified control room for model management, speaker profiles, wake words, engine status, and real-time session monitoring.
Personality Prompt — Configurable system prompt that rewrites the fan-in aggregated response through an LLM, giving Lucia a customizable personality (pirate speak, formal assistant, casual friend — you name it).
Conversation Command Parser — New POST /api/conversation endpoint with pattern-matching pipeline that executes common smart home commands (lights, climate, scenes) directly via skills — zero LLM latency for recognized commands, SSE-streamed LLM fallback for everything else.
Response Templates — Customizable response templates stored in MongoDB with {placeholder} interpolation. Manage templates per skill/action from the dashboard with guided dropdowns and token insertion buttons.
Home Assistant Component v1.2 — Simplified integration migrated from A2A JSON-RPC to structured REST. No more agent catalog selection — just point to the host and go.
Separate Model Support — Personality rewriting can use a different (cheaper/faster) model than the orchestrator, selectable from a dropdown of configured chat-type providers.
Zero-Cost Opt-In — When no personality is configured, the pipeline is unchanged — no LLM call, no added latency.
Pluggable Data Providers — Agnostic persistence layer with InMemory (replaces Redis) and SQLite (replaces MongoDB) alternatives, configurable via DataProvider:Cache and DataProvider:Store.
Home Assistant Mono-Container — New Dockerfile.ha for single-container deployment with zero external dependencies, CPU-only ONNX, and embedded dashboard.
CpuOnly Build Flag — MSBuild property /p:CpuOnly=true excludes GPU ONNX packages for clean CPU-only container builds.

✨ Features

Wyoming Protocol Server

Full Wyoming satellite protocol implementation (audio-start, audio-chunk, audio-stop, transcribe, transcript, detect, not-detected)
Persistent TCP connections with per-session state management
Zeroconf/mDNS service advertisement for automatic Home Assistant discovery
VAD-driven voice activity detection with configurable speech/silence thresholds
Wake word detection with custom phrase support and per-speaker calibration

Multi-Engine Speech-to-Text

HybridSttEngine — Streams audio through a lightweight online model, then re-transcribes the full utterance with a high-accuracy offline model for best-of-both-worlds latency and accuracy
SherpaSttEngine — Pure streaming CTC/transducer inference for ultra-low-latency transcription
SherpaOfflineSttEngine — Offline NeMo Parakeet TDT+CTC support for high-quality batch transcription
GraniteOnnxEngine — IBM Granite 4.0 1B Speech ONNX with 3-model pipeline (audio encoder, embed tokens, auto-regressive decoder with 40-layer KV cache)
Progressive re-transcription with burst detection and stability-based early stopping
Per-engine model catalog with download, install, activate, and delete lifecycle

Speaker Verification & Voice Profiles

Sherpa-onnx speaker embedding extraction with cosine similarity matching
Enrolled speaker profiles with configurable verification threshold
Provisional auto-discovery profiles for unknown speakers with interaction tracking
Adaptive profile updates (exponential moving average) for high-confidence matches
Guided onboarding flow with audio quality validation (SNR, duration checks)
Profile merge, rename, and provisional-to-enrolled promotion via API and dashboard
Audio clip capture, playback, reassignment, and per-profile management
Redis-cached enrolled profiles for sub-millisecond lookup latency
MongoDB-backed persistent storage with in-memory fallback

ONNX Provider Auto-Detection

OnnxProviderDetector singleton probes OrtEnv.Instance().GetAvailableProviders() at startup
Automatic selection priority: CUDA → ROCm → OpenVINO → DirectML → CoreML → CPU
Applied to all six engines: SherpaDiarization, HybridSTT, SherpaStt, SherpaOfflineSTT, GraniteOnnx, GtcrnSpeechEnhancer
Detected provider exposed in /api/wyoming/status and displayed on the dashboard status card with GPU badge
Graceful fallback — if an accelerated provider fails to initialize, falls through to CPU

Speech Enhancement

GTCRN (Gated Temporal Convolutional Recurrent Network) streaming noise reduction
Per-session enhancement with isolated overlap-add state
Raw audio preserved separately for speaker verification to avoid spectral mismatch with enrollment data

Model Management System

Centralized model catalog with architecture-aware compatibility filtering
Background download with real-time progress tracking via SSE
Multi-stage progress bars (download → extract → validate)
Hot-reload model activation via ActiveModelChanged events — no restart required
Per-engine model views (STT, VAD, Wake Word, Speaker Embedding, Speech Enhancement)
Startup model validation with dummy inference warmup

Voice Platform Dashboard

Status tab — Engine readiness tiles, active model indicators, ONNX provider detection display, and guided next-steps checklist
Models tab — Browse, download, activate, and delete models per engine type with real-time download progress
Profiles tab — View enrolled and provisional speaker profiles, inline rename, promote to enrolled, merge profiles, manage audio clips with playback
Wake Words tab — Register custom wake phrases with optional speaker enrollment, manage and delete entries
Monitor tab — Real-time session monitoring via SSE with audio level meters, transcript log, and connection status
Config panel — Voice verification threshold, provisional profile settings, adaptive profiles, and retention controls — persisted to MongoDB via ConfigStoreWriter

🎭 Personality Prompt (PR #80)

Users can now define a personality prompt on the /configuration page under the new Personality Prompt section. When set, the ResultAggregatorExecutor passes the composed multi-agent response through an LLM with the personality instructions as the system prompt, rewriting the output to match the desired tone.

PersonalityPromptOptions — New config model with Instructions (the personality system prompt) and ModelConnectionName (optional separate LLM for rewriting)
WorkflowFactory — Accepts IOptionsMonitor<PersonalityPromptOptions> for live config reload without restart. Conditionally resolves a separate IChatClient when ModelConnectionName is set, or falls back to the orchestrator's default model.
ResultAggregatorExecutor — After composing the raw message from agent responses, calls the personality LLM with system=instructions + user=composed message. Graceful fallback to raw message on LLM failure or empty response.
Resilient resolution — A misconfigured ModelConnectionName logs a warning and disables personality for that request instead of failing the entire orchestration pipeline.
Cancellation-safe — OperationCanceledException propagates correctly through the personality rewrite path.

🖥️ Dashboard

Model provider dropdown — The ModelConnectionName field renders as a searchable dropdown populated from configured chat-type model providers, matching the pattern used in Agent Definitions.
Textarea field type — New textarea field type in the configuration page for multi-line prompt editing with vertical resize support.
Auto-discovery — The Personality Prompt section appears automatically in the configuration sidebar via schema API.
Conversation Test Page — Interactive chat interface at /conversation for testing the command parser endpoint directly from the dashboard, with HA context simulation, SSE streaming, and response metadata badges showing whether commands were parsed locally or handled by the LLM.
Response Templates Page — Full CRUD management of response templates at /response-templates, grouped by skill. Features pattern-driven SkillId/Action dropdowns populated from the parser, token insertion buttons for {entity}, {action}, {area} placeholders, and template preview with sample values.
Activity Dashboard Metrics — Three new summary cards: Command Parsed count, LLM Fallback count, and Parser Rate percentage showing the ratio of commands handled locally vs forwarded to the LLM.

💬 Conversation Command Parser

POST /api/conversation — New REST endpoint accepting structured JSON with user text and separated device/session context object. Returns instant JSON for parsed commands or SSE streaming for LLM fallback.
True LLM Bypass — Pattern-matched commands call skill methods (LightControlSkill, ClimateControlSkill, SceneControlSkill) directly against Home Assistant — zero LLM involvement, sub-50ms execution.
Command Pattern Matching — Leverages the existing CommandPatternRouter with template syntax ({name}, {name:opt1|opt2}, [optional]), confidence scoring, and priority-based tiebreaking.
DirectSkillExecutor — Dispatches matched routes to concrete skill methods: light toggle/brightness, climate set temperature/adjust, scene activate. Resolves entities via captured text and device area context.
Response Template System — MongoDB-stored templates with simple {placeholder} interpolation, random variant selection for natural variety, and seed defaults for all supported commands. Managed via CRUD API and dashboard.
Context Reconstructor — Rebuilds system prompt from structured context JSON for LLM fallback, matching the format the orchestrator expects.
Conversation Telemetry — OpenTelemetry counters (conversation.command_parsed, conversation.llm_fallback, conversation.command_parsed.errors) and duration histograms, plus in-memory stats for the activity dashboard.
Multi-Turn Continuity — Server generates stable conversationId (GUID) when client doesn't provide one, used consistently across command and LLM paths.
GET /api/conversation/patterns — Exposes registered command patterns with their SkillId, Action, placeholder tokens, and example templates for dashboard tooling.

🏠 Home Assistant Component (v1.2.0-preview.1)

REST Migration — Component migrated from A2A JSON-RPC 2.0 to the new POST /api/conversation endpoint with structured context objects.
Simplified Setup — Removed agent catalog fetch and agent selection. Configuration requires only host URL + API key.
SSE Streaming — Handles both instant JSON (command parsed) and SSE event streams (LLM fallback) from the conversation endpoint.
Structured Context — Device ID, area, type, user ID, timestamp, and location sent as typed JSON fields instead of embedded in prompt text.
Auto-Generated Conversation IDs — Generates UUID for first-turn conversations, maintaining multi-turn continuity via the tracker.
Optional Prompt Override — Configurable prompt template override in the HA options flow.

🧺 Project Laundry

SDK Pinned — Pinned .NET 10 to the latest appropriate minimal SDK
Removed outdated docs — Removed a lot of outdated docs and old AI assistance templates and prompts.
Package Updates — Updated all central projects to the latest versions from .NET 10.0.4 security patch

🔌 Pluggable Data Providers

The new lucia.Data project introduces provider-agnostic persistence, enabling Lucia to run without Redis or MongoDB:

Cache Providers (replacing Redis):

InMemorySessionCacheService — Multi-turn conversation state with sliding expiration
InMemoryDeviceCacheService — Home Assistant entity caching with TTL support
InMemoryPromptCacheService — Routing and response caching with SHA256 + semantic similarity matching
InMemoryTaskStore — A2A task persistence with configurable TTL and periodic cleanup
InMemoryEntityLocationService — Floor/area/entity graph with hybrid entity matching

Store Providers (replacing MongoDB):

17 SQLite repository implementations covering all 4 MongoDB databases (luciaconfig, luciatraces, luciatasks, luciawyoming)
Hybrid schema: indexed columns for query performance + JSON blob for full document storage
Code-first migrations with SqliteMigrationRunner and versioned schema tracking
SqliteConfigurationProvider with polling-based hot-reload matching the MongoDB provider pattern

Configuration:

{
  "DataProvider": {
    "Cache": "InMemory",
    "Store": "SQLite",
    "SqlitePath": "./data/lucia.db"
  }
}

Interface Abstractions:

IConfigStoreWriter — New interface for configuration CRUD (extracted from ConfigStoreWriter)
ITaskIdIndex — New interface for task ID enumeration (extracted from Redis-specific key scanning)
ConfigSeeder refactored to use IConfigStoreWriter instead of IMongoClient
TaskArchivalService refactored to use ITaskIdIndex instead of IConnectionMultiplexer
All API endpoints updated to use abstractions (no direct IMongoClient or IConnectionMultiplexer injection)

📦 Home Assistant Mono-Container

New infra/docker/Dockerfile.ha for resource-constrained deployment:

Zero Dependencies — No Redis, MongoDB, or GPU drivers required
CPU-Only ONNX — CpuOnly=true MSBuild flag excludes Microsoft.ML.OnnxRuntime.Gpu.Linux, verified ONNX provider detection with graceful CPU fallback
Embedded Dashboard — React SPA built and bundled into wwwroot/ in a multi-stage Docker build
SQLite Persistence — Single /data/lucia.db file for all configuration, traces, tasks, and voice data
Volume Mounts — /data (SQLite DB), /app/models (Wyoming voice models), /app/plugins (script plugins)
HA Add-on Support — ha-addon/config.yaml with ingress, port mapping, and options schema

🔍 ONNX Provider Verification

OnnxProviderDetector now verifies each accelerated provider by attempting SessionOptions.AppendExecutionProvider_*() before selecting it. This prevents sherpa-onnx native crashes when CUDA shared libraries are absent — the detector gracefully falls back to CPUExecutionProvider.

🐛 Bug Fixes

CUDA GPU acceleration not activating — Microsoft.ML.OnnxRuntime NuGet package was CPU-only. Replaced with Microsoft.ML.OnnxRuntime.Gpu.Linux 1.23.2 and aligned managed/native versions (was mismatched 1.23.2/1.22.0), enabling automatic CUDAExecutionProvider detection on hosts with NVIDIA GPUs. Docker voice image verified working with RTX 4090 + CUDA 12.8 + cuDNN 9.20.
Wyoming describe response missing STT — WyomingServiceInfo injected a single ISttEngine? via DI, which resolved to the last registered engine (SherpaSttEngine). If that engine wasn't ready, STT was omitted from the info response even when HybridSttEngine was ready. Now injects IEnumerable<ISttEngine> and reports STT available if any engine is ready.
Parakeet model download "not found" — ModelCatalogService.GetModelById(string) only searched EngineType.Stt, but Parakeet TDT is registered as EngineType.OfflineStt. Now searches all engine types. Also fixed the download endpoint to resolve the model base path per engine type instead of hardcoding the STT path.
Activating Parakeet crashes server — ModelManager.SwitchActiveModelAsync(string) hardcoded EngineType.Stt, causing the streaming OnlineRecognizer to load an offline transducer model (native crash: 'window_size' does not exist in the metadata). Now resolves engine type from the catalog, routing offline models to HybridSttEngine correctly.
Cannot switch back to streaming STT after activating offline model — Both engines remained ready with no user preference tracking, so FirstOrDefault(e => e.IsReady) always picked HybridSttEngine (registered first). Added ModelManager.PreferredSttEngineType that tracks the user's last activation choice. Status endpoint, session engine selection, and active model API all respect the preference.
HuggingFace API key not persisting in dashboard — ConfigurationPage.entriesToValues stripped only the first colon segment from stored keys (e.g., Wyoming:HuggingFace:ApiToken → HuggingFace:ApiToken). For nested config sections, this didn't match the schema property name ApiToken. Fixed to strip the full section prefix.
Speaker identification always returning unknown — Speech enhancement was altering audio used for embedding extraction, causing ~0.39 cosine similarity against enrollment profiles. Now uses raw (unenhanced) audio for speaker verification.
Verification threshold changes from GUI ignored — SpeakerVerificationThreshold config was never passed to IdentifySpeaker(). Now read via IOptionsMonitor.CurrentValue at identification time.
Embedding dimension mismatches silently failed — After model changes, CosineSimilarity threw for mismatched dimensions, caught by outer try/catch returning null. Now gracefully skips with a warning log.
Voice config written to local JSON file — VoiceConfigApi was writing to voiceconfig.json instead of the platform's MongoDB ConfigStoreWriter. Migrated to match established pattern.
Thread pool deadlock in STT finalization — GetFinalResult blocked the thread pool. Converted to async with proper continuation.
Aspire resilience timeout killing model downloads — 3-minute default timeout was insufficient for large model downloads. Bypassed Aspire service discovery for external URLs.
Progressive re-transcription mixing raw and enhanced audio — STT now always receives raw audio; enhanced audio used only for clip storage.
WAV protocol wire format corrections — Proper transcribe/transcript event ordering per Wyoming protocol spec.

⚡ Performance

Hybrid STT achieves ~90ms finalization latency at 0% WER on benchmark audio
Redis-cached speaker profiles reduce diarization lookup from ~500ms to ~1ms
Short command fast-path skips progressive re-transcription overhead
ONNX model warmup on startup eliminates cold-start inference latency
CUDA 12.8 + cuDNN 9 GPU Dockerfile for accelerated voice inference
Workstation GC mode and thread pool tuning for development responsiveness
OTEL export frequency reduced from 1s to 5s to lower telemetry overhead

📊 Test Coverage

Area	Tests
Wyoming session integration	50+
Speaker verification components	17
Speech enhancement validation	20+
Engine hot-reload	10+
Engine readiness	10+
Wyoming status API	2
Command pattern matching	30+
Profile store (InMemory)	10+
Conversation command processor	5
Direct skill executor	4
Response template renderer	4
Context reconstructor	4
Total Wyoming Tests	230
Total Conversation Tests	17

🔄 Upgrade Notes

New infrastructure dependencies — MongoDB (luciaconfig database) is used for speaker profiles, voice config persistence, and response templates. Redis is optional but recommended for profile caching.
Home Assistant component upgrade — The HA custom component has been migrated to v1.2.0-preview.1. Breaking change: remove and re-add the Lucia integration in Home Assistant. The new setup flow only requires the host URL and API key — agent selection is no longer needed.
Response templates — Default templates are seeded automatically on first launch. Customize them via the Response Templates page in the dashboard or POST /api/response-templates.
Model downloads required — On first launch, navigate to the Voice Platform → Models tab to download and activate at least one STT model and supporting models (VAD, Wake Word, Speaker Embedding).
Wyoming integration — Add the Lucia Wyoming satellite in Home Assistant under Settings → Devices & Services → Add Integration → Wyoming. The server advertises via Zeroconf automatically.
GPU acceleration — The project now ships with Microsoft.ML.OnnxRuntime.Gpu.Linux for automatic CUDA support. For local development, install CUDA Toolkit 12.x and cuDNN 9.x. The OnnxProviderDetector will find and use CUDA automatically — no configuration required. The Docker voice image (Dockerfile.voice) includes all GPU dependencies out of the box.
Existing voice config — If you previously had a voiceconfig.json, those settings will need to be re-entered through the dashboard Voice Platform config panel (they now persist to MongoDB).
No breaking changes for data providers — Default behavior (Redis + MongoDB) is unchanged. The new DataProvider configuration section is optional; omitting it preserves existing behavior. Set DataProvider:Cache to InMemory and DataProvider:Store to SQLite to switch to the embedded providers.

Navigation Menu

v1.2.0-preview.14 - Pulsar

Choose a tag to compare

Sorry, something went wrong.