Persona Engine v3.0.2

Patch Notes — v3.0.2 🔧 Startup reliability

Short one. v3.0.1 shipped the first-run installer, but a handful of users on fresh machines hit llama.dll: DllNotFoundException on the very first launch — the bootstrapper downloaded CUDA correctly, we just tried to use it one step too early. This release fixes that (and a couple of other load-time gotchas). Full diff: v3.0.1...v3.0.2.

Important

If v3.0.1 crashed for you with a DllNotFoundException pointing at llama.dll or cudart64_12.dll, drop v3.0.2 next to the existing Resources/ folder and launch. No reinstall needed.

🐛 Fix: fresh-install DLL load order (#26)

Program.Main was pre-loading runtimes/win-x64/native/cuda12/llama.dll before running the bootstrapper and registering the CUDA redistributables. Users without a machine-wide CUDA install hit a DllNotFoundException at startup because llama.dll's transitive deps (ggml-cuda.dll → cudart64_12.dll, cublas64_12.dll) couldn't resolve: the bootstrapped DLLs existed on disk under Resources/cuda/, they just weren't mapped into the process yet.

Fixed by reordering startup:

1. Bootstrap runs first (downloads / verifies CUDA redistributables).
2. PreloadCudaRuntime pins cudart / cublas / cufft / cudnn into the loader.
3. Then PreloadLlamaBackend and ONNX Runtime GPU init run — their imports now resolve against the bootstrapped copies.

While we were in there, Program.cs got split for SRP: NativeLibraryLoader owns every CUDA / LLama preload P/Invoke, LoggingConfiguration owns Serilog + global handlers + the LLama log bridge, and Program stays at high-level orchestration only (~120 lines, down from 285).

🎨 Shader robustness (#27)

Every HLSL / GLSL shader now lives on disk under Resources/Shaders/ and flows through a new ShaderRegistry with:

• ASCII validation. Rejects non-ASCII characters (em-dashes, smart quotes, degree signs) with a precise error naming the offending character, line, and column. A recurring source of cryptic "unexpected end of file" failures on users' machines that didn't reproduce locally — IDE autoformat would silently substitute an em-dash and HLSL's code-page-dependent marshalling did the rest.
• #include preprocessor with relative-path resolution and cycle detection.
• First-call cache. Shaders are read, validated, preprocessed, and cached once.

ImGui, RouletteWheel, TextRenderer, and every overlay D3D11 pipeline (Arrow, Button, Outline, Icon, Quad) now load through the registry instead of duplicating const string blobs in their .cs files.

🛠️ Dev escape hatch: skip bootstrapper (#28)

Running from Visual Studio / Rider / dotnet run no longer triggers the first-run installer against a populated checkout.

• New --skip-bootstrap CLI flag.
• New PERSONAENGINE_SKIP_BOOTSTRAP env var (1 / true / yes enable it).
• Properties/launchSettings.json sets the env var automatically for IDE / dotnet run launches, with a second "(with bootstrap)" profile if you want to exercise the installer.

End users are unaffected — the published executable still runs the bootstrapper normally.

🔧 Under the hood

• Release pipeline hardening. build-release.ps1 now verifies every shipped shader (GLSL + HLSL) is present before zipping, on top of the fonts + prompts check added in v3.0.1 — catches the "csproj Content glob matched nothing" regression from either side of a directory layout change.
• ConfigWriter.Flush() — new public sync-drain API. Useful for host shutdown paths that need the on-disk file up-to-date before exit, and it makes the debounce test suite deterministic instead of racing the Timer callback against ThreadPool starvation on CI.
• Config template refresh. Default model + current-context fields bumped in appsettings.template.json.
• README. Embedded 3-minute install walkthrough video.

Persona Engine v3.0.1

Patch Notes — v3.0.1 ✨ Installer, UI, and a new voice

Hey everyone! This one's been cooking for a while. Since v2.0.0 the engine has grown a proper first-run installer, a completely reworked control panel, a second TTS engine, higher-fidelity lip-sync, and a built-in transparent overlay so you can hang out with Aria without launching OBS. Highlights below; full diff is v2.0.0...v3.0.1.

Important

If you're upgrading from v2.0.0: the asset layout changed when the installer landed. Your old Resources/Models/ and Resources/Live2D/Avatars/ folders are ignored — the installer re-downloads into the new locations on first launch. Free up ~16 GB and feel free to delete the old folders once it finishes.

🧳 First-run installer (#22)

Release zips used to be multi-gigabyte bundles with every model and CUDA DLL baked in. Not anymore. Now you download a lean runtime, double-click PersonaEngine.exe, and pick a profile — the app handles the rest.

• Three profiles: Try it out (smallest), Stream with it (balanced), Build with it (largest, highest quality).
• Verified, resumable downloads. Every asset is SHA-256 checked, tag-pinned on HuggingFace, and resumes on interruption.
• NVIDIA runtime included. CUDA 12.4 + cuDNN 9.1.1 + CUDA 13 redists are pulled from NVIDIA's CDN on demand — no more machine-wide CUDA install.
• Flags for power users: --profile=try|stream|build, --reinstall, --repair, --verify, --offline, --non-interactive, --skip-gpu-check.
• GPU preflight. The installer detects your CUDA/driver level and warns if the chosen profile won't fit.
• Features gate themselves. RVC, Audio2Face, and Vision only wire up when their assets are actually present — no silent failures, no crashes.

Tip

Picked Build with it? You still have to flip the switches — the profile downloads the bigger models, but the UI defaults to the light ones. Head to Voice → Expressive, Listening → Accurate, and Avatar → Audio2Face to activate them. Full walkthrough in INSTALLATION.md.

🎛️ Control panel rework (#21)

The control panel got the full makeover treatment. New widgets, new layout, new panels.

• Dashboard first. Presence strip of subsystem health cards (LLM, TTS, Mic, Listening) plus a controls row for cancel / retry / mute.
• Dedicated panels for LLM Connection, Voice, Listening, Avatar, Subtitles, Screen Awareness, and the new Overlay.
• Live reachability. LlmConnectionProbe hits /models on your endpoint so you know whether Groq/Ollama/OpenAI is actually answering before you speak.
• Cancel and retry. The FSM grew a Cancelled state and RetryRequested trigger — if a turn goes sideways, you can stop it or retry it from the dashboard.
• Mic mute gating. Calibration and setup screens quietly hold the mic so you don't get transcribed while fiddling.
• Hot-reloadable LLM. Change endpoint or model while idle and the kernel swaps in without restarting. (Blocked while a turn is active — no mid-sentence surprises.)
• Subtitle WYSIWYG. Live preview renders through an FBO so you can see your styling changes as you type.
• Shared widget library: status chips, endpoint pickers, model pickers, toggle switches, live meters, pill badges.

🪟 Built-in overlay (#21)

You don't need OBS to see Aria anymore. The engine ships a transparent, always-on-top window that mirrors the avatar render target.

• D3D11 + DirectComposition backbone.
• Drag to move, drag the border to resize, reset position/size from the panel.
• Stateless-managed state machine so show/hide/move/resize stay consistent.

OBS + Spout still works exactly the same for streamers.

🗣️ Qwen3 TTS engine (#15)

A second TTS backend, living alongside Kokoro. Switch between them in the Voice panel.

• Qwen3-TTS via llama.cpp with top-P nucleus sampling and CTC forced alignment for word-level timing.
• Unified TTS abstraction — ISentenceSynthesizer / ISynthesisSession so the orchestrator is now a thin coordinator instead of a Kokoro-specific blob.
• Phonemizer moved upstream into SentenceProcessor so every engine gets pre-computed, aligned PhonemeResult.
• Kokoro hardening along the way: buffer corruption fix, off-by-one EOS fix, ArrayPool lifetime fix, thread-safe session locking.

👄 Audio2Face lip-sync (#16)

New higher-fidelity lip-sync option for avatars that need it.

• Streaming ONNX processor with ARKit-to-Live2D blendshape mapping.
• BVLS and PGD solvers for constrained blendshape fitting, with parameter smoothing and sync compensation.
• IoBinding inference + pre-allocated tensors → ~8 MB saved per window.
• Hot-reloadable — flip between VBridger and Audio2Face from the Avatar panel.
• Centralised OnnxSessionFactory for all ONNX session creation across the codebase.

🧠 Conversation & LLM plumbing

• OpenAI-compat endpoint validation at startup and on hot-reload — catches typos in your endpoint before they turn into silent failures.
• Polly resilience on LLM calls (60s timeout, 3 retries, exponential backoff) via named HTTP client.
• FSM polish: StateChanged dispatch now serialised in transition order; Error → Idle recovery path; hierarchical substates for turn-scoped cancellation.
• Self-contained publish. The release exe has no .NET Runtime prereq, and no machine-wide CUDA dependency beyond what the bootstrapper pulls.

🔧 Under the hood

• God-class decomposition. ConversationSession split into partial files with dedicated collaborators (TurnMetricsTracker, LlmStreamHandler, TtsAudioHandler, …). PhonemizerG2P decomposed into SRP classes (TokenRetokenizer, TokenAligner, PhonemeApplier).
• Shared utilities: SpanMathExtensions, StringDistanceExtensions, TextExtensions replace half a dozen copy-pasted helpers.
• Utils reorganised into Audio/, IO/, Numerics/, Pooling/, Text/ namespaces; dead code (AsyncQueue, AtomicCounter, WavUtils, AdapterManagerService) removed.
• EmotionProcessor rewrite using character-offset resolution instead of marker injection.
• ModelId replaces ModelType enum with a type-safe nested system.
• Audio: sub-window RMS sampling for smooth mic meter; new FloatRingBuffer utility.
• CSharpier v1.0.0 formatting enforced in CI.
• Tests: new PersonaEngine.Lib.Bootstrapper.Tests project (downloader, zip extractor, NVIDIA manifest, Spectre UI), plus wider Lib test coverage (SentenceProcessor, subtitle timeline, CTC, lip-sync, utilities, asset catalog).

🐛 Notable fixes

• v3.0.1 hotfix (#24): shipped Resources/Fonts/, Resources/Shaders/, and Resources/Prompts/ in the release zip (missing from v3.0.0), plus a post-publish sanity check so it can't silently regress again.
• Latency tracking only counted the first LLM token — now covers the full response.
• Silero VAD ONNX session switched to sequential/CPU execution after stability issues on some GPUs.
• Lip-sync race on the animation thread, stale pipeline event handling, silence-influenced frame replacement, ONNX disposal ordering.
• ConfigWriter shutdown race that could crash on exit.
• Concurrent modification during sentence-scoped phoneme tracking.

📚 Docs

• INSTALLATION.md rewritten around the installer and the three profiles.
• CONFIGURATION.md (new) — every appsettings.json field annotated.
• README restructured around the new UI with a panel screenshot grid, profile comparison table, and collapsible depth.
• assets-source/README.md documents the HuggingFace publish pipeline for maintainers.

---

Full changelog: v2.0.0...v3.0.1

Come say hi on Discord — happy to help with setup, rigging, or just to show off the fine-tuned LLM live.

Persona Engine V2.0.0

Patch Notes - April 19, 2025 ✨ So Much New Stuff! ✨

Hello everyone! I've been a super busy bee 🐝, and I'm thrilled to bring you this update! It includes some really big behind-the-scenes changes I made to make everything run smoother, plus some sparkly new features and fixes. Let's dive in!

🚀 Mega Makeover: Shiny New Architecture! (#4)

I've given the core way conversations are handled a major glow-up! ✨ It's now built around a clever state machine and uses nifty event channels for communication.

Why the change? Honestly, the old way was getting a bit tangled! Too much was happening in one place, making it tricky for me to manage, update, and keep things running happily side-by-side.

What's new?

• Specialized Helpers! Instead of one overworked manager, there's now a system of specialized components (like Input Adapters, a Transcription Service, an Utterance Aggregator, and more!) that chat with each other using events. 💌
• Benefits Galore:
  • State Management Zen: Keeping track of the conversation state is now neatly handled by the ConversationOrchestrator. Phew! 🙏
  • Smooth Operations: Components handle their async tasks gracefully, and events make coordination a breeze.
  • Interrupt Power! Handling interruptions (barge-in) and cancellations is much cleaner now.
  • Ready for the Future: Adding new ways to interact (like text input or cool function calls) will be much easier for me! 🚀

✨ Sparkly New Features!

• Memory Magic: Introducing ConversationContext to remember all the chat details and turns, ConversationSession to manage the whole chat lifecycle like a pro, and the ConversationOrchestrator to juggle multiple chats at once! (07a0b3d)
• Speed Tracker: I'm now keeping an eye on how fast the LLM, TTS, and audio bits are working! ⏱️ (040c9f6)
• Mic Check, 1, 2! 🎤
  • You can now configure your microphone right from the UI! (8b86527)
  • Settings update on the fly! How cool is that? (b5bc87a)
• "Excuse Me?" - Barge-In Power! Now you can interrupt the assistant much more smoothly, thanks to improved detection! (b97bfa9) Plus, you get new settings to control exactly when and how interruptions happen. Your chat, your rules! (e52d756)
• Audio Updates:
  • Added a StopPlaybackAsync button for audio streams. Handy! (ea2bfa5)
  • Fixed the TTS Config Editor playback to work with the new system. Hooray! (270b857)
• ASR Goodies: More options for Automatic Speech Recognition, including new Whisper settings! 🗣️ (bac3d91)
• Live2D Magic & Meet Aria! 🎨💖
  • Added services for Emotions and Idle/Blinking animations for Live2D models! Get ready for more expressive characters! (0148ec4)
  • Made LAppModel even smarter at understanding motion groups. (d9d0a2d)
  • Built a whole system just for tracking emotions! (770b622)
  • 🌟 Ta-da! Meet Aria! 🌟 To show off all these amazing new Live2D possibilities (like emotions and lip-sync!), I've included a brand new, fully rigged demo model named Aria! This little cutie (who's maybe just a tad smug and sarcastic 😉) was drawn and rigged entirely by hand, from scratch, by me, learning Live2D just for this! 🥰 It took a ton of effort and love, so please check her out and see the new features in action!
• Lip Sync Upgrade: Added shiny new VBridger-based lip sync! (9e27050)
• Rendering Sparkle:
  • Render components now have priorities! (9e27050)
  • Added support for rendering emojis in the GUI! 🎉🥳🎈 (15f67e5)

🛠️ Tinkering & Tidying Up

• Mic Input Polish: Made the microphone input code a bit simpler and tidier. ✨ (753ab69, 789ec27)
• Audio Player Lifecycle: Made the audio player easier to manage behind the scenes. (89e2c12)
• Code Cleanup: Removed some old viseme bits that weren't needed anymore. Spring cleaning! 🧹 (9e27050)
• Live2D Tweaks:
  • Updated motion priorities and added clearer comments. (d9d0a2d)
  • Adjusted how models are loaded in the Live2DManager. (d9d0a2d)

🐛 Bug Squashing!

• TTS Chit-Chat: Fixed a funny bug where the TTS system would sometimes read speaker tags out loud. Oops! 🤭 (3b1f78e)
• Lost & Found Links: Updated some links in the instructions so they actually go to the right place! (bff6851)

📚 Story Time: Documentation Updates

• README Refresh: Gave the main README file a nice makeover. (b4fc703)
• Live2D Guide: Added a guide to help with Live2D rigging! 🎨 (b4fc703)
• Clearer Instructions: Updated links in the usage instructions for less confusion. (bff6851)

---

I hope you love all the changes and new features! Especially give a warm welcome to Aria! Let me know what you think! 💖

PersonaEngine_v1.0.0

Clarify install process README.md

Whisper Models

Whisper Speech Recognition Models

High-Performance Model (e.g., ggml-large-v3-turbo.bin):

• Models like large-v3 or the optimized large-v3-turbo offer high accuracy for speech recognition.
• However, larger models require significant computational resources (CPU, RAM, VRAM).
• This resource demand can lead to higher latency (slower processing times). The large-v3-turbo variant is a distilled version of large-v3, designed to be faster with a minor trade-off in accuracy.

Improving Latency with Alternative Models:

• If lower latency (faster processing) is a priority, especially on less powerful hardware, consider using smaller or quantized Whisper models.
• These models trade some accuracy for reduced size and faster inference speed.
• Common sizes include: tiny, base, small, medium. Quantized versions (e.g., q5_0, q8_0) further reduce resource usage.

Where to Find Models:

You can download various pre-converted Whisper models in the ggml format from Hugging Face repositories:

1. https://huggingface.co/sandrohanea/whisper.net/tree/main
2. https://huggingface.co/ggerganov/whisper.cpp/tree/main

Warning

If you download an alternative model from the Hugging Face links provided (i.e., any model other than the default one provided below, such as ggml-base.bin, ggml-small.bin, or a quantized version):

• You must rename the downloaded file exactly to: ggml-large-v3-turbo.bin

This renaming step is crucial because the engine is configured to load only a file with this exact name. Failing to rename the alternative model file will likely result in the application being unable to find and load it.