[Field Report] Windows desktop v0.12.0 — MMD view persistence, wake-word initialization, and voice input stability

[xuanhai0]

[Field report][Windows desktop v0.12.0] MMD view persistence, audio lifecycle, wake-word startup, and character UX

Environment

• AIRI desktop: v0.12.0 stable, Windows 11 (x64).
• Avatar: imported PMX model with optional VMD idle animation. No asset names or files are included.
• Speech: an OpenAI-compatible transcription provider and an OpenAI-compatible TTS provider, plus AIRI's built-in wake-word detection. Provider identities and endpoints are intentionally omitted.
• All observations come from hands-on use of the desktop binary, not CI. Reproduction frequency varies.

Summary

I encountered several mostly independent issues while configuring a persistent voice-interactive PMX desktop character. I am reporting them separately below to avoid conflating confirmed defects with possible bugs and feature requests. No proprietary model, avatar identity, voice asset, local path, account identifier, or conversation transcript is included.

Ref	Category	Summary	Evidence / status
A	Bug, locally fixed	Saved PMX X/Y position and scale are loaded, but the model is visually re-centered/resized on reload	Reproducible before local workaround; workaround validated on Windows
B	Possible bug	PMX eyes become progressively narrowed after extended runtime/conversation	Recovers upon avatar reload; root cause not isolated
C	Bug / possible regression	Wake-word detector stays in preparing after app restart	Switching away from and back to the model/control recovers readiness; potentially related to #3091
D	Bug / integration issue	Recognized voice input remains in listening/processing state and is never submitted	Disabling both End-of-Speech Judge and Transcript Rephrase avoids the stall; individual cause not isolated
E	Intermittent error	Capture reader fell behind during microphone transcription	Seen at least once; not yet reliably reproducible
F	Feature request	Remember the last active character card across restarts	Startup returns to built-in default card
G	Feature request	Optionally activate the visible avatar when its wake word is detected	Wake words route sessions but do not activate the visible avatar by design
H	Feature request	Discover voices from OpenAI-compatible speech providers	Voice generation works with a manually entered ID even when UI lists no voices

Most actionable finding: PMX camera re-framing

In the packaged MMD scene runtime, the load path applies persisted position, scale, and rotationY to modelGroup before calling frameCamera(). That camera function computes the bounding box of the already transformed model and re-centers/re-sizes camera framing accordingly. Consequently, persisted transforms can be present in storage and reflected in the UI but visually canceled by the camera. A local proof-of-concept workaround frames the camera with a baseline transform, then reapplies the persisted transform afterward. It restored the intended view across app restarts after a one-time recalibration. This is not an upstream fix and has not been validated on other platforms or versions.

See the expandable detailed report A below for the code-path details and acceptance criteria.

Audio issues

Wake-word preparation sometimes remained pending after cold startup; toggling the selection restored readiness. When both optional LLM-assisted end-of-speech judgement and transcript rephrasing were enabled, a wake-triggered turn could stay in listening/processing state despite visible recognized words. Disabling both resolved the symptom in this environment but does not identify which module failed. One separate capture-buffer exception was also seen.

See expandable detailed reports C, D, and E below.

UX considerations

Re-opening the app selects the built-in default card rather than the last manually selected card. Character wake words correctly route a speech turn to a matching session, but do not activate the visible avatar. For multi-character desktop companions, optional restoration/visible activation would materially improve the experience. Voice IDs from generic OpenAI-compatible providers are similarly usable in TTS requests but may not be listed by the selector.

Related upstream material

• Before opening an issue: Before opening an issue: please search first #2117
• Potentially related KWS CI issue (not proven same cause): bug(testing-audio): wake word test cases time out while preparation stays preparing in the CI. #3091
• Stage UI voice architecture: https://github.com/moeru-ai/airi/blob/main/packages/stage-ui/README.md
• Audio test documentation: https://github.com/moeru-ai/airi/blob/main/packages/testing-audio/README.md

Happy to provide redacted diagnostics or test a targeted patch. I would prefer not to publish my private model/voice assets or unredacted system logs.

---

Detailed reports (expand the section you want to read)

A — [Bug][MMD][Desktop v0.12.0] Saved model offset and scale are visually canceled by camera re-framing after reload

[Bug][MMD][Desktop v0.12.0] Saved model offset and scale are visually canceled by camera re-framing after reload

Environment

• AIRI v0.12.0 stable, Windows 11 desktop.
• Imported PMX avatar, optional VMD idle animation.
• No model-specific information is needed to reproduce.

Steps to reproduce

1. Load an imported PMX avatar in desktop stage.
2. Change the MMD scene's X/Y offset and scale to non-default values.
3. Confirm the new view looks correct; exit AIRI completely.
4. Relaunch AIRI and load the same PMX avatar.
5. Compare the view with the saved values shown in the settings panel and local storage.

Observed

The on-screen model framing does not match the manually adjusted view after reload. The scene controls and their localStorage values do retain the updated X/Y/scale values; the idle VMD loads as expected. Window position is independently restored and is not the issue.

Expected

Reloading an imported PMX should preserve the same visual framing (position and apparent scale) for the same viewport, or explain and persist separate camera framing parameters.

Narrowed cause from packaged v0.12.0 JS

The following is a simplified outline of the existing path in packages/stage-ui-mmd/src/components/scenes/MMD.vue:

modelGroup = new Group()
modelGroup.add(mesh)
applyTransform() // applies saved X/Y/scale to the model
scene.add(modelGroup)
await animation.init()
resize()
frameCamera()  // computes bounds of the already transformed model

frameCamera() derives a transformed world-space Box3 and sets camera.position, controls.target and distance from its center/size. This follows the user's translation/scale and can cancel their visual effect.

Locally verified workaround

An unofficial local modification uses a baseline transform only while calculating initial camera framing, then re-applies the persisted transform afterward. After one-time re-calibration of offsets for that new camera baseline, restarting/reloading the same PMX restored the desired framing. This workaround was verified by a desktop user; it is not a cross-platform fix or an official PR.

The workaround does not modify the PMX, VMD, or saved-value keys. It is still worth validating against orbit controls, resize, physics, and avatar changes before upstreaming.

Suggested tests / acceptance criteria

• Model X/Y/scale persist visually through app restart and PMX unload/reload.
• VMD playback and physics still initialize normally.
• No unexpected zoom or vertical offset changes at first load.
• Camera framing remains predictable for different PMX model dimensions and window aspect ratios.
• Avoid tying the visual result to order-dependent camera re-centering of an already user-transformed object.

B — [Possible bug][MMD] Eye openness gradually changes during extended dialogue/idle sessions

[Possible bug][MMD] Eye openness gradually changes during extended dialogue/idle sessions

Environment

• AIRI v0.12.0 stable, Windows 11 desktop.
• Imported PMX avatar with eye/facial morphs and an idle VMD.

Observation

After an extended period of dialogue and/or remaining on the stage, the avatar's eyes become noticeably more narrowed than immediately after loading. Switching to another avatar and then reloading the original restores normal eye openness.

What is known

• The phenomenon is runtime-dependent and resets with model reload.
• A model-independent root cause is not yet confirmed.
• This report does not claim that numerical morph weights accumulate; a persistent emotion morph, blink/emotion interaction, or idle VMD morph tracks could all be involved.

Suggested investigation

Check whether MMD eye/blink/smile morph influences are reliably returned to neutral after an emotion transition or a voice turn, and whether blink, VMD facial tracks, and expression controllers can write incompatible values to the same morph channel. Comparing morph influences immediately after load versus after a prolonged dialogue would be useful.

Expected

Idle animation and temporary emotions should not leave the avatar permanently squinting after a long session unless a specific persistent expression was intentionally requested. A neutral reset should return eye morphs to their baseline.

Reproduction status

Observed in one imported PMX on the desktop app. Needs isolation with VMD disabled and with emotion events disabled; please treat this as a possible bug, not a confirmed engine regression.

C — [Bug?][Wake word][Windows desktop v0.12.0] KWS preparation remains stuck after app restart; selection toggle recovers it

[Bug?][Wake word][Windows desktop v0.12.0] KWS preparation remains stuck after app restart; selection toggle recovers it

Environment

• AIRI v0.12.0 stable, Windows 11 desktop.
• Wake-word listening enabled and at least one character card contains valid keyword pronunciations.
• Recognition/TTS provider details omitted; KWS uses AIRI's own detector.

Steps to reproduce

1. Configure and successfully test a card's wake word(s).
2. Fully exit the AIRI desktop app.
3. Reopen AIRI, restore the wake-word listening mode.
4. Observe the wake-word model preparation indicator.

Actual

The UI occasionally/consistently remains on preparing / “preparing wake-word model” and does not become ready. Changing the selected speech model/related listening configuration away from and back to its original value triggers initialization again; the detector then becomes ready and the existing wake words can be used.

Expected

A previously configured valid KWS model should initialize on cold start without a manual switch/reinitialization, or produce a bounded timeout and a recoverable error/retry button.

Investigation pointers

Possible startup ordering/race between loading saved card pronunciations, starting capture, creating the KWS worker, and reporting readiness. Please log whether a rejected or cancelled initialization leaves preparing uncleared.

Related report

Issue #3091 concerns KWS preparation timeouts in CI, not necessarily the same issue on Windows desktop: #3091 . This is offered as additional real-world evidence, not proof of identical cause.

D — [Bug?][Voice input][v0.12.0] Wake-triggered transcription remains in listening/processing state and is not auto-submitted

[Bug?][Voice input][v0.12.0] Wake-triggered transcription remains in listening/processing state and is not auto-submitted

Environment

• AIRI v0.12.0 stable, Windows 11 desktop.
• Wake-word capture mode, auto-send enabled, functional speech-to-text provider.
• Both optional End-of-Speech Judge and Transcript Rephrase enabled.

Steps to reproduce

1. Enable wake-word input and auto-send.
2. Enable both End-of-Speech Judge and Transcript Rephrase.
3. Trigger a wake word and speak one short complete sentence.
4. Stop speaking and observe the transcript and listening state.

Actual

The spoken text is recognized and displayed, but the interface remains in listening / processing / arranging state and does not submit the turn. The transcript is not missing; the workflow fails to finish. Turning both optional LLM-assisted controls off allows normal end-of-speech finalization and auto-send in the same setup.

Expected

A completed transcript should either be submitted after a finite silence/processing budget or return to an editable draft with an actionable error if the optional post-processing/judge fails or times out.

Limitations

Disabling both options simultaneously solved this user's symptom; the individual failing step was not independently isolated, and no LLM/provider identification or credentials are included. This could be an external provider timeout that AIRI did not recover from, rather than a defect in the judge algorithm itself.

Suggested diagnostics

Use the built-in IO trace to distinguish Voice activity detection, Speech recognition, End of speech judgement, Transcript rephrase, and Interaction turn. Consider timeout/fallback paths so optional processing cannot permanently block a ready transcript.

E — [Intermittent issue][Voice input][Desktop v0.12.0] "Capture reader fell behind" during transcription

[Intermittent issue][Voice input][Desktop v0.12.0] "Capture reader fell behind" during transcription

Environment

• AIRI stable v0.12.0, Windows 11 desktop.
• Continuously listening microphone / wake-word workflow.

Actual

The voice input UI has shown the message Voice transcription failed: Capture reader fell behind (localized text around this error may differ). Voice capture needed to be restarted to continue.

Expected

Sustained listening should tolerate normal processing and audio playback loads without losing the capture reader. If a backlog develops, the pipeline should safely recover and resume without requiring a full application restart.

Reproduction status and scope

Only an intermittent observation so far. No timestamped full logs or reliable load correlation are available. This is not currently linked causally to TTS activity, KWS initialization, or End-of-Speech Judge; it may merit consolidation into an existing capture-buffer issue rather than a standalone ticket.

Suggested diagnostics

Capture reader lag, ring-buffer/queue occupancy, VAD/STT work duration, audio device changes, and TTS playback overlap; keep logs privacy-safe and without raw microphone audio by default.

F — [Feature][Desktop] Restore the last active character card after restarting AIRI

[Feature][Desktop] Restore the last active character card after restarting AIRI

Current behavior

On Windows desktop v0.12.0, manually activating an imported/custom card works for the current session, but the next full application start selects the built-in default card again. Character card data itself persists; this request concerns the selection.

Proposed behavior

Add an optional preference such as Startup character: Last active / Built-in default / Specific card. In Last active mode, persist the active card ID and restore it after card storage is ready. Fall back safely to the built-in default if a saved card no longer exists or cannot load.

Why this matters

Persistent desktop avatar setups otherwise require repeating a manual card switch after every restart. This is especially inconvenient when the chosen card owns a distinct display model and audio preferences.

Acceptance criteria

• Last selected card is restored after a clean restart if the preference is enabled.
• Stale/missing IDs gracefully fall back to the built-in card.
• Does not duplicate cards or overwrite sessions/voice preferences.
• The default behavior can remain unchanged for users who prefer it.

This is a feature request, not a claim that the existing startup choice is an accidental regression.

G — [Feature][Wake word / Stage] Optionally switch the visible active character to the wake-word owner

[Feature][Wake word / Stage] Optionally switch the visible active character to the wake-word owner

Current behavior

The wake-word system correctly routes an utterance to a chat session owned by the matched card, but the currently activated card and rendered PMX/Live2D/VRM avatar remain unchanged. The stage therefore can show one character while another character responds.

AIRI's audio tests document that the wake selects the session without changing the active card: https://github.com/moeru-ai/airi/blob/main/packages/testing-audio/README.md .

Proposed enhancement

Provide an opt-in "Activate character on wake" setting. When a wake word selects a target card, also activate that card's display model and (where appropriate) its voice preferences without rerouting/cancelling the already accepted input.

Design considerations

• Keep existing session routing semantics and in-progress turn ownership.
• Avoid double-loading on rapid repeated keywords.
• Switching may need to wait for the stage model asset to load; preserve the transcript during transition.
• Keep the feature off by default if seamless transitions are not guaranteed.
• Respect cancellation, audio playback and avatar animation lifecycle.

Acceptance criteria

Calling a configured wake word can, optionally, bring the matching visible avatar on stage while keeping the interaction bound to the correct card and session.

H — [Feature][Speech provider] Surface available voices for custom OpenAI-compatible TTS backends

[Feature][Speech provider] Surface available voices for custom OpenAI-compatible TTS backends

Current behavior

A generic OpenAI-compatible speech provider accepts manually entered voice IDs and can synthesize/play audio successfully. However, the voice selector may show "No available voices" despite the backend exposing a voice catalog endpoint. This is confusing when each character card should use a different voice ID.

Proposal

For generic OpenAI-compatible TTS, offer an optional capability to fetch a provider-specific voice listing (e.g. a configurable or well-known /v1/audio/voices route). Do not assume all compatible services implement that endpoint. When listing is unavailable, retain a clear manual Voice ID field rather than presenting an apparent configuration error.

Acceptance criteria

• Supports manual IDs for compatible providers regardless of discovery support.
• Optional discovery can show IDs with labels, and refresh after files/profiles change on the backend.
• No silent substitution with a default voice when a card's voice ID is missing.
• Per-card voice ID remains stable across switching and restart.

This is a compatibility/UX feature request, not an assertion that the OpenAI API itself standardizes the voice-list endpoint.