[codex] Harden LLM command decoding

[IchenDEV]

Summary

• Harden local ASR transcript extraction across realistic provider shapes: segment arrays, word/token/element payloads, monologues, stable/final transcript wrappers, channels, n-best/alternatives/candidates/beams, ASR-specific result wrappers, JSONL and data: event wrappers, serialized JSON payload wrappers, runner JSON log envelopes, and final/partial metadata.
• Parse ASR-specific nested result wrappers such as asr_result, transcription_result, and recognitionResult so runner envelopes do not hide the actual transcript.
• Parse stable/unstable and final/partial wrapper payloads so streaming ASR adapters that separate committed text from tentative text return the stable/final transcript without treating boolean metadata as transcript content.
• Parse serialized ASR payload strings inside structural wrapper fields such as result, data, and payload, while preserving literal JSON-like dictation when no transcript signal is present.
• Ignore JSON runner log envelopes that expose fields such as level, logger, or severity without transcript signals, so startup/progress messages are not inserted as dictated text.
• Make nested ASR wrapper signal detection value-aware: wrapper keys such as data, message, payload, and stable/final wrappers now count only when their nested object, array, or serialized JSON contains a real transcript signal, so plain log strings are skipped while log-shaped transcript payloads are preserved.
• Share direct ASR transcript signal detection across JSONL and structured-output paths, including aliases such as normalized_text, recognizedText, recognizedPhrases, candidate/beam lists, typed element arrays, monologue wrappers, stable/final wrappers, and nested result wrappers.
• Parse typed ASR element payloads such as { "type": "text", "value": "..." } and punctuation elements, while leaving untyped value metadata alone.
• Parse Deepgram-style channel.alternatives transcript wrappers and speech_final metadata so final channel hypotheses are selected over interim channel hypotheses.
• Recognize common ASR endpoint/finality boolean aliases such as sentence_end, utterance_end, end_of_speech, and is_eos, plus endpoint string status values such as SentenceEnd, UtteranceEnd, and EndOfSpeech.
• Recognize compound ASR finality status values such as FinalTranscript, PartialTranscript, FinalResult, and PartialResult from type/status/message_type style metadata.
• Join multiple final ASR transcript segments/results instead of keeping only the last one, including results arrays whose final entries contain alternatives, while treating cumulative final updates as replacements so partial/full repeats do not duplicate text.
• Preserve JSONL/SSE transcript fragments that arrive before terminal events such as done, so stream completion markers do not cause the earlier dictated text to be dropped.
• Parse ASR confidence aliases such as conf, confidence_value, confidence_percent, and confidence_pct, plus envelopes such as { "confidence": { "value": "0.93" } } and normalized confidence wrappers, so high-confidence alternatives/candidates can still win when providers vary score fields.
• Share no-space script joining across final ASR output and live streaming preview, including Han plus Japanese Hiragana/Katakana/halfwidth Katakana, so Japanese dictation no longer gains artificial spaces in either final text or preview text.
• Skip tokenizer/model control tokens in token-level ASR output, including metadata-marked special/control tokens and Whisper-style <|...|> markers, so model scaffolding does not leak into dictated text.
• Improve streaming speech preview merging so partial fragments continue naturally, stale or shorter tails are ignored, and CJK/Japanese/Latin spacing stays readable.
• Make remote LLM response parsing tolerant of OpenAI, Anthropic, and Responses API variants, including streamed events, NDJSON event streams, SSE event names carried outside data: payloads, content arrays/objects, typed OpenAI content delta blocks, parsed payloads, singular/plural tool call deltas, Anthropic camelCase stream aliases, Anthropic content-block index aliases and single-block streams without explicit indexes, tool/function arguments, typed final text/content-part done blocks, final-text value envelopes, wrapped replacement payloads, and case-insensitive field names.
• Decode JSON objects and arrays that a model returns as JSON string literals, so double-encoded edit-command payloads like "{...}" still resolve to the intended command while earlier quoted examples do not override later final output.
• Keep edit-command behavior LLM-owned by expanding structured action/target/intent/replacement/confidence decoding instead of adding local hardcoded phrase cleanup, including intent goal aliases such as edit_instruction, rewrite_instruction, goal, and objective, intent preset objects that use type or kind values such as { "intent": { "type": "summary" } }, target aliases such as target, scope, object, and edit_target, target objects such as { "kind": "selection" } and { "entity": "lastInsertion" }, target boolean flags such as { "selection": true } and { "lastInsertion": true }, nested target or scope fields inside action objects such as { "action": { "type": "rewrite", "target": { "kind": "selection" } } }, action parameter containers such as { "action": { "name": "rewrite", "parameters": { "target": { "kind": "selection" } } } }, action boolean flags such as { "rewrite": true } and { "replaceLast": true }, top-level replacement text aliases such as to_text, correctedText, and current, command_type/operationType action aliases, action+target pairs such as replace + last_insertion or rewrite + selection, and percent/percentage/pct/certainty confidence aliases from model-generated command JSON, including certainty/justification when they appear as metadata inside action, target, intent, replacement, or final-text objects.
• Fence focused text, screen context, personal memory, and prompt examples so context can guide the LLM without being mistaken for an executable command.

Why

Real ASR and LLM providers do not always return the ideal shape. They stream partial fragments, emit final segment/result lists, wrap transcripts in provider-specific result/channel/SSE envelopes, serialize inner ASR JSON into wrapper strings, interleave transcript lines with structured runner logs, attach log-like metadata such as severity to real transcript payloads, vary finality metadata names and status values, split no-space-language text into tokenizer pieces, include tokenizer control markers, wrap confidence scores, send terminal events separately from transcript text, name competing ASR hypotheses as candidates or beams, put transcript text in typed value elements under monologues/elements, split committed and tentative text into stable/unstable or final/partial wrappers, name confidence as conf, confidence_value, or percentage fields, put event types in SSE event: lines instead of data: JSON payloads, rename or omit stream block indexes in single-block adapters, double-encode JSON payloads as JSON strings, or return final text and edit-command replacement text through tool arguments, singular tool-call deltas, Anthropic camelCase stream fields, typed OpenAI content delta blocks, content-part done events, nested value envelopes, adapter-specific payload fields, action fields named as command/operation types, edit actions or targets expressed as boolean flags, edit targets named separately as scope/object/target fields, nested target objects, or boolean target flags, intent fields named as goals/objectives/instructions, replacement text fields emitted directly at the top level, and confidence fields named as percentages instead of canonical confidence. The voice path should recover the model's intended transcript or edit command when the signal is present, while keeping semantic cleanup and command decisions with the LLM.

Validation

• swift test --quiet --filter LocalASRStableWrapperTests — 4 passed
• swift test --quiet --filter LocalASRElementOutputTests — 3 passed
• swift test --quiet --filter LocalASRConfidenceTests — 4 passed
• swift test --quiet --filter LocalASRCandidateOutputTests — 3 passed
• swift test --quiet --filter LocalASR — 83 passed
• swift test --quiet --filter LLMStructuredOutputTests — 7 passed
• swift test --quiet --filter SpokenEditCommandQuotedJSONTests — 2 passed
• swift test --quiet --filter SpokenEditCommandLLMRobustnessTests — 19 passed
• swift test --quiet --filter SpokenEditCommandIntentValueTests — 9 passed
• swift test --quiet --filter SpokenEditCommandReplacementValueTests — 6 passed
• swift test --quiet --filter SpokenEditCommandActionValueTests — 8 passed
• swift test --quiet --filter SpokenEditCommandConfidenceValueTests — 3 passed
• swift test --quiet --filter SpokenEditCommandMetadataValueTests — 5 passed
• swift test --quiet --filter FormattedOutputCleanerMetadataTests — 2 passed
• swift test --quiet --filter FormattedOutputCleaner — 24 passed
• swift test --quiet --filter SpokenEditCommand — 72 passed
• swift test --quiet --filter LocalASRJSONLinesOutputTests — 11 passed
• swift test --quiet — 503 passed, 3 skipped
• swift build --quiet
• xcodebuild -downloadComponent MetalToolchain
• ./scripts/build-app.sh --app-only --sign=-
• codesign --verify --deep --strict --verbose=2 dist/OpenType.app
• bash scripts/ci-basic-checks.sh
• git diff --check
• git diff --cached --check