feat(uia): proper Narrator support — host provider, system caret, full Text pattern#48
Merged
Merged
Conversation
…eaders Foundation for real Narrator support (T2-14). Two root-cause fixes verified with a UIAutomationClient probe: - GetHostRawElementProvider now returns UiaHostProviderFromHwnd instead of 0. The provider was UNPARENTED — Narrator couldn't anchor its focus rectangle (it landed somewhere random) and notification/property events didn't route reliably (Settings interactions weren't spoken). Now the element reports a real BoundingRectangle and the HWND Window/Transform patterns, and events route through the hosted element. - System caret (CreateCaret/SetCaretPos/HideCaret, WM_SETFOCUS/WM_KILLFOCUS): a hidden caret tracks the active pane's text cursor each paint, so Magnifier "follow the text cursor", IME composition placement, and screen-reader caret queries have a correct position. Hidden so it never double-draws over our rendered cursor. Probe confirms: ControlType=Document, LocalizedControlType="terminal", Name=live screen text, BoundingRectangle=window rect (was empty). Tight per-caret focus + line navigation is the follow-on ITextProvider stage. 110/110 Core, 38/38 Pty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(proper Narrator support) Stage 2 of T2-14. Implements ITextProvider + ITextRangeProvider so Narrator/ NVDA can read and navigate the terminal like a document and box the caret where you type — the fix for "Narrator reads the prompt but not the output" and "the focus rectangle lands somewhere random." - The active pane's recent scrollback (capped 2000 lines) + live screen are flattened into one document (lines joined by '\n'); a text range is a [start,end) char span. Program.BuildUiaTextSnapshot builds it under the session lock with the caret offset + a document-line -> on-screen-row map. - ITextRangeProvider: GetText, ExpandToEnclosingUnit (char/word/line/document), Move / MoveEndpointByUnit (char + line), GetBoundingRectangles (screen rects for the visible portion — degenerate caret range -> a 1-cell box), GetSelection (caret), GetVisibleRanges, FindText, Clone/Compare/CompareEndpoints. SAFEARRAY builders (VT_R8 rects, VT_UNKNOWN range arrays) for the source-gen COM out-params. - GetPatternProvider advertises the Text pattern (10014); a text-changed event fires on new output so the reader re-reads. Verified with a UIAutomationClient probe (the identical COM path Narrator uses): TextPattern advertised; DocumentRange.GetText returns the buffer; GetSelection -> caret; ExpandToEnclosingUnit(Line) reads the caret line; GetBoundingRectangles -> a tight one-cell box AT the caret (737,330,14,21); Move(Line,+1) walks the buffer line by line. 110/110 Core, 38/38 Pty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Delivers the "proper Narrator support" the terminal was missing — driven by live testing (Narrator read the prompt but not
diroutput; the focus rectangle landed somewhere random; Settings weren't narrated).Three layers, all verified with a
UIAutomationClientprobe (the same COM path Narrator uses)1. Host provider (routing + anchoring)
GetHostRawElementProvider→UiaHostProviderFromHwnd(was 0). The provider was unparented, so Narrator couldn't anchor its focus box and events didn't route (Settings silence). Now it reports a realBoundingRectangleand hosts properly.2. System caret (Magnifier / IME / caret follow)
Hidden
CreateCaret/SetCaretPostracks the active pane's text cursor each paint — Magnifier "follow the text cursor" and IME placement now work. Hidden so it never double-draws over our rendered cursor.3. Full Text pattern (
ITextProvider+ITextRangeProvider) — the big oneThe pane's recent scrollback + live screen flatten into one document. Ranges support
GetText,ExpandToEnclosingUnit(char/word/line/document),Move/MoveEndpointByUnit,GetSelection(caret),GetVisibleRanges,FindText, andGetBoundingRectangles— a degenerate caret range returns a one-cell box at the caret, which is what makes the focus rectangle track where you type. A text-changed event fires on new output.Probe results
110/110 Core, 38/38 Pty.
Still Stage 3 (backlog)
Word-move refinement, live selection via Select(), and alt-screen TUI (Far/vim) reading — those redraw in place and need a different model. This covers reading, caret tracking, and line/char navigation — the core of using Claude Code with a screen reader.
🤖 Generated with Claude Code