Stage 8: UI Automation integration (fixes #1)#2
Merged
zergius-eggstream merged 5 commits intomainfrom Apr 18, 2026
Merged
Conversation
…tings UI Prep work for #1. No UIA behavior yet; just the scaffolding: - Cargo.toml: enable `Win32_UI_Accessibility` and `Win32_System_Com` features in the `windows` crate. - config: new `[general] use_uia = true` toggle (default on). Old configs without the field deserialize fine via the serde default. - settings window: new checkbox under "Start with Windows" titled "Use UI Automation for reading selections", with a sub-caption "(uncheck if switching behaves oddly in some program)". Window height bumped by 46 px to fit the checkbox and caption. - save_settings: reads the new checkbox and persists to config. Refs #1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Primary user-visible win: the smart-copy bug in Notepad Win11, VS Code,
and Electron editors is gone when UIA is available. `perform_conversion`
now asks UIA for the actual selection before touching the clipboard, so
an unselected current line can never be confused for an explicit
selection.
- New `src/uia.rs`: thread-local COM init, safe wrappers for
`IUIAutomation`, `IUIAutomationElement`, `IUIAutomationTextPattern`,
`IUIAutomationTextRange`. COM objects are apartment-threaded, so the
cache lives in a `thread_local!` `RefCell` on the main message-loop
thread where init and all read calls happen.
- `main.rs`: call `uia::set_enabled(config.general.use_uia)` then
`uia::init()` at startup. If the checkbox is off or init fails,
the helpers short-circuit to `None` and callers fall back.
- `conversion.rs`:
* `perform_conversion` tries `uia::get_selected_text()` first.
Falls back to clipboard + `Ctrl+C` only when UIA didn't return
text (no focused TextPattern element, empty selection, or
toggle off).
* `perform_word_conversion` tries `uia::select_word_at_caret()`,
which expands the caret's zero-length range to a word unit
and calls `Select()` — the subsequent paste replaces just that
word. Falls back to `Ctrl+Shift+Left` + `Ctrl+C` when UIA
isn't available.
- Paste still goes through clipboard + `Ctrl+V`. This is deliberate:
`ValuePattern.SetValue` support varies wildly across apps, while
clipboard paste is universally reliable, and by this point we
already know the real selection so there's no smart-copy risk.
Refs #1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two user-reported bugs with the UIA word path: 1. In Notepad Win11 and Word, UIA's TextUnit_Word includes the trailing space, so after paste the caret lands past the space, inside the next word. Cycling then re-converts the wrong word. 2. UIA's TextUnit_Word splits on punctuation. A converted "ghb,jh" (ua "прибор" → en) is two UIA-words. Pressing the hotkey again cycles only one half: user sees "ghb,ор" instead of round-tripping back to "прибор". Switch to whitespace-delimited definition: expand the caret range to TextUnit_Line, read line text, locate the caret column by reading the prefix range, then walk outward in Rust until whitespace is hit on each side. Then shrink line_range's endpoints by the measured prefix/tail char counts to end up with exactly [start_col, end_col). Result: - `ghb,jh` is one unit → converts to `прибор` round-trip. - Trailing/leading whitespace never included, caret lands inside the pasted text, cycling works in Notepad/Word/everywhere. - Caret on whitespace → returns None, falls back to clipboard flow or does nothing (matches earlier no-op semantics). Refs #1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Notepad++ and a few other editors don't expose a UIA TextPattern on the focused edit control, so the word-conversion fallback path kicked in — which used OS Ctrl+Shift+Left. That stops at punctuation, so "ghb,jh" (round-trip of "прибор") ended up as two separate words and cycling only converted one half. The UIA path already uses a whitespace-delimited definition of "word" (commit 436b980). This commit teaches the clipboard fallback the same trick, without needing UIA: - `input::send_select_to_line_start()` — Shift+Home, one keypress that selects from the caret all the way to the line start. - Read the clipboard, walk the selected chars from the right until the first whitespace → that's where our word begins. - `input::send_select_n_right(n)` — Shift+Right × N to shrink the selection from the left, so only the word remains highlighted. - Then hand off to `convert_and_paste` as usual; Ctrl+V replaces the shrunk selection with the converted text. Result: in Notepad++ (and any editor without UIA), "прибор" now round-trips correctly — `[fallback] word:` line in the log replaces the old `[uia] selected word:`. Dead code: the original `send_select_word_left` (Ctrl+Shift+Left) is no longer used anywhere and has been removed. Refs #1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Bumps the crate version and snapshots the CHANGELOG. The 0.3.0 section covers everything on the feat/stage-8-uia branch — infrastructure, UIA read path, whitespace-bounded word selection (both UIA and clipboard-fallback flavours), the About dialog, the MIT license, and the GitHub Actions workflows. Build metadata (Cargo.toml version + build.rs timestamp) now lines up with the tag the next commit will create: when the user clicks "About..." in the tray they'll see "LightSwitch v0.3.0". Refs #1. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft PR for Stage 8 work tracked in #1.
What's in this PR so far
Commit 1 — infra only:
Cargo.toml: enabledWin32_UI_AccessibilityandWin32_System_Comfeatures in the
windowscrate.src/config.rs: new[general] use_uia = truetoggle (default on).src/ui.rs: new checkbox in the settings window — "Use UI Automationfor reading selections", with caption "(uncheck if switching behaves
oddly in some program)".
No behavior change yet: the toggle is saved to config but nothing reads
it. UIA wiring lands in the next commits.
What still needs to land before merge
src/uia.rs— safe Rust wrappers overIUIAutomation,IUIAutomationElement,IUIAutomationTextPattern,IUIAutomationTextRange. COM init/shutdown lifecycle.perform_conversionandperform_word_conversion. Fall back to clipboard+SendInput whenUIA isn't available or the user disabled it.
Chrome address bar and web textarea, Word, Telegram, Outlook.
Out of scope (tracked separately)
DocumentRange— separate future feature.ValuePattern.SetValuefor single-line fields — optimization, later.Acceptance criteria — see #1 for the full list.