Skip to content

Port modifier overlay to Tauri preview - #507

Merged
enaboapps merged 5 commits into
mainfrom
agent/tauri-modifier-overlay-505
Aug 5, 2026
Merged

Port modifier overlay to Tauri preview#507
enaboapps merged 5 commits into
mainfrom
agent/tauri-modifier-overlay-505

Conversation

@enaboapps

Copy link
Copy Markdown
Contributor

Summary

Closes #505.

Ports the shipping Windows modifier-key overlay behavior to the Rust/Tauri preview on macOS and Windows.

Changes

  • Add a dedicated transparent, non-activating, click-through Tauri overlay window with source-compatible styling and active-monitor placement.
  • Show canonical modifier state with Windows labels on Windows and platform-native labels on macOS.
  • Keep modifier tracking and overlay state consistent across duplicates, injection failures, disconnects, accessibility refreshes, Bluetooth unsubscribe, and runtime cleanup.
  • Keep switch-profile held outputs separate from explicit modifier toggles.
  • Add revisioned startup state and focused Rust/frontend coverage.

Validation

  • npm run lint (Node 24.13.0)
  • npm test (8 tests)
  • npm run build
  • cargo fmt --manifest-path src-tauri/Cargo.toml --check
  • cargo clippy --manifest-path src-tauri/Cargo.toml --all-targets -- -D warnings
  • cargo test --manifest-path src-tauri/Cargo.toml (64 tests including config)
  • npm run tauri -- build (macOS .app and DMG)
  • Launched the packaged macOS app and verified the modifier window remains hidden and does not take focus at startup.

Windows compilation and interactive modifier smoke coverage remain delegated to the repository workflow/Windows environment.

@enaboapps

Copy link
Copy Markdown
Contributor Author

@greptileai

@greptile-apps

greptile-apps Bot commented Aug 5, 2026

Copy link
Copy Markdown

Greptile Summary

The PR ports the modifier-key overlay to the Tauri preview and adds explicit modifier-release recovery across lifecycle transitions.

  • Adds a transparent, click-through overlay window with platform-native labels and active-monitor placement.
  • Tracks explicit modifiers separately from switch-profile outputs and retains failed modifier releases for later cleanup.
  • Integrates overlay and input cleanup into macOS and Windows connection, accessibility, and shutdown paths.
  • Adds revision-aware frontend rendering and focused Rust/React tests.

Confidence Score: 4/5

The PR is not yet safe to merge because a failed mouse-button release is forgotten during lifecycle cleanup and can leave dragging stuck at the OS level.

release_held_button deliberately retains held_button when OS injection fails, but the newly added end_control_session immediately clears that state across disconnect and accessibility-refresh paths, eliminating every later retry opportunity.

Files Needing Attention: cross-platform/src-tauri/src/input.rs

Important Files Changed

Filename Overview
cross-platform/src-tauri/src/input.rs Adds typed modifier state and release retries, but session cleanup drops failed mouse-button release state.
cross-platform/src-tauri/src/macos.rs Integrates modifier-overlay cleanup and preserves the existing input object when accessibility-refresh releases fail.
cross-platform/src-tauri/src/windows_runtime.rs Connects modifier overlay state to the Windows runtime and performs input-session cleanup on disconnect.
cross-platform/src-tauri/src/modifier_overlay.rs Implements the dedicated overlay window, canonical labels, revisioned state, placement, and lifecycle notifications.
cross-platform/src/ModifierOverlay.tsx Implements revision-aware overlay event handling and rendering for the dedicated Tauri route.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Modifier command] --> B[DesktopInput state]
  B --> C[OS input injector]
  B --> D[Modifier overlay notifier]
  D --> E[Tauri overlay event]
  E --> F[React overlay view]
  G[Disconnect or runtime cleanup] --> H[release_all]
  H --> I[end_control_session]
  I --> D
Loading
Prompt To Fix All With AI
### Issue 1
cross-platform/src-tauri/src/input.rs:987
**Failed button releases lose state**

When an OS button-up injection fails during disconnect, accessibility refresh, Bluetooth unsubscribe, runtime reset, or shutdown, `release_held_button` retains `held_button`, but `end_control_session` immediately clears it. The OS button remains pressed while subsequent cleanup has no state with which to retry the release, leaving dragging stuck.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (6): Last reviewed commit: "Merge branch 'main' into agent/tauri-mod..." | Re-trigger Greptile

Comment thread cross-platform/src-tauri/src/input.rs Outdated
@enaboapps

Copy link
Copy Markdown
Contributor Author

@greptileai

Comment thread cross-platform/src-tauri/src/input.rs Outdated
@enaboapps

Copy link
Copy Markdown
Contributor Author

@greptileai

Comment thread cross-platform/src-tauri/src/macos.rs Outdated
@enaboapps

Copy link
Copy Markdown
Contributor Author

@greptileai

# Conflicts:
#	cross-platform/src-tauri/src/macos.rs
@enaboapps

Copy link
Copy Markdown
Contributor Author

@greptileai please review the latest commit 7479201 and provide a full score.

@enaboapps
enaboapps marked this pull request as ready for review August 5, 2026 16:34
@enaboapps
enaboapps merged commit 74566f2 into main Aug 5, 2026
5 checks passed
pub fn end_control_session(&mut self) {
self.text_streams.clear();
self.switch_session = None;
self.held_button = None;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Failed button releases lose state

When an OS button-up injection fails during disconnect, accessibility refresh, Bluetooth unsubscribe, runtime reset, or shutdown, release_held_button retains held_button, but end_control_session immediately clears it. The OS button remains pressed while subsequent cleanup has no state with which to retry the release, leaving dragging stuck.

Prompt To Fix With AI
This is a comment left during a code review.
Path: cross-platform/src-tauri/src/input.rs
Line: 987

Comment:
**Failed button releases lose state**

When an OS button-up injection fails during disconnect, accessibility refresh, Bluetooth unsubscribe, runtime reset, or shutdown, `release_held_button` retains `held_button`, but `end_control_session` immediately clears it. The OS button remains pressed while subsequent cleanup has no state with which to retry the release, leaving dragging stuck.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@greptile-apps

greptile-apps Bot commented Aug 5, 2026

Copy link
Copy Markdown

Want your agent to iterate on Greptile's feedback? Try greploops.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port modifier overlay to Tauri preview

1 participant