fix(cua-driver): make GNOME/KDE Wayland input actually land via libei (#2105, #1982)

[ai-ag2026]

Summary

Makes synthetic input actually land on GNOME/Mutter (and KDE/KWin) Wayland.
The first revision of this PR fixed doctor reporting + libei readiness but not
dispatch — input still silently no-oped on the compositors it claimed to
support. This revision fixes the whole chain, verified end-to-end on a real
Ubuntu 26.04 GNOME/Wayland session (click lands in ~3 ms; type_text lands
in a focused entry).

What was broken (root causes, in order)

1. Routing gate — input tools gated the Wayland branch on is_wayland(),
   which requires DISPLAY unset. GNOME/KDE always run XWayland → DISPLAY
   set → is_wayland() false → input fell to X11 XTEST (dropped by Mutter),
   while doctor reported wayland_backend: pass. Silent false-success.
2. EIS handshake rejected — the client advertised ei_device v2, omitted
   ei_pingpong, and never answered Ping. Mutter closed the EIS connection
   immediately after our finish() (confirmed: one Handshake event, then
   read() errors).
3. Portal session died early (GNOME/Wayland: no input-injection backend available, no way to get the doctor-recommended build #2105) — open_eis_context dropped the
   ashpd proxy/session + runtime before the EIS loop ran.
4. Command/handshake race — commands ran before Resumed.
5. Wrong device — Mutter announces multiple devices; the code took the first.
6. No ei_text on Mutter — type_text via ei_text fails.

Changes

• wayland_input_enabled() predicate + 9 input call sites (capture / list_windows
  keep is_wayland() — native enum needs wlr-foreign-toplevel, absent on Mutter).
• Handshake: ei_device v1, advertise ei_pingpong, answer Ping — matches
  reis 0.7 examples/type-text.rs.
• Hold the portal RemoteDesktop session + runtime for the worker lifetime
  (multi-thread(1) runtime so the zbus connection stays driven).
• Queue input commands until a device is negotiated (input_ready + timeout).
• device_with_interface(...) selects the device carrying the needed interface.
• Keycode typing fallback via ei_keyboard when ei_text is absent, mapped
  through the compositor's actual xkb keymap (read from the ei_keyboard
Keymap event via xkbcommon) so typing is layout-correct; a US-layout table
  is the fallback when no keymap arrives.
• Single-shot drag (press → interpolated motion → release) via ei_button
  Press/Released — text selection / drag-and-drop / sliders.

Validation (Ubuntu 26.04, Mutter, Wayland; --features portal-libei)

Action	Result
EIS handshake	Handshake → Seat → Device → Resumed (was: disconnect after finish())
click / scroll / move	✅ lands (click OK ~3 ms)
press_key	✅
type_text (incl. layout-specific chars)	✅ lands layout-correctly — verified on a German QWERTZ keymap (test_YZ_yz_123 typed verbatim)
drag (select / drag-and-drop)	✅ drag-select verified — dragging over a filled entry then typing replaced the whole selection (XXXXXXXXXX → Z)
doctor wayland_backend	pass and input lands (mismatch gone)
first call	one-time xdg-desktop-portal consent dialog

Honest deferrals (documented, not silently broken)

• ei_text: used when the compositor advertises it (libei 1.6+); Mutter
  doesn't, hence the keycode + xkb-keymap fallback.
• Button held across separate tool calls (mouse_button_down / mouse_button_up,
  the persistent-cursor mouse_drag) and parallel_mouse_drag (multi-pointer /
  MPX) stay on the X11 path — they need cross-call emulation state / multiple
  pointers the single-shot libei command model doesn't provide, so on native
  Wayland windows they no-op (they still work on XWayland windows). Single-shot
  drag (above) does work via libei.
• Modifier hotkeys: a bare-key hotkey now routes via libei; true modifier
  chords (Ctrl+C, …) aren't wired into the libei keyboard yet and error loudly
  rather than silently mis-firing.
• Per-window background input is impossible on Wayland by design; the
  substitute is delivery_mode:"foreground".

Refs

Closes the runtime half of #1982 / #2105. Pairs with #2111 (publishes the
portal-libei artifact users need). Builds on #1966 / #2008 (libei machinery).

[vercel]

@ai-ag2026 is attempting to deploy a commit to the Cua Team on Vercel.

A member of the Team first needs to authorize it.

[coderabbitai]

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6f9fba30-139a-4133-bd9b-97cd070fcf17

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds a portal/libei RemoteDesktop reachability probe integrated into Wayland health-check classification, and introduces a libei-based input fallback path (pointer clicks, scroll, drag, move, typing, hotkeys) used when the wlroots virtual-pointer manager is unavailable, gated by a new NO_VPTR_MARKER and feature flag.

Changes

Wayland libei/portal fallback

Layer / File(s)	Summary
Portal RemoteDesktop probe and health classification libs/cua-driver/rust/crates/platform-linux/src/health_report.rs	Adds probe_portal_remote_desktop() using ashpd/Tokio, wires reachability into classify_wayland_backend for pass/fail messaging, updates partial-pass criteria, and adds unit tests.
Virtual-pointer session detection and fallback marker libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs	Adds NO_VPTR_MARKER constant and with_libei_fallback/is_no_vptr helpers; reorders open_vptr_session to detect missing virtual-pointer manager and defer foreign-toplevel requirement.
Pointer operations routed through libei fallback libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs	Adds output_dimensions()/normalize_click_xy(); refactors click, scroll, move_cursor_absolute, drag to route through with_libei_fallback.
Typing and key operations with libei adapter implementations libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs	Refactors type_text, press_key, hotkey to fall back to libei on wtype failure; implements libei adapters (click, scroll, move, type, press_key, key_to_evdev) and drag-unsupported error.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant HealthReport as check_wayland_backend
  participant Probe as probe_portal_remote_desktop
  participant Ashpd as ashpd RemoteDesktop
  participant Classifier as classify_wayland_backend

  HealthReport->>Probe: run probe (portal-libei feature)
  Probe->>Ashpd: RemoteDesktop::new()
  Ashpd-->>Probe: reachable / not-found error
  Probe-->>HealthReport: portal_libei_backend bool
  HealthReport->>Classifier: classify(virtual_pointer, portal_libei_backend, globals)
  Classifier-->>HealthReport: pass / fail with hint

Loading

sequenceDiagram
  participant Caller
  participant WithLibeiFallback as with_libei_fallback
  participant VptrImpl as click_vptr/scroll_vptr/...
  participant LibeiImpl as libei_click/libei_scroll/...

  Caller->>WithLibeiFallback: click/scroll/move/drag
  WithLibeiFallback->>VptrImpl: try wlroots vptr operation
  VptrImpl-->>WithLibeiFallback: NO_VPTR_MARKER error (if missing)
  WithLibeiFallback->>LibeiImpl: fallback (portal-libei enabled)
  LibeiImpl-->>Caller: result

Loading

Possibly related issues

• GNOME/Wayland: no input-injection backend available, no way to get the doctor-recommended build #2105: Adds/adjusts portal/libei RemoteDesktop backend handling in the Wayland health-check classification path.
• cua-driver-rs (Linux/Wayland): KDE Plasma 6.7 — agent cursor renders but input doesn't dispatch (portal-libei feature gated off in published binary) #1982: Adds portal/libei reachability-aware Wayland health checks addressing missing diagnostics when libei is unavailable.

Possibly related PRs

• trycua/cua#1966: Updates the same health_report.rs classification and wayland/mod.rs virtual-pointer fallback logic for portal-libei integration.
• trycua/cua#1992: Modifies the same missing zwlr_virtual_pointer handling path based on portal-libei compile flag.

Suggested reviewers: r33drichards

Poem

A rabbit hopped where wlroots failed,
Found libei's door, and portals trailed,
With markers set and fallbacks true,
Clicks and keys now find their way through 🐰
No missing pointer stops the play!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: routing GNOME/KDE Wayland input through libei fallback.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@libs/cua-driver/rust/crates/platform-linux/src/health_report.rs`:
- Around line 522-524: The readiness check in RemoteDesktop::new() is too
shallow because it only confirms the portal proxy exists, not that the libei
input path is usable. Update the health probe around the existing rt.block_on
match so it exercises the full RemoteDesktop handshake by continuing through
create_session, select_devices, start, and connect_to_eis, or else change the
reported verdict/message to indicate only “RemoteDesktop portal reachable”
rather than full input backend readiness. Use the RemoteDesktop::new flow and
the surrounding health_report logic to locate the fix.

In `@libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs`:
- Around line 1413-1495: The libei-backed input helpers are still treating
missing EIS capabilities as success, so requests can disappear silently. Update
the capability checks in the libei path so the relevant operations fail when the
negotiated device lacks PointerAbsolute, Button, Scroll, Text, or Keyboard, and
make sure the callers such as libei_click, libei_scroll, libei_move_absolute,
libei_type_text, and libei_press_key propagate that error instead of returning
Ok(()).

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 0177a824-f19a-47fb-bd81-b1ecd81b68cc

📥 Commits

Reviewing files that changed from the base of the PR and between ce2da2d and c214a29.

📒 Files selected for processing (2)

• libs/cua-driver/rust/crates/platform-linux/src/health_report.rs
• libs/cua-driver/rust/crates/platform-linux/src/wayland/mod.rs

[ai-ag2026]

Addressed the two CodeRabbit findings in 60bb3015:

• health_report.rs: the Wayland health message now reports only RemoteDesktop portal reachability and says command execution will attempt the full portal/EIS handshake, instead of implying the whole input backend is already proven ready by RemoteDesktop::new().
• wayland/libei.rs: libei command execution now fails loudly when the negotiated EIS device lacks the required interface (PointerAbsolute, Button, Scroll, Text, or Keyboard) instead of silently returning Ok(()).

Verification run in a clean rust:1.90-bookworm container:

cargo test -q -p platform-linux --features portal-libei health_report::tests -- --nocapture
cargo check -q -p cua-driver --features portal-libei
cargo check -q -p cua-driver
git diff --check

All passed. Existing warnings are pre-existing in adjacent code. Vercel is still blocked by deploy authorization, not by this code path.

[ai-ag2026]

@f-trycua — this PR (and #2111) are fork PRs, so their workflow runs (incl. the blocking nix-wayland.yml gate) sit at action_required until a maintainer approves them. Could you kick them off?

This one now makes GNOME/KDE Wayland input actually land via libei — it fixes the EIS handshake Mutter was rejecting, the DISPLAY-gated dispatch, the #2105 session lifetime, device selection, and adds a keycode typing fallback since Mutter has no ei_text. Verified end-to-end on Ubuntu 26.04 GNOME (click OK ~3 ms; type_text landed in a focused entry). #2111 ships the portal-libei artifact users need. Happy to iterate on the gnome nix-wayland scenarios once CI can run.

[ai-ag2026]

Rebased this PR onto current origin/main; no conflict resolution or code changes beyond replaying the existing commits were needed.

Local verification after rebase:

git diff --check origin/main...HEAD
cargo test -q -p platform-linux --features portal-libei health_report::tests -- --nocapture
cargo check -q -p cua-driver --features portal-libei
cargo check -q -p cua-driver

All passed. The Rust commands still emit the existing adjacent warnings, but no errors.