Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Can't get enigo working in Tauri app on MacOS #174

Open
garrrikkotua opened this issue Apr 13, 2023 · 9 comments
Open

Can't get enigo working in Tauri app on MacOS #174

garrrikkotua opened this issue Apr 13, 2023 · 9 comments
Labels
macOS macOS specific question

Comments

@garrrikkotua
Copy link

Describe your Question
Can't get enigo working in Tauri app on macOS.

Describe your Goal
I want to have enigo typing (key_sequence function) in Tauri app, but it doesn't work at all. I want it to type in the app itself and other apps too. This functionality works great on Windows.

Environment (please complete the following information):

  • OS: macOS
  • Rust: 1.68.1
  • Library Version: "0.1.2"

Note
I use set_activation_policy(tauri::ActivationPolicy::Accessory) in Tauri, so the app behaves like a spotlight window. Maybe it prevents some functionality
@pentamassiv
Copy link
Collaborator

We already have an issue for Tauri: #153
Is your problem different?

@garrrikkotua
Copy link
Author

Yeah, I think it is related. Basically key_sequence is a sequence of key_clicks, right?

But in my case the app doesn't crush, just nothing happens. I thought this might be due to some permissions on macOS, etc

@pentamassiv
Copy link
Collaborator

key_sequence is not just a sequence of key_click. key_sequence enters the whole text at the same time. It should probably be renamed to text_input or something like that. Could you please try key_click and see if it panics?

@garrrikkotua
Copy link
Author

Yeah I will try later when I have mac :) Perhaps tommorrow

@pentamassiv pentamassiv mentioned this issue Apr 24, 2023
Closed
@pentamassiv pentamassiv added the macOS macOS specific label Aug 31, 2023
@pentamassiv
Copy link
Collaborator

I am not familiar with Tauri, but I had to grant permissions in a popup. Did you grant the application the Accessibility permissions? https://support.apple.com/guide/mac-help/allow-accessibility-apps-to-access-your-mac-mh43185/mac

@pentamassiv
Copy link
Collaborator

Is this still an issue or can I close it?

@skyslide22
Copy link

i have the same issue, enigo does not fire anything
MacOS Sonoma 14.1.2 with M3
Enigo 0.1.3 & 2.0rc2 tested

#[tauri::command]
async fn switch_space(left: bool) {
    let mut enigo= Enigo::new(&Settings::default()).unwrap();
    enigo.key(Key::Control, Direction::Press);
    if left {
        enigo.key(Key::LeftArrow, Direction::Click);
    } else {
        enigo.key(Key::RightArrow, Direction::Click);
    }
    enigo.key(Key::Control, Direction::Release);
    println!("switch space left = {left}") // this is printed but key did not fire at all
}

@A-Shleifman
Copy link

I'm writing my first rust script now, so I have no idea what I'm doing, but maybe the problem is a level lower. My script is not using Tauri, but it's using tao, which is what Tauri runs on top of from what I've read.

use global_hotkey::{ hotkey::{HotKey, Code}, GlobalHotKeyEvent, GlobalHotKeyManager, HotKeyState };
use tao::event_loop::{ ControlFlow, EventLoopBuilder };
use enigo::{ Enigo, Keyboard, Settings };

fn main() {
    let event_loop = EventLoopBuilder::new().build();
    let mut enigo = Enigo::new(&Settings::default()).unwrap();
    let hotkey_manager = GlobalHotKeyManager::new().unwrap();

    enigo.text("Launched!").unwrap(); // 👈 this is working as expected

    let hotkey = HotKey::new(None, Code::F12);

    hotkey_manager.register(hotkey).unwrap();

    let global_hotkey_channel = GlobalHotKeyEvent::receiver();

    event_loop.run(move |_event, _, control_flow| {
        *control_flow = ControlFlow::Wait;

        if let Ok(event) = global_hotkey_channel.try_recv() {
            if hotkey.id() == event.id && event.state == HotKeyState::Pressed {
                println!("F12 Pressed!");

                enigo.text("input").unwrap(); // 👈 this doesn't do anything
            }
        }
    })
}

In this script enigo.text works outside tao's event_loop, but not inside. This could make hunting for the problem a bit easier.

@thewh1teagle
Copy link

thewh1teagle commented May 6, 2024 •
edited
Loading

I also facing the same issue with tauri app.
enigo works great outside of tauri, also it works with tauri in dev mode. but once I install the app and launch through applications then enigo keys doesn't work.
Production log:

tauri installed app logs 
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::app_delegate] Triggered `applicationDidFinishLaunching`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window] Creating new window
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window] Locked shared state in `set_fullscreen`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window] Unlocked shared state in `set_fullscreen`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidBecomeKey:`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidBecomeKey:`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow`
[2024-05-06T23:06:12Z TRACE tao::platform_impl::platform::app_delegate] Completed `applicationDidFinishLaunching`
[2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidResignKey:`
[2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidResignKey:`
[2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::app_delegate] Triggered `applicationSupportsSecureRestorableState`
[2024-05-06T23:06:13Z TRACE tao::platform_impl::platform::app_delegate] Completed `applicationSupportsSecureRestorableState`
[2024-05-06T23:06:16Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidBecomeKey:`
[2024-05-06T23:06:16Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidBecomeKey:`
[2024-05-06T23:06:18Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidResignKey:`
[2024-05-06T23:06:18Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidResignKey:`
[2024-05-06T23:06:23Z DEBUG enigo] using default settings
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:23Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeUp, direction: Click)�[0m
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:23Z DEBUG enigo] using default settings
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:23Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeUp, direction: Click)�[0m
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:06:23Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:25Z DEBUG enigo] using default settings
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:25Z DEBUG mobslide::cmd] Pressing VOL_DN
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeDown, direction: Click)�[0m
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:25Z DEBUG enigo] using default settings
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:25Z DEBUG mobslide::cmd] Pressing VOL_DN
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeDown, direction: Click)�[0m
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key
[2024-05-06T23:06:25Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:31Z DEBUG enigo] using default settings
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:31Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeUp, direction: Click)�[0m
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:31Z DEBUG enigo] using default settings
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:31Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeUp, direction: Click)�[0m
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:06:31Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:34Z DEBUG enigo] using default settings
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:34Z DEBUG mobslide::cmd] Pressing VOL_DN
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeDown, direction: Click)�[0m
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:06:34Z DEBUG enigo] using default settings
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] �[93mconnection established on macOS�[0m
[2024-05-06T23:06:34Z DEBUG mobslide::cmd] Pressing VOL_DN
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] �[93mkey(key: VolumeDown, direction: Click)�[0m
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key
[2024-05-06T23:06:34Z DEBUG enigo::platform::macos_impl] released all held keys
tauri dev mode logs (works) 
    Finished dev [unoptimized + debuginfo] target(s) in 3.65s
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::app_delegate] Triggered `applicationDidFinishLaunching`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window] Creating new window
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window] Locked shared state in `set_fullscreen`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window] Unlocked shared state in `set_fullscreen`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `validAttributesForMarkedText`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Triggered `viewDidMoveToWindow`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::view] Completed `viewDidMoveToWindow`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::app_delegate] Completed `applicationDidFinishLaunching`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidBecomeKey:`
[2024-05-06T23:09:38Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidBecomeKey:`

🌼   daisyUI 4.10.5
├─ ✔︎ 2 themes added             https://daisyui.com/docs/themes
╰─ ★ Star daisyUI on GitHub     https://github.com/saadeghi/daisyui

[2024-05-06T23:09:42Z TRACE tao::platform_impl::platform::window_delegate] Triggered `windowDidResignKey:`
[2024-05-06T23:09:42Z TRACE tao::platform_impl::platform::window_delegate] Completed `windowDidResignKey:`
[2024-05-06T23:09:44Z DEBUG enigo] using default settings
[2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] connection established on macOS
[2024-05-06T23:09:44Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click)
[2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:09:44Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:09:46Z DEBUG enigo] using default settings
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] connection established on macOS
[2024-05-06T23:09:46Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click)
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:09:46Z DEBUG enigo] using default settings
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] connection established on macOS
[2024-05-06T23:09:46Z DEBUG mobslide::cmd] Pressing VOL_UP
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] key(key: VolumeUp, direction: Click)
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeUp key
[2024-05-06T23:09:46Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:09:47Z DEBUG enigo] using default settings
[2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] connection established on macOS
[2024-05-06T23:09:47Z DEBUG mobslide::cmd] Pressing VOL_DN
[2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click)
[2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key
[2024-05-06T23:09:47Z DEBUG enigo::platform::macos_impl] released all held keys
[2024-05-06T23:09:48Z DEBUG enigo] using default settings
[2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] connection established on macOS
[2024-05-06T23:09:48Z DEBUG mobslide::cmd] Pressing VOL_DN
[2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] key(key: VolumeDown, direction: Click)
[2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] special case for handling the VolumeDown key
[2024-05-06T23:09:48Z DEBUG enigo::platform::macos_impl] released all held keys
cargo.toml 
enigo = { version = "0.2.0" }

Update

Found the cause, it's permission error

For manage that permission you need to open settings -> accessibility

screenshot

Then click the + button to add your app from application and allow it to control the computer
If you have already the app it's possibly the dev mode app, remove it and add again the installed one.
Once you remove it and reopen the app it should open the following prompt to ask for the permission:

screenshot

When the permission allowed enigo should work.
By the way, it will be nice if enigo can detect that the app doesn't have that required permission and show error

https://stackoverflow.com/questions/6933510

Checking it is simple as

permission.rs 
use std::{error::Error, ptr};
use accessibility_sys::{kAXTrustedCheckOptionPrompt, AXIsProcessTrustedWithOptions};
use core_foundation_sys::dictionary::{CFDictionaryAddValue, CFDictionaryCreateMutable};
use core_foundation_sys::base::{CFRelease, TCFTypeRef};
use core_foundation_sys::number::{kCFBooleanFalse, kCFBooleanTrue};


fn check_accessibility(ask_if_not_allowed: bool) -> Result<bool, Box<dyn Error>> {
    let is_allowed;
    unsafe {
        let options =
            CFDictionaryCreateMutable(ptr::null_mut(), 0, std::ptr::null(), std::ptr::null());
        let key = kAXTrustedCheckOptionPrompt;
        let value = if ask_if_not_allowed {kCFBooleanTrue} else {kCFBooleanFalse};
        if !options.is_null() {
            CFDictionaryAddValue(
                options,
                key.as_void_ptr(),
                value.as_void_ptr(),
            );
            is_allowed = AXIsProcessTrustedWithOptions(options);
            CFRelease(options as *const _);
        } else {
            return Err("options is null".into());
        }
    }
    Ok(is_allowed)
}

fn main() {
    let is_allowed = check_accessibility(true).unwrap();
    println!("Accessibility permission enabled: {}", is_allowed);
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
macOS macOS specific question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@A-Shleifman @skyslide22 @garrrikkotua @thewh1teagle @pentamassiv