Repository navigation
Buddy 1.12.0: the agent pointer follows the real click in every app, and on the Mac too
Buddy 1.12.0: the agent pointer follows the real click in every app, and on the Mac too
Until now the pointer flew to the pressed thing only in Chrome and Edge pages. In any other program it stood on the window and never on the click.
Windows
- Every window of a followed task that is not a browser is listened to (Notepad, a settings box, File Explorer, an Office window), from the moment it is followed: a button, tick box, list entry, tab or menu entry a program presses there through Windows (UI Automation) gets the pointer on that thing.
- A click a program makes with the mouse itself is followed in every window, also in an app you had open already. Windows gives such a press no device; your own mouse and touchpad always have one, so your own clicks are never taken for it.
- Honest about whose click it was. In a window that is not the task's own nobody can say which program clicked: it is taken for the followed task most likely at work (an AI agent before a scheduled task) and the trace says (not its own window: taken for its click). With no followed task the trace still has a program nobody follows, with the program and the point.
- Fixed on the way: the screen listener could fail to start when its thread was the first one to load COM.
- What it cannot show: a press through Windows in an app you had open already that is not a followed window (no mouse, no news), a round option box in a Windows Forms program chosen that way, a box filled without the keyboard, and programs that tell Windows nothing (there only mouse clicks are followed). Firefox and command windows are not followed.
Mac (Buddy Lite)
- New: the agent pointer. When a program clicks with the mouse, in any app, a second pointer in another colour flies to the click, shows a ring and names the program: Terminal clicked here. It leaves 2.5 s after the last click.
- Every such click is a line in
~/Library/Logs/Buddy/buddy.log. No new permission is asked. Menu: Show where an agent clicks. - Not shown on a Mac: a press a program makes without the mouse (through Accessibility), and typing. Clicks passed on by mouse tools (BetterTouchTool, LinearMouse, Mac Mouse Fix, SteerMouse, USB Overdrive, Logitech, Karabiner) and screen sharing count as your own hand.
Checked
- Windows: the tests (
keyit/tests491 passed,pips/tests13 passed) on a desktop that is never shown; one island test needs a real screen and passes on GitHub. Two real test windows kept off the screen, one classic and one Windows Forms, pressed through UI Automation from another program: button, tick box, list entry and tab of both were told (and the classic round option box), the windows never came to the front, all listeners were gone afterwards. The self-test of the built app passed. - Mac: on GitHub's Macs (macOS 15, Apple Silicon and Intel) with the built app: a click another program posted (
osascript) was heard with that program on it at the exact point and the pointer said osascript clicked here; a click Buddy posted itself was left alone.
Not checked
- Windows: no click was made on a real screen for the mouse rule (tests only), and the touchpad rule was not tried by hand.
- Mac: a real mouse or trackpad (GitHub's machines have none), the mouse tools in the list, more than one display, and a MacBook Pro 2019.
Windows: Buddy-win64.zip, 181321300 bytes, SHA-256 085625538c84510675ce305236ced81d95166d145abda8da65bcf074e6be8c54.
Mac: Buddy-mac.zip is attached by the Mac build of this release.
iPhone stays 1.1.0.