Repository navigation
v1.2.0
pip install --upgrade mcp-server-macos
Your assistant can now see what an app shows (its buttons, text fields, links, checkboxes, tabs and menus) as text, and use them by name: no screenshot to find what to click, and no coordinates to guess. A model without vision can operate apps too.
New
- The
uitoolset, over the Accessibility tree. Like the other toolsets, your assistant turns it on when a task needs it (enable_toolsets), or you list it with--toolsets.- Seeing:
get_ui_elementslists what an app shows, each element with an id, its name, its value and where it is.find_ui_elementsandwait_for_ui_elementlook for one by name or role, exact names first ("Save" before "Don't Save").get_focused_ui_elementtells where keystrokes go,get_ui_element_atwhat's at a point, andread_ui_element_textreads all the text an element holds, scrolled out of view included. - Using:
press_ui_element,click_ui_element,set_ui_element_value,focus_ui_elementandperform_ui_actionact on an element by its id.list_menu_itemsgives an app's menus with their keyboard shortcuts, andclick_menu_itemchooses one by its path (File > Export > PDF…). - Cheap to read: a field left out is empty, so a button costs
{id, role, name, x, y, width, height}, andget_ui_elementslists 300 elements at most unless asked for more. An element the app doesn't place has no position, rather than 0, 0. An id is never reused while the server runs; one from long ago asks for the list again. - Safe by default: every tool needs Accessibility, asked for as the windows tools do. Pressing, clicking, acting and choosing a menu item are marked open ended, since what a button does is the app's (send, delete, buy), so your client asks you first.
- Seeing:
Changed
- pymacos 1.40 or newer is required, for
macos.ui.pip install --upgrade mcp-server-macosbrings it. - The server's instructions mention what apps show among what it covers, so your assistant looks for these tools before taking a screenshot.
See the README for how to add the server to a client.