Tell your agent:
Install the MCP tools here https://github.com/adammcarter/annotate
- Learn a new app
- Get a pointer in a game
- Find help with a complex idea
Annotate allows your agent to draw directly on your screen.
Wanted to learn Blender but found it too complex?
Not sure where to put your next drop in a tower defence game?
Working on a new idea but something’s not quite right?
Your agent could tell you the exact x and y coordinates, but that’s not very helpful...
Now your agent can point directly on your screen.
Talking of learning Blender, here’s a small demo of Claude giving a walkthrough of Blender’s tools.
annotate_circle |
Loop something. The workhorse — takes a label. |
annotate_underline |
Underline a phrase, keeping the words readable. |
annotate_highlight |
Wash a marker over a region. |
annotate_arrow |
Point at something you can’t draw around. |
annotate_text |
Say something when there’s no control to indicate. |
annotate_locate |
Find a real element’s position, so nothing is guessed. |
annotate_screens |
Which display, and at what scale. |
annotate_clear |
Erase everything. |
-
Permissions. Annotate needs Accessibility (System Settings → Privacy & Security → Accessibility) the first time it runs. Nothing else does.
-
Privacy. Annotate draws, and reads UI element positions. It never captures your screen and makes no network connections.
-
Trust. Drawing and reading are not held to the same standard, because they are not the same authority.
Once Annotate is installed, the MCP server is already on disk — it ships inside the app bundle:
/Applications/Annotate.app/Contents/MacOS/annotate-mcp
That path is the whole configuration, and it is the same everywhere. It speaks MCP over stdio and launches the app itself if it isn’t already running.
claude mcp add annotate /Applications/Annotate.app/Contents/MacOS/annotate-mcp
[mcp_servers.annotate]
command = "/Applications/Annotate.app/Contents/MacOS/annotate-mcp"{
"mcpServers": {
"annotate": {
"command": "/Applications/Annotate.app/Contents/MacOS/annotate-mcp"
}
}
}Found an issue? Or something you like? I’d love to know about it!
Feel free to open an issue.
agent ──MCP──▶ annotate-mcp ──unix socket──▶ Annotate.app ──▶ your screen
Packages/AnnotateCore is pure geometry and the wire protocol — no AppKit, no UI.
Every mark is seeded from its annotation id, so it has its own character and is
exactly reproducible, which is what lets the drawing be pinned by golden tests
instead of eyeballed. Annotate/ is the app: a click-through overlay per screen,
Core Animation rendering, and the control socket. Packages/AnnotateMCP is the
stdio server that turns tool calls into socket commands.
docs/DESIGN.md covers why the ink looks the way it does.
docs/adr/ has the decisions, including the alternatives that were
tried and rejected.
Feel free to fork, update or fix this repo — ideally open an issue first.
Runs on macOS 14+. Building requires Xcode 26, which needs macOS 26 — the app icon is an Icon Composer bundle and only that toolchain compiles it.
open Annotate.xcodeproj # ⌘R — it appears in the menu bar
bash Tools/guided-tour.sh # the 45-second pitch, drawn on your own screen
swift test --package-path Packages/AnnotateCore
xcodebuild -project Annotate.xcodeproj -scheme Annotate -destination 'platform=macOS' testTwo things to know before changing anything that draws:
Generator draw count and draw order are the pixel contract. Moving a draw
between functions is safe; reordering one silently changes every existing mark.
There’s a longer note at the top of PenStroke.swift.
An offline render is not proof. Several bugs here looked perfect in a rendered PNG and wrong on a real composited screen. Confirm by running the app.
The film at the top is film/, and it doesn’t redraw the ink by hand —
Tools/export-marks.swift dumps the real AnnotateCore geometry and the reel
replays it, wipe plan included.
MIT — see LICENSE.