Repository navigation
clicking by selector or by coordinates
Two ways to click the same thing. Measured on 2026-09-17 on one button, three
runs each, on a page served from 127.0.0.1:
| median | |
|---|---|
browser_click with a selector |
0.23 s |
browser_click_at with the coordinates from the snapshot |
0.59 s |
The coordinate click is about two and a half times slower, because it is a real pointer being moved to a place rather than an element being located and activated. It also hands back a picture of the page afterwards, where the selector click hands back a sentence.
Neither of those is the reason to choose. The reason is the next section.
A stale selector fails loudly. If the element has moved, been replaced, or was never there, the call reports it. You get an error, the agent can react, and nothing happened to the page.
Stale coordinates succeed wrongly. There is always something at an x and a y. If the layout shifted between the snapshot and the click, the pointer lands on whatever is there now: a different button, a link, an advert. The call reports success, the wrong thing happened, and nothing in the run says so.
That asymmetry is worth more than two-fifths of a second. Prefer the selector because its failure is visible, and reach for coordinates when the selector cannot describe the target at all.
Layout moving between the look and the click is not rare. It is the normal behaviour of a page with images and lazy content, and it has its own page.
Three cases, two of them measured on their own pages here:
- Inside an iframe. The selector times out after fifteen seconds; the coordinate click lands. Measured, in an agent and an iframe.
- A component with no usable selector, typically inside a shadow root with nothing stable exposed. Clicking into a shadow root by selector does work when you have one, which is its own surprise; coordinates are for when you do not.
- Anything not made of elements. A canvas, a map, a chart, a slider drawn with divs, a signature pad, a picture with hotspots. There is nothing to select, and a position is the only way to say where you mean.
There is also a fourth that is about behaviour rather than structure: a control
that only responds to a genuine pointer sequence in one place. Rare on ordinary
pages, real on drag handles and custom sliders, where hold_seconds on the
coordinate click is the tool that has the shape of the gesture.
Take them from the snapshot, which reports at: [x, y] in viewport pixels for
every element it lists, and use them immediately. They are a description of the
page at the moment of the read, and their accuracy decays as soon as anything
moves.
The sequence that works:
snapshot -> pick the element -> snapshot again -> click_at the fresh position
The second read costs about a hundredth of a second, per how long an AI browser agent takes per step, and it closes the gap from a model turn to a tool call.
When the element is not in the snapshot at all, the coordinates have to come from a picture: take a screenshot, find the thing in it, click where it is. That is the third rung, and it is the expensive one, at about ninety times the text of the page in what the agent should read.
Because it is the rung you use when you could not describe the target, so the only honest confirmation is what the page looks like now. On the measured run that reply was a 17 KB image.
Treat it as a confirmation to look at when something seems wrong rather than something to keep. Ten coordinate clicks in a loop is ten images in the context, which is the budget problem described on the reading page.
Selector when you can name it, coordinates when you can only point at it, screenshot when you cannot even find it to point at. Go down the ladder only when the rung above genuinely does not work, and go back up as soon as it does.
Which is faster, selector or coordinates? Selector: 0.23 s against 0.59 s on the same button. The difference is a real pointer being moved.
Why prefer the selector if both work? Because when it is wrong it fails visibly. A wrong coordinate clicks something else and reports success.
When do I have to use coordinates? Inside an iframe, on a component with no usable selector, and on anything drawn rather than built from elements: canvas, maps, sliders.
Where do the coordinates come from? The snapshot gives at: [x, y] per
element. Re-read immediately before clicking, because they age as soon as the
layout moves.
Why did my coordinate click do the wrong thing? The page moved between the read and the click. That failure is silent by construction, which is the argument for the selector.
See also: why did the AI agent click the wrong thing, where this asymmetry is the first of six causes, and what a page snapshot costs, per control, since the snapshot is where both the selector and the coordinates come from.
- Measured 2026-09-17 through this project's MCP server over stdio, three runs of each call against the same button on a page served from
127.0.0.1, with a fresh navigation before each run:browser_clickmedian 0.23 s,browser_click_atat the snapshot's reported position median 0.59 s. The coordinate call returned a 17,392-byte image alongside its result.browser_press_keyon the same page took 0.14 s.
Two and a half times slower is the headline and the least important line here. The one that matters is that only one of these two tells you when it was wrong.
- OpenAI Operator alternatives
- Open-source Operator-style agents
- Is OpenAI Operator still available?
- OpenAI Operator vs Claude computer use
- browser-use alternatives
- Choosing an AI browser agent
- Open-source AI browser agents
- Open-source computer-use agents
- What is an AI web agent?
- AI browser agents vs traditional scraping
- Cloud browser infrastructure for AI agents, explained
- Browserbase alternatives
- Firecrawl vs an AI browser agent
- Skyvern alternatives
- Stagehand vs browser-use
- Project Mariner is gone: what replaced it
- Manus alternatives
- Gemini computer use vs Claude computer use
- A stealth browser MCP, reviewed honestly by its own wiki
- AI browser vs AI browser agent: which one do you want?
- AI browser agent vs RPA: which one fits the job
- AI browser agent vs n8n, Zapier and Make
- Vercel agent-browser alternatives, compared honestly
- What is an agentic browser? Definition and the two kinds
- Open-source agentic browsers: the three layers, compared
- Choosing an MCP server for browser automation: four axes
- Stealth MCP servers compared: Camoufox, nodriver, Patchright
- Playwright MCP alternatives, and the three you don't need
- Autonomous browser agents: the four rungs of autonomy
- What is actually free in the AI browser agent stack
- browser-use on GitHub: what the repo actually gives you
- Playwright MCP vs Chrome DevTools MCP: different jobs
- How to choose among MCP servers: a map by category
- Which MCP servers are worth adding to Claude Code
- MCP on GitHub: finding servers and judging them fast
- MCP vs an API: the decision, and what the wrapper costs
- MCP alternatives: when the protocol is the wrong shape
- Why does my AI agent get blocked?
- The timing signal AI agents give off
- Agent retry loops trip rate limits, not fingerprints
- Claude computer use detected as a bot
- browser-use getting blocked: what you can and cannot change
- Playwright MCP session blocked: four causes, four fixes
- Playwright MCP and captchas: what actually gets you past
- Cloudflare and a browser MCP server: what is being read
- Can an AI agent solve a captcha? The honest answer
- Getting an AI agent to fill out forms
- The best model for an MCP browser agent, and what it really costs
- Browser problem or model problem?
- How to use a Playwright MCP server with Claude Code
- Extracting data to a CSV with an AI agent
- Monitoring a page for changes with an AI agent
- How to let Claude Desktop control a browser
- How to add a browser to Cursor as an MCP server
- Using an AI agent to hunt for apartments
- Getting website data into Google Sheets with an AI agent
- Using an AI agent to download invoices from portals
- AI agents for web research
- Using an AI agent to test your own website
- How to add a browser to Cline as an MCP server
- Posting to social media with an AI agent
- Posting to Facebook with an AI agent
- Posting to Instagram with an AI agent
- Posting to X with an AI agent
- Automating LinkedIn posts: read this first
- Appointment bots: what they are and what an agent can legitimately do
- Track prices across sites with an AI agent
- Build a lead list with an AI browser agent
- Run an AI browser agent on a schedule
- AI browser agent with a local LLM: what changes
- Should you log your AI agent into your accounts?
- How to write a task an AI browser agent can follow
- Move data between two web apps with an AI agent
- The MCP server
- How the tools are shaped, and why
- Playwright MCP vs the Playwright CLI: which fits when
- Playwright MCP: browser is already in use, and the fix
- Playwright MCP best practices: four decisions that matter
- Playwright MCP with a proxy, and the three leaks it leaves
- A browser MCP server in GitHub Copilot: setup and limits
- Using a browser MCP server for web scraping: the pattern
- Which LLM for browser automation: the four properties
- How to build a browser agent, and what to take instead
- Getting an AI agent to log into a website: three routes
- MCP tools, resources and prompts: who controls each
- How many MCP tools is too many? The context arithmetic
- How to build an MCP server: the decisions, not the scaffold
- Local or remote MCP server: what changes, and what does not
- Writing an MCP client in Python: the thirty-line version
- Self-hosted AI agent: what one actually costs to run
- How long an AI browser agent takes per step, measured
- Text, HTML, snapshot or screenshot: what the agent should read
- Giving an AI browser agent a stopping condition
- Keeping an AI browser agent out of destructive actions
- Why did the AI agent click the wrong thing
- When the page changes under the AI agent
- Running one AI agent task across a list of sites
- Seeing a page as it appears in another country
- Getting data out of a dashboard with no export button
- Two browsers in one session: main and support
- Finding the dead links on a site with an AI agent
- Filling a CRM record from a company's website
- One form submission per spreadsheet row, with an AI agent
- Dated screenshots of a page as evidence
- Checking order and delivery status with an AI agent
- Reading a PDF that opens inside the browser
- Summarising a long page or thread with an AI agent
- Collecting every image on a page with its caption
- Collecting event and course listings with an AI agent
- Cancelling a subscription with an AI agent
- What an AI agent can and cannot do inside an iframe
- Shadow DOM and an AI agent: you can click it, you cannot read it
- How to add a browser to Codex as an MCP server
- What a page snapshot costs, per control
- How to let Gemini CLI use a browser
- Native selects and the ones that only look like selects
- Clicking by selector or by coordinates
- How long the agent waits before it gives up
- What a second browser costs
- Uploading a file with an AI agent, and why this one cannot
- Watching the agent work, and when it is worth it
- When not to use an AI browser agent
- Agent or script: deciding once instead of every time
- Using the keyboard instead of the mouse
- Secrets in an agent task: where they end up
- What an agent run should log
- Deduplicating what an AI agent collects
- Normalising values across sites
- Validating an AI agent's output
- Reading a table with an AI agent
- Driving a site's own search and filters
- The task works headed and fails headless
- How to let the AdaL CLI use a browser