-
Notifications
You must be signed in to change notification settings - Fork 5.4k
[Bug]: [MCP] Accessibility tree snapshot includes off-screen/non-viewport elements causing AI agents to generate invalid test cases on unreachable UI elements #39955
Description
Version
1.49.0
Steps to reproduce
- Set up Playwright MCP and connect an AI agent to it
- Navigate to https://wearecollins.com/story/TribecaFilm using browser_navigate
- Call browser_snapshot to get the accessibility tree
- Observe the snapshot output carefully
- Notice footer elements like "Case Studies", "Programs", "LinkedIn",
"Work with us", "Team", "Careers" appear in the snapshot - These elements are NOT visible in the current viewport — they are
hundreds of pixels below the fold - An AI agent reads this snapshot and generates test cases for these
footer elements treating them as directly accessible
Expected behavior
The accessibility tree snapshot should either:
- Only return elements that are currently visible in the viewport, OR
- Include a property on each element (e.g. "inViewport: true/false")
so AI agents can filter elements that are actually reachable
without scrolling
Elements that are off-screen, inside closed modals, or not reachable
without additional user actions should NOT be treated as directly
interactive elements.
Actual behavior
The MCP accessibility tree snapshot returns ALL elements in the DOM
regardless of whether they are in the current viewport or not.
Example — on page load of https://wearecollins.com/story/TribecaFilm,
the snapshot includes:
- "Case Studies" link ← in footer, not in viewport
- "Programs" link ← in footer, not in viewport
- "Arts & Culture" link ← in footer, not in viewport
- "Work with us" button ← in footer, not in viewport
- "Team" button ← in footer, not in viewport
- "LinkedIn" link ← in footer, not in viewport
- Email input ← in footer, not in viewport
All of these pass through as valid interactive elements in the
accessibility tree even though a real user cannot interact with them
without first scrolling to the bottom of the page.
This causes AI agents to generate invalid test cases for unreachable
elements.
Additional context
This issue directly impacts AI agent workflows built on Playwright MCP
for automated test case generation.
The core problem:
- Accessibility tree is designed to expose ALL elements for screen readers
- Screen readers can navigate non-linearly (jump to any element)
- AI agents need ONLY what a real user can see and click RIGHT NOW
- These two purposes conflict — the accessibility tree is not designed
for viewport-aware element discovery
Workaround we attempted:
Using page.evaluate() with getBoundingClientRect() to manually filter
viewport-visible elements:
const vh = window.innerHeight;
const vw = window.innerWidth;
Array.from(document.querySelectorAll('*')).filter(el => {
const r = el.getBoundingClientRect();
return r.top >= 0 && r.bottom <= vh && r.left >= 0 && r.right <= vw;
});
This is unreliable and requires manual maintenance per page.
Feature Request:
Add a viewport-only mode to browser_snapshot, for example:
browser_snapshot({ viewportOnly: true })
This would make Playwright MCP significantly more useful for
AI agent test generation use cases.
Environment
System:
OS: Windows 11 10.0.26200
CPU: (12) x64 12th Gen Intel(R) Core(TM) i5-1235U
Memory: 6.13 GB / 23.69 GB
Binaries:
Node: 20.19.6 - C:\nvm4w\nodejs\node.EXE
npm: 10.8.2 - C:\nvm4w\nodejs\npm.CMD
IDEs:
VSCode: 1.110.0 - C:\Users\ZayedBinAliBabsail\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
Cursor: 1.7.54 - C:\Program Files\cursor\resources\app\bin\cursor.CMD
npmPackages:
@playwright/mcp: ^0.0.51 => 0.0.51
playwright: ^1.49.0 => 1.57.0