Skip to content

Add ui-test skill for agentic UI testing#52

Closed
shubh24 wants to merge 1 commit intomainfrom
shub24/ui-test-skill
Closed

Add ui-test skill for agentic UI testing#52
shubh24 wants to merge 1 commit intomainfrom
shub24/ui-test-skill

Conversation

@shubh24
Copy link
Contributor

@shubh24 shubh24 commented Mar 25, 2026

Summary

  • Adds ui-test skill — AI-powered UI testing that catches what Playwright can't
  • Reads frontend codebases to generate test suites covering: accessibility, visual quality, responsive design, console health, UX heuristics (Laws of UX + Nielsen's 10), error states, data display, and exploratory testing
  • Uses the browse CLI for lightweight execution — no Node.js dependency
  • Progressive disclosure: 163-line SKILL.md entry point + reference files for deep knowledge

Structure

skills/ui-test/
├── SKILL.md                            # Skill definition (163 lines)
├── README.md
├── rules/
│   └── ux-heuristics.md                # 6 evaluation frameworks
├── references/
│   ├── browser-recipes.md              # Copy-paste browse CLI recipes
│   ├── codebase-analysis.md            # 8-step test generation guide
│   └── exploratory-testing.md          # Agent-driven QA guide
└── examples/
    └── browserbase-dashboard-suite.yml # Example suite

Install

npx skills add browserbase/skills --skill ui-test

🤖 Generated with Claude Code


Note

Low Risk
Low risk: this PR only adds new documentation/config files for a new ui-test skill and does not modify existing runtime code paths.

Overview
Introduces a new skills/ui-test skill definition (SKILL.md) plus supporting docs (README.md, references/*, rules/ux-heuristics.md) describing how to generate/run UI test suites via the browse CLI, including coverage-gap analysis and autonomous suite updates.

Adds an example generated suite (examples/browserbase-dashboard-suite.yml) and copy-paste browser check recipes to standardize accessibility, visual, responsive, console, UX-heuristics, error-state, and exploratory testing.

Written by Cursor Bugbot for commit 7cca436. This will update automatically on new commits. Configure here.

Reads frontend codebases to generate test suites covering what Playwright misses:
accessibility (axe-core + keyboard nav), visual quality, responsive design,
console health, UX heuristics (Laws of UX + Nielsen's), error states, data
display, and exploratory testing.

Uses the browse CLI for lightweight execution — no Node.js dependency.
Progressive disclosure via reference files for deep knowledge.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@shubh24 shubh24 marked this pull request as draft March 25, 2026 02:30
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for all 3 issues found in the latest run.

  • ✅ Fixed: Personal filesystem path leaked in example file
    • I replaced the leaked absolute generated_from path with the relative example-safe value ./src.
  • ✅ Fixed: Console capture recipe loses state on navigation
    • I updated the recipe to open and load TARGET_URL before injecting the console override so captured logs persist for the page under test.
  • ✅ Fixed: Focus ring detection misses box-shadow based indicators
    • I changed hasFocus detection to treat either a visible outline or a non-none box-shadow as a valid focus indicator and updated the guidance text accordingly.

Create PR

Or push these changes by commenting:

@cursor push c0e806252d
Preview (c0e806252d)
diff --git a/skills/ui-test/examples/browserbase-dashboard-suite.yml b/skills/ui-test/examples/browserbase-dashboard-suite.yml
--- a/skills/ui-test/examples/browserbase-dashboard-suite.yml
+++ b/skills/ui-test/examples/browserbase-dashboard-suite.yml
@@ -1,6 +1,6 @@
 version: 1
 base_url: http://localhost:3000
-generated_from: /Users/shubhankar/browserbase/core/apps/core/src
+generated_from: ./src
 generated_at: 2026-03-24T16:30:00Z
 notes: >
   Example test suite generated against the Browserbase dashboard.

diff --git a/skills/ui-test/references/browser-recipes.md b/skills/ui-test/references/browser-recipes.md
--- a/skills/ui-test/references/browser-recipes.md
+++ b/skills/ui-test/references/browser-recipes.md
@@ -82,10 +82,11 @@
 "

-For comprehensive console capture, inject a console override early:
+For comprehensive console capture, inject a console override on the target page:

-browse open "about:blank"
+browse open "TARGET_URL"
+browse wait load
browse eval "
  window.__logs = [];
  const orig = { error: console.error, warn: console.warn };
@@ -94,8 +95,7 @@
  window.addEventListener('error', e => window.__logs.push({type:'uncaught', text: e.message}));
  window.addEventListener('unhandledrejection', e => window.__logs.push({type:'rejection', text: String(e.reason)}));
"
-browse open "TARGET_URL"
-browse wait load
+# Perform the actions you want to test while capture is active, then read logs:
browse eval "JSON.stringify(window.__logs)"

@@ -110,7 +110,7 @@

Tab through elements one at a time

browse press Tab
-browse eval "JSON.stringify({tag: document.activeElement?.tagName, text: document.activeElement?.textContent?.trim().slice(0,40), role: document.activeElement?.getAttribute('role'), ariaLabel: document.activeElement?.getAttribute('aria-label'), hasFocus: window.getComputedStyle(document.activeElement).outlineStyle !== 'none'})"
+browse eval "JSON.stringify({tag: document.activeElement?.tagName, text: document.activeElement?.textContent?.trim().slice(0,40), role: document.activeElement?.getAttribute('role'), ariaLabel: document.activeElement?.getAttribute('aria-label'), hasFocus: (() => { const s = window.getComputedStyle(document.activeElement); const hasOutline = s.outlineStyle !== 'none' && s.outlineWidth !== '0px'; const hasRing = s.boxShadow && s.boxShadow !== 'none'; return hasOutline || hasRing; })()})"

Repeat browse press Tab + eval to build the full tab order

Stop when activeElement returns BODY (looped back)

@@ -119,7 +119,7 @@
What to check in the results:

  • Every interactive element should appear in the tab order
  • Order should follow visual layout (top-to-bottom, left-to-right)
    -- hasFocus should be true for every element (visible focus ring)
    +- hasFocus should be true for every element (visible outline or focus ring)
  • No elements should be skipped or appear out of order

Responsive Screenshot Sweep


</details>


<sub>This Bugbot Autofix run was free. To enable autofix for future PRs, go to the <a href="https://www.cursor.com/dashboard?tab=bugbot">Cursor dashboard</a>.</sub>
<!-- BUGBOT_AUTOFIX_REVIEW_FOOTNOTE_END -->

@@ -0,0 +1,223 @@
version: 1
base_url: http://localhost:3000
generated_from: /Users/shubhankar/browserbase/core/apps/core/src
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personal filesystem path leaked in example file

Medium Severity

The generated_from field contains a developer's personal absolute path /Users/shubhankar/browserbase/core/apps/core/src. This leaks internal directory structure and a developer's username. Since this is an example file intended for public distribution, it exposes private information and looks unprofessional. A relative path like ./src (as used in the SKILL.md example) would be appropriate.

Fix in Cursor Fix in Web

"
browse open "TARGET_URL"
browse wait load
browse eval "JSON.stringify(window.__logs)"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Console capture recipe loses state on navigation

Medium Severity

The "comprehensive console capture" recipe injects window.__logs and console overrides on about:blank, then navigates to TARGET_URL via browse open. Navigating to a new URL resets the JavaScript execution context, so the monkey-patched console.error/console.warn and window.__logs array are all lost. The final browse eval "JSON.stringify(window.__logs)" will return null or throw, silently producing no results. The override needs to be injected after the target page loads, or via a mechanism that persists across navigations.

Fix in Cursor Fix in Web


# Tab through elements one at a time
browse press Tab
browse eval "JSON.stringify({tag: document.activeElement?.tagName, text: document.activeElement?.textContent?.trim().slice(0,40), role: document.activeElement?.getAttribute('role'), ariaLabel: document.activeElement?.getAttribute('aria-label'), hasFocus: window.getComputedStyle(document.activeElement).outlineStyle !== 'none'})"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Focus ring detection misses box-shadow based indicators

Medium Severity

The hasFocus check uses outlineStyle !== 'none' to detect visible focus rings, but most modern CSS frameworks (Tailwind's ring-*, Chakra UI, Radix, etc.) implement focus indicators via box-shadow while setting outline: none. This recipe will report hasFocus: false for elements that do have visible focus indicators, causing agents to generate false accessibility violations on virtually every modern app.

Additional Locations (1)
Fix in Cursor Fix in Web

shrey150 added a commit that referenced this pull request Mar 26, 2026
Builds on #52 with three key additions:

1. Local/remote mode selection — localhost uses local browser (no API key),
   deployed sites use Browserbase via cookie-sync for authenticated testing

2. Diff-driven testing — analyze git diff, generate targeted tests for what
   changed, execute with before/after snapshot comparison

3. Structured assertion protocol — STEP_PASS/STEP_FAIL markers with evidence,
   deterministic checks (axe-core, console errors, overflow detection), and
   adversarial testing patterns (XSS, empty submit, rapid click, keyboard-only)

Smoke-tested against a local Next.js app: found real bugs (Escape not closing
modals, undersized mobile touch targets) that confirmed the adversarial patterns
work. Fixed browse eval recipes (no top-level await, console capture on-page not
about:blank).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@shubh24 shubh24 closed this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant