Repository navigation
Browser Tools Testing
Phase 5.4: Testing & Iteration | Version: 3.0.0 | Last Updated: March 5, 2026
This guide provides step-by-step play scripts for manual testing of agentic browser tools with the web-accessibility-wizard and web-issue-fixer agents.
Scope:
- Test on Windows and macOS
- Test with VS Code 1.110+
- Test browser tool availability (enabled/disabled)
- Test all 5 usage patterns (fix verification, visual verification, interaction testing, failure handling, graceful degradation)
- Test performance (screenshot capture time, DOM analysis time)
- Verify visual evidence (screenshots are clear, metadata is captured)
Duration: ~2 hours (split across 3-4 test sessions)
Required:
- VS Code 1.110+ (March 2026 or later)
- Node.js LTS, 26, 24 or 22 (for the dev server)
- A simple web project with known accessibility issues:
- Example project:
example/directory in this repo - Or: Your own project with a dev server running on http://localhost:3000
- Example project:
Optional:
- Multiple machines for Windows and macOS testing
- Different frameworks (React, Vue, vanilla HTML)
Option A: Enable browser tools globally
// File: ~/Library/Application Support/Code/User/settings.json (macOS)
// or: %APPDATA%\Code\User\settings.json (Windows)
{
"workbench.browser.enableChatTools": true
}Option B: Enable for this workspace only
// File: <your project>/.vscode/settings.json
{
"workbench.browser.enableChatTools": true
}For example/ directory:
cd example/
npm install
npm run dev
# Server will be available at http://localhost:3000For your own project:
cd your-project/
npm install
npm run dev
# Note the URL (e.g., http://localhost:5173 for Vite)Goal: Verify browser tools are available and callable.
Duration: 5 minutes
-
Open Copilot Chat in VS Code (Ctrl+Alt+I / Cmd+Cmd)
-
Invoke the accessibility-lead agent:
/accessibility-lead can you check if browser tools are available? -
Expected response:
- Agent acknowledges browser tools setting
- Shows if
workbench.browser.enableChatToolsis true/false - If true: "Browser verification will be available for fix verification"
- If false: "To enable browser verification, set workbench.browser.enableChatTools to true"
-
If disabled, toggle setting and try again:
- Open settings: Ctrl+, / Cmd+,
- Search: "browser chat tools"
- Toggle: "Enabled"
- Result: Agent should now show browser tools as available
- Browser setting is detected correctly
- Agent responds with availability status
- Toggling setting changes reported availability
Goal: End-to-end test of applying a fix and verifying it in the browser.
Duration: 20 minutes
Use the example project with known issues:
<!-- example/index.html -->
<img id="hero" src="hero.png"> <!-- Missing alt text -->
<button style="color: #666; background: #AA9977;">Click me</button> <!-- Low contrast -->-
Start web-accessibility-wizard:
/web-accessibility-wizard audit the example project -
Answer Phase 0 questions:
- URL:
http://localhost:3000 - Framework:
vanilla(or auto-detected) - Thoroughness:
standard - Screenshots:
yes
- URL:
-
Wizard will scan and identify issues including:
- Missing alt text on #hero
- Low contrast on button
-
At Phase 11 (Follow-up Actions), select:
"Verify fixes in browser" -
Accept the browser verification invitation:
"Yes - verify all fixes" -
Observe wizard behavior:
- Detects dev server at http://localhost:3000
- Opens page in integrated browser
- Takes screenshot of #hero image
- Reports: "Alt text added and verified"
- Takes screenshot of button
- Reports: "Contrast improved to 14:1 (WCAG AAA)"
- Embeds before/after screenshots in report
-
Check output files:
-
WEB-ACCESSIBILITY-AUDIT.mdexists -
.a11y-screenshots/directory created - Screenshots present with naming:
{timestamp}-fix{n}-{selector}.png
-
- Wizard detects issues correctly
- Fixes are proposed and applied to code
- Browser verification is offered when
workbench.browser.enableChatToolsis enabled - Screenshots are captured and stored
- Verification results are accurate ("PASS" or "NEEDS REVIEW")
- Report includes visual evidence
- Page reloads after fix application (shows updated code)
- If dev server not detected, wizard should ask user to provide URL
- If browser not available, wizard should show graceful degradation message
- If screenshot fails, wizard should continue with other fixes and report partial evidence
Goal: Test browser-assisted verification of color/contrast fixes.
Duration: 15 minutes
Create test file with contrast issue:
<!-- test-contrast.html -->
<button style="color: #999; background: #EEE;">Low Contrast</button>-
Start web-accessibility-wizard:
/web-accessibility-wizard scan test-contrast.html at http://localhost:3000/test-contrast.html -
Wizard identifies: Low contrast on button (3.2:1)
-
At fix phase, ask wizard to:
Fix this button by changing text color to #000 -
Observe:
- Code is updated:
style="color: #000; background: #EEE;" - Browser verification takes screenshot
- Accessibility analyzer computes contrast: 14.5:1
- Reports: "Contrast improved from 3.2:1 to 14.5:1 (WCAG AAA)"
- Screenshot shows button with improved contrast
- Code is updated:
-
Verify metadata in screenshot:
- File contains timestamp
- URL is captured
- Fix number is in filename
- Contrast fix is applied to code
- Browser screenshot verifies new contrast
- Contrast ratio is computed correctly
- Before/after comparison shown
- Report shows "PASS" for verification
Goal: Test browser verification of keyboard/focus fixes.
Duration: 20 minutes
Create test file with focus issue:
<!-- test-focus.html -->
<style>
button:focus { outline: none; } /* Removes focus indicator */
</style>
<button id="submit">Submit Form</button>-
Start web-accessibility-wizard:
/web-accessibility-wizard scan test-focus.html for focus management -
Wizard identifies: Missing focus indicator on button
-
At fix phase, accepts proposal:
Yes, fix the focus indicator -
Browser verification performs:
- Adds
:focus-visiblerule with outline - Simulates Tab key press in browser
- Captures screenshot showing focus ring
- Reports: "Focus ring visible and correct"
- Screenshot shows blue outline around button when focused
- Adds
-
Review verification evidence:
- Screenshot clearly shows focus indicator
- File timestamp matches fix application time
- Metadata captured with URL and timestamp
- Focus fix is applied (CSS rule added)
- Browser simulates Tab key interaction
- Focus ring is visible in screenshot
- Verification reports "PASS"
Goal: Test graceful degradation when browser tools disabled.
Duration: 10 minutes
-
Disable browser tools:
- Open settings: Ctrl+,
- Search: "browser chat tools"
- Toggle OFF:
workbench.browser.enableChatTools
-
Reload window: Ctrl+Shift+P > Reload Window
-
Run web-accessibility-wizard:
/web-accessibility-wizard audit the example project -
When wizard offers Phase 12 (Browser Verification):
"Would you like to verify fixes in the browser?" -
Expected response:
Browser verification is not available in this environment. To enable browser-assisted verification: 1. Open VS Code settings 2. Search for "Browser Chat Tools" 3. Enable: workbench.browser.enableChatTools 4. Reload window Without browser tools, fixes are applied but not visually verified. -
Wizard still applies fixes but skips verification:
- Fixes are applied to code
- No screenshots captured
- Report shows "verification: NOT_AVAILABLE"
- Graceful degradation message displays correctly
- Fixes are still applied even without browser tools
- Wizard doesn't crash or hang
- User can enable browser tools and retry
Goal: Test behavior when Dev server is unreachable.
Duration: 10 minutes
-
Stop dev server:
# If running: Ctrl+C in terminal to stop npm run dev # This will be stopped
-
Keep browser tools enabled:
- Verify
workbench.browser.enableChatToolsis true
- Verify
-
Run web-accessibility-wizard:
/web-accessibility-wizard audit http://localhost:3000 -
At Phase 12 (Browser Verification), start fixes:
-
Browser attempts to navigate to http://localhost:3000
-
Connection times out after 10 seconds
-
Expected response:
Warning: The page didn't load within 10 seconds. Possible causes: - Server is not running at http://localhost:3000 - Network connection issue - Page has infinite loop Fix applied to code. Would you like to: - Start the dev server and retry verification - Continue without verification
-
-
Start dev server and retry:
- Open new terminal
npm run dev- Retry verification to Success
- Timeout handling works correctly
- User gets helpful error message with solutions
- Fixes are still applied even if server not running
- User can retry after starting server
Goal: Test handling of selector mismatch.
Duration: 10 minutes
-
Run web-accessibility-wizard on test page:
/web-accessibility-wizard audit http://localhost:3000 -
Wizard identifies and proposes fix:
- Issue: Missing alt on image with ID
#non-existent-img - Proposed fix: Add alt attribute to
#non-existent-img
- Issue: Missing alt on image with ID
-
Apply fix:
-
Code is updated but element doesn't exist on page
-
Browser verification attempts:
document.querySelector('#non-existent-img') -
Returns null
-
Expected response:
Warning: Element #non-existent-img not found on page at http://localhost:3000 Fix applied to code, but element is not on the current page. Possible causes: - Element ID changed or was removed - Selector is page-specific (not on homepage) - Element is inside a shadow DOM or modal Suggestions: 1. Navigate to the correct page where element exists 2. Verify element ID in HTML source 3. Manually test in browser
-
-
Observe behavior:
- Fix still applied to HTML
- Verification status: "FAILED" or "NEEDS REVIEW"
- Screenshot shows full page (for context)
- Error is reported clearly
- Missing elements are detected correctly
- User gets helpful error message
- Fixes are still applied to code
- Wizard doesn't crash
- User can navigate to correct page and retry
Goal: Test fallback when visual analysis cannot extract colors.
Duration: 10 minutes
-
Create test page with unusual content:
- Animated element (colors change)
- Element partially off-screen
- Very low opacity -Canvas/SVG with dynamic content
-
Run web-accessibility-wizard:
/web-accessibility-wizard audit test page with animated content -
Wizard identifies contrast issue on animated button:
-
Browser verification captures screenshot:
-
Screenshot is captured
-
Visual analyzer attempts to extract colors
-
Fails because element is animated (colors inconsistent)
-
Expected response:
Warning: Visual analysis couldn't verify the complete fix. The animated button made color analysis unreliable. Workaround: Open DevTools and manually inspect: - Right-click button > Inspect - Look for: computed color property - Check against background color Screenshot captured for your reference.
-
-
Observe:
- Screenshot is still saved even analysis failed
- User gets fallback instructions
- Verification status: "NEEDS_REVIEW"
- Wizard continues (doesn't crash)
- Visual analysis tries but gracefully fails
- Screenshot still captured
- User gets helpful fallback instructions
- Wizard continues without crashing
- Report shows partial evidence
Goal: Verify browser tools work across different frameworks.
Duration: 30 minutes (10 min per framework)
# Create React test project
npx create-react-app test-a11y-react
cd test-a11y-react
npm start # Runs on http://localhost:3000Run web-accessibility-wizard on React app:
/web-accessibility-wizard audit http://localhost:3000
Expected: Browser tools work with React
# Create Vue test project
npm create vite@latest test-a11y-vue -- --template vue
cd test-a11y-vue
npm install && npm run dev # Runs on http://localhost:5173Run web-accessibility-wizard:
/web-accessibility-wizard audit http://localhost:5173
Expected: Browser tools work with Vue
# Use example/ in this repo
cd example/
npm run dev # Runs on http://localhost:3000Run web-accessibility-wizard:
/web-accessibility-wizard audit http://localhost:3000
Expected: Browser tools work with vanilla HTML
- Browser tools work on React apps
- Browser tools work on Vue apps
- Browser tools work on vanilla HTML
- Dev servers with different ports all work
- Framework-specific CSS classes don't break screenshot capture
Goal: Verify screenshots are clear, timestamped, and usable.
Duration: 15 minutes
-
Run full audit with 5+ fixes:
/web-accessibility-wizard full audit of example project -
Apply all fixes and verify in browser
-
Check
.a11y-screenshots/directory:ls -la .a11y-screenshots/ # Expected output: # 2026-03-05-14-30-fix1-hero.png # 2026-03-05-14-30-fix2-button.png # 2026-03-05-14-30-fix3-nav.png # ... etc
-
Open screenshots in image viewer:
- File: 2026-03-05-14-30-fix1-hero.png
- Verify: Screenshot is clear, readable, not corrupted
- Verify: Timestamp is accurate
- Verify: Element is visible and clearly shows the fix
- Verify: Metadata (URL, timestamp) visible or in EXIF
-
Check report embedding:
- Open
WEB-ACCESSIBILITY-AUDIT.md - Verify: Screenshots are embedded with alt text
- Verify: Screenshot path format is valid:
.a11y-screenshots/2026-03-05-14-30-fix1-hero.png - Verify: Before/after comparison is clear
- Open
-
Test JPEG artifacts:
- PNG format expected (not JPEG)
- Verify: All screenshots are
.png - Verify: No compression artifacts
- Screenshots are clear and readable
- Naming convention is consistent:
{YYYY-MM-DD-HH-mm}-fix{n}-{selector}.png - Timestamps are accurate and match fix application time
- Metadata is captured (URL, browser, viewport)
- Embedded in report with working links
- Files are PNG format (lossless)
- All screenshots display correctly when opened
Goal: Document performance baselines for browser tools.
Duration: 20 minutes
For each test session, time the following operations:
| Operation | Target | Measured | Pass? |
|---|---|---|---|
| Open page in browser | < 5 sec | ? | Pass/Fail |
| Navigate to element | < 1 sec | ? | Pass/Fail |
| Take screenshot | < 2 sec | ? | Pass/Fail |
| Analyze screenshot | < 5 sec | ? | Pass/Fail |
| Report generation | < 3 sec | ? | Pass/Fail |
| Full fix cycle (5 fixes) | < 30 sec | ? | Pass/Fail |
# Terminal
time /accessibility-lead run browser verification for exampleOr measure manually with stopwatch:
- Screenshot 1: Test 2 (Fix Verification) - actual time taken
- Screenshot 2: Test 3 (Contrast) - time for each fix
- Screenshot 3: Test 4 (Focus) - total time for full report
Record in: BROWSER-TOOLS-PERFORMANCE.md
- Page opens within 5 seconds
- Screenshot capture < 2 seconds
- Full 5-fix cycle completes within 30 seconds total
- No hangs or timeouts
- Performance is consistent across supported OSes
Goal: Verify browser tools work on Windows and macOS.
Environment: VS Code on Windows 10/11
- Enable browser tools in settings
- Run Test 2 (Fix Verification)
- Check screenshot directory permissions
- Verify file paths use backslashes correctly
- Result: Pass/Fail
Environment: VS Code on macOS 13+
- Enable browser tools in settings
- Run Test 2 (Fix Verification)
- Check screenshot directory permissions
- Verify file paths use forward slashes
- Result: Pass/Fail
- Browser tools work on both supported OSes
- File paths are handled correctly per OS
- Permissions don't block screenshot creation
- Timestamps are consistent across timezones
- Screenshots are identical quality on all OSes
Tester: [Name]
Date: [YYYY-MM-DD]
Duration: [X hours]
OS: [Windows / macOS]
VS Code Version: [e.g., 1.110 (March 2026)]
| Test # | Description | Result | Notes |
|---|---|---|---|
| 1 | Browser tools availability | PASS | Browser tools detected correctly |
| 2 | Fix verification workflow | PASS | Full end-to-end workflow completed |
| 3 | Visual verification (contrast) | PASS | Contrast fix applied and verified |
| 4 | Interaction testing (focus) | PASS | Focus ring captured in screenshot |
| 5 | Failure: Browser not available | PASS | Graceful degradation works |
| 6 | Failure: Dev server not running | PASS | Helpful error message shown |
| 7 | Failure: Element not found | PASS | Fallback instructions provided |
| 8 | Failure: Analysis fails | PASS | Manual fallback works |
| 9 | Cross-framework testing | PASS | Works on React, Vue, vanilla HTML |
| 10 | Screenshot & evidence quality | PASS | Screenshots clear and timestamped |
Overall Result: ALL TESTS PASS
Issues Found:
- None
Recommendations:
- Phase 5.4 testing complete - ready for public release
Sign-off:
- [Tester Name]
- [Date]
- Approved for v3.0.0 release
After release, continuous tests to run:
- Open example project, run full audit
- Verify browser verification for 3+ fixes
- Check screenshots are captured
- Verify report generation
- Run test suite on every commit to
skills/web-*/SKILL.md - Verify no syntax errors in agent definitions
- Test agent invocation from CLI (if applicable)
- Check cross-agent compatibility
- Monitor GitHub issues for browser tool failures
- Track which failure modes users encounter most
- Plan Phase 5.5 enhancements based on feedback
- Documentation created (this file)
- Test 1-10 designed with step-by-step instructions
- Pass criteria defined for each test
- OS-specific testing documented
- Performance metrics framework established
- Results summary template created
- Actual testing performed on Windows
- Actual testing performed on macOS
- All tests passed
- Screenshots reviewed for quality
- Performance baselines recorded
- Continuous testing plan established
- Agents Overview
- Accessibility Lead
- Web Accessibility Wizard
- Document Accessibility Wizard
- Markdown Assistant
- Developer Hub
- GitHub Hub