Skip to content

roostergrin/qa-audit

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

3 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

QA Audit Toolkit

A comprehensive website quality assurance tool that crawls your site, captures screenshots, runs automated accessibility checks, pulls PageSpeed scores, and uses Claude to analyze everything and produce a detailed audit report.

Two Ways to Use It

1. Standalone Web App (no Claude Code required)

Run the web UI with just an Anthropic API key. Submit audits from your browser, track progress in real-time, and view reports — no CLI needed.

# Set your API key
export ANTHROPIC_API_KEY=sk-ant-...

# Start with Docker
docker compose up --build -d     # → http://localhost:3000

# Or start directly
./scripts/setup.sh               # one-time setup
npm start                        # → http://localhost:3000

2. Claude Code Slash Command

If you use Claude Code, run audits directly from the CLI:

/qa-audit https://example.com

Claude gathers all data, analyzes it, and saves a markdown report to reports/. It can also offer to fix issues if your source code is in the same repo.

What It Does

Give it a URL, and it will:

  1. Crawl every page (up to a configurable limit) using Playwright
  2. Screenshot each page at desktop (1280px) and mobile (375px) viewports
  3. Run axe-core accessibility scans on every page
  4. Fetch PageSpeed Insights scores (mobile + desktop) via the Google PSI API
  5. Check color contrast ratios against WCAG 2.1 AA standards
  6. Validate links — broken internal links, broken anchors, dead external links
  7. Analyze everything with Claude across 11 QA categories
  8. Generate a structured report with prioritized findings, severity ratings, and fix suggestions

The 11 Audit Categories

Category What It Checks
Accessibility (WCAG 2.1 AA) Alt text, contrast, ARIA, landmarks, heading structure, keyboard access
SEO & Meta Title tags, meta descriptions, Open Graph, Twitter Cards, canonical URLs, structured data
Security Mixed content, exposed secrets, missing rel="noopener", XSS vectors
Performance Image optimization, lazy loading, render-blocking scripts, font loading, page weight
Navigation & Links Broken links, dead anchors, navigation structure, link behavior
Visual & Layout Overflow, broken images, spacing issues, visual hierarchy (from screenshots)
Responsive Design Mobile vs desktop comparison, touch targets, stacking, text readability
Content & Copy Placeholder text, typos, broken template variables, missing legal pages
Forms & Interactivity Labels, validation, input types, interactive element states
Cross-Browser Compatibility CSS feature support, missing fallbacks, vendor prefixes
Print Styles Print-specific CSS, background colors on paper, hidden nav/footer

Example Findings

The tool catches real issues that matter, like:

  • A footer whose contrast ratio (3.85:1) fails WCAG AA (needs 4.5:1)
  • Canonical URLs pointing to a staging domain instead of production
  • noindex, nofollow accidentally left on a live site
  • A contact page showing metadata from a completely different business
  • Office hours displaying "64:30 pm" instead of "4:30 pm"
  • Broken internal navigation links across the entire site

Every finding includes the severity, exact element/selector, why it matters, and a concrete fix.

Example Report Output

Here's a trimmed example from a real audit (6 pages, 61 findings):

# QA Audit Report: https://example-orthodontics.com/
## Audit Date: 2026-02-09
## Audit Tier: standard

## Data Sources Used
- [x] Visual analysis (desktop + mobile screenshots)
- [x] HTML source analysis
- [x] axe-core automated accessibility scan — 18 violations across 6 pages
- [x] Google PageSpeed Insights (Lighthouse) — Mobile 68/100, Desktop 86/100
- [x] Plugin check results — 4 broken anchor links detected

## Executive Summary
The site has serious SEO issues requiring immediate attention. Every page has its
canonical URL pointing to a staging domain, all pages block search engines with
noindex/nofollow, and two pages display metadata from entirely different businesses.
The site also has widespread WCAG 2.1 AA color contrast failures in the footer and
section headings across all 6 pages.

## Performance Scores
| Metric                   | Mobile     | Desktop    |
| ------------------------ | ---------- | ---------- |
| Overall Score            | 68/100     | 86/100     |
| Largest Contentful Paint | 6.8 s      | 1.6 s      |
| Cumulative Layout Shift  | 0.002      | 0.003      |
| Total Blocking Time      | 320 ms     | 140 ms     |

## Score Overview
| Category                     | Issues | Severity Breakdown              |
| ---------------------------- | ------ | ------------------------------- |
| Accessibility (WCAG 2.1 AA)  | 15     | 5 critical, 8 warning, 2 info  |
| SEO & Meta                   | 10     | 5 critical, 4 warning, 1 info  |
| Security                     | 4      | 0 critical, 3 warning, 1 info  |
| Performance                  | 7      | 0 critical, 4 warning, 3 info  |
| Navigation & Links           | 5      | 1 critical, 2 warning, 2 info  |
| ...                          |        |                                 |
| **TOTAL**                    | **61** | **16 critical, 31 warning, 14 info** |

## Critical Issues (must fix)

### CRIT-001: Canonical URL points to staging template domain on all pages
- **Page:** All 6 pages
- **Category:** SEO & Meta
- **Source:** HTML
- **What:** Every page's `<link rel="canonical">` points to a staging domain.
- **Where:** `<link rel="canonical" href="https://staging.example.com/">`
- **Why it matters:** Search engines will index the wrong domain.
- **Fix:** Update the Nuxt.js config to generate correct canonical URLs per page.

### CRIT-002: Footer color contrast fails WCAG AA on all pages (3.85:1)
- **Page:** All 6 pages
- **Category:** Accessibility
- **Source:** axe-core
- **What:** All footer text uses white (#fff) on #6186a3, yielding 3.85:1.
  Normal text requires 4.5:1.
- **Where:** `footer .footer__link`, `.footer__copyright-item`
- **Fix:** Darken the footer background to at least #4e6f8c.
- **WCAG:** 1.4.3 Contrast (Minimum)

...

## Recommended Fix Order
1. CRIT-001 + CRIT-003 — Fix canonical + og:url (single config change, all 6 pages)
2. CRIT-002 — Remove noindex/nofollow (site is invisible to Google)
3. CRIT-006 + CRIT-007 — Fix contrast (one CSS variable change, all pages)
...

Quick Start

Prerequisites

  • Node.js 18+
  • An Anthropic API key (ANTHROPIC_API_KEY)

Setup

git clone <this-repo>
cd qa-audit
./scripts/setup.sh               # installs deps + Playwright Chromium

Option A: Web UI

export ANTHROPIC_API_KEY=sk-ant-...
npm start                        # → http://localhost:3000

Open the UI in your browser, paste a URL, pick a tier and model, and submit. The audit runs in the background and the report appears when done.

Option B: Claude Code

# Inside Claude Code:
/qa-audit https://example.com

# For sites with anti-bot protection (Cloudflare, Wordfence):
/qa-audit https://example.com --remote

Option C: Batch Audits

# Audit multiple sites from a file (one URL per line)
./scripts/batch-audit.sh urls.txt

Docker

# Build and run everything in a container
docker compose up --build -d     # → http://localhost:3000
docker compose logs -f           # view logs
docker compose down              # stop

Set ANTHROPIC_API_KEY in your .env file or pass it via the Docker environment.

How It Works

URL submitted (Web UI or /qa-audit)
     │
     ▼
┌─────────────────────────┐
│  Data Gathering          │
│                         │
│  1. crawl.js            │  Playwright crawl → screenshots + HTML + axe-core
│  2. mcp-call.cjs        │  PageSpeed Insights API (mobile + desktop)
│  3. mcp-call.cjs        │  AccessLint contrast checks
│  4. run-checks.cjs      │  Link validation plugins
│  5. data-summary        │  Generates manifest + summary
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────┐
│  Claude Analysis        │  Reads screenshots, HTML, axe results,
│                         │  PageSpeed data, contrast data
│  11 categories ×        │  Analyzes every page against the full
│  every page             │  QA checklist
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────┐
│  Markdown Report        │  Saved to reports/<domain>-<timestamp>-audit.md
│                         │
│  - Executive summary    │
│  - Prioritized findings │
│  - Page-by-page breakdown│
│  - Recommended fix order│
└─────────────────────────┘

Configuration

All settings live in config.json:

{
  "crawl": {
    "maxPages": 20,
    "delayMs": 1000,
    "screenshots": { "desktop": { "width": 1280 }, "mobile": { "width": 375 } }
  },
  "output": {
    "reportsDir": "reports",
    "filenameFormat": "<domain>-<timestamp>-audit.md"
  },
  "accessibility": {
    "standard": "WCAG2AA",
    "contrastRatios": { "normalText": 4.5, "largeText": 3.0 }
  }
}

See config.json for the full set of options including model selection, analysis token limits, API server settings, and audit tiers.

Audit Tiers

Tier What It Does
quick Fewer categories, faster results
standard All 11 categories (default)
full All categories with deeper analysis

Models

Configurable in config.json under models. Supports both Anthropic (Claude) and OpenAI providers. Set OPENAI_API_KEY if using OpenAI models.

Environment Variables

Variable Required Description
ANTHROPIC_API_KEY Yes Anthropic API key for Claude
OPENAI_API_KEY No OpenAI API key (only if using OpenAI models)
PORT No Server port (default: 3000)
DEFAULT_TIER No Default audit tier (default: standard)
DEFAULT_MODEL No Default model (default: sonnet)
MAX_CONCURRENT_JOBS No Concurrent audit limit (default: 1)

API Endpoints

The Express server exposes a REST API for programmatic access:

Method Endpoint Description
POST /api/audits Submit a new audit ({ url, tier?, model?, maxPages?, remote? })
GET /api/audits List recent audits
GET /api/audits/:id Job status + report when complete
GET /api/audits/:id/report Raw markdown report
GET /api/audits/:id/pages Crawled pages with screenshot paths
PATCH /api/audits/:id/archive Archive/unarchive an audit
GET /api/health Health check

Project Structure

├── config.json                 # All settings (single source of truth)
├── server.js                   # Express API + static file server
├── worker.js                   # Background job queue and processing
├── lib/
│   ├── config.js               # Config loader with helper functions
│   ├── categories.js           # 11 QA categories with prompts
│   ├── prompts.js              # Report templates and severity definitions
│   └── gather-data.js          # Data gathering orchestration
├── scripts/
│   ├── qa-audit-full.sh        # Main entry point — runs all data gathering
│   ├── crawl.js                # Playwright crawler
│   ├── run-checks.cjs          # Link validation plugins
│   ├── mcp-call.cjs            # PageSpeed + AccessLint data fetching
│   ├── analyze-page.cjs        # Per-page AI analysis
│   ├── batch-audit.sh          # Multi-site batch runner
│   ├── generate-checklist.js   # Generates qa-checklist.md from lib modules
│   ├── setup.sh                # One-time setup (installs deps + Chromium)
│   └── preflight-check.sh      # Verify environment is ready
├── frontend/                   # React web UI (Vite + Tailwind + Radix)
├── .claude/commands/
│   └── qa-audit.md             # Slash command definition for Claude Code
├── tests/                      # Vitest test suite
├── reports/                    # Generated audit reports
├── Dockerfile
└── docker-compose.yml

Testing

npm test             # run all tests
npm run test:watch   # watch mode

Dependencies

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors