Add daily documentation testing workflow with beginner perspective

[Copilot]

Create Documentation Testing Agentic Workflow ✅

Successfully created a "noob" agentic workflow that tests the documentation site and reports confusing or broken steps.

Completed Tasks

• Explore repository structure and understand existing workflows
• Build gh-aw binary successfully
• Create new workflow file: .github/workflows/docs-noob-tester.md
• Configure workflow with:
  • ✅ Daily schedule (10 AM UTC)
  • ✅ GitHub Copilot engine
  • ✅ Playwright for browsing (unpinned version for flexibility)
  • ✅ Safe outputs for discussion creation
  • ✅ Screenshot upload capability
  • ✅ Bash tools (all commands allowed with wildcard)
  • ✅ Network access with defaults + node
  • ✅ Minimal permissions (contents: read only)
• Test workflow compilation with ./gh-aw compile
• Verify generated lock file
• Run unit tests to ensure no regressions (all pass)
• Format code (all formatted)
• Manual validation of workflow structure
• Verify workflow shows in gh-aw status
• Address all PR feedback

Recent Changes (PR Feedback)

• Removed Playwright version pinning (using latest by default)
• Removed npm run dev* from bash commands (not needed)
• Added defaults to network allowed list (follows convention)
• Removed discussions: write permission (safe-outputs handles permissions automatically)
• Simplified bash configuration to use wildcard "*" (allows all commands)
• Switched from codex to copilot engine (GitHub Copilot agent for better integration and network firewalling support)

Implementation Summary

Workflow File: .github/workflows/docs-noob-tester.md
Compiled File: .github/workflows/docs-noob-tester.lock.yml

Key Configuration:

on:
  schedule:
    - cron: '0 10 * * *'  # Daily at 10 AM UTC
  workflow_dispatch:  # Can also be triggered manually
permissions:
  contents: read  # Minimal permissions - safe-outputs handles write permissions
engine: copilot  # GitHub Copilot agent
tools:
  playwright:
    allowed_domains: ["localhost", "127.0.0.1"]
  bash:
    - "*"  # Allow all bash commands
safe-outputs:
  upload-assets:
  create-discussion:
    category: "General"
network:
  allowed:
    - defaults
    - node

What the Workflow Does:

1. Builds Documentation Site: Uses npm to install dependencies and build the docs
2. Serves Locally: Starts preview server on port 4321
3. Acts as Beginner: Navigates documentation as a complete novice
4. Tests Key Pages: Home, Quick Start, CLI Commands, Creating Workflows, Examples
5. Takes Screenshots: Captures images of confusing or broken sections
6. Categorizes Issues: 🔴 Critical / 🟡 Confusing / 🟢 Working well
7. Creates Discussion: Generates detailed report with findings and recommendations
8. Cleans Up: Stops the preview server

Testing:

• ✅ Workflow compiles successfully
• ✅ All unit tests pass
• ✅ Ready for manual trigger via workflow_dispatch
• ✅ Follows repository conventions for safe-outputs workflows
• ✅ Uses GitHub Copilot agent for improved performance and network firewalling support

Original prompt

Create a noob agentic workflow that runs the documentation and tries to discover and get started with the project. Report confusing or broken steps discovered in the docs.

See docs.yml for steps to setup local documentation server, enable playwright for browsing

Use codex

Run daily

Post screenshots of confusing pages.

Create discussion with results.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[Copilot]

Pull Request Overview

This PR adds an automated documentation quality assurance workflow that tests the documentation from a beginner's perspective. The workflow uses the Codex engine with Playwright to navigate the documentation site, identify confusing or broken steps, and report findings in GitHub discussions.

Key Changes:

• Daily automated testing of documentation from a novice user perspective
• Browser automation using Playwright to navigate key documentation pages
• Comprehensive reporting via GitHub discussions with screenshots and categorized findings

Reviewed Changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated no comments.

File	Description
.github/workflows/docs-noob-tester.md	Main workflow definition with Codex engine, Playwright tools, and safe-outputs for discussion creation
.github/workflows/docs-noob-tester.lock.yml	Compiled GitHub Actions workflow (auto-generated, 3967 lines)

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[pelikhan]

@copilot do not pin

[Copilot]

Removed version pinning. Playwright will now use the latest version. (4bc0611)

[pelikhan]

@copilot remove, not needed thanks to safe outputs

[Copilot]

Removed discussions: write permission. Safe-outputs handles write permissions automatically. (378c948)

[pelikhan]

@copilot allow * in bash

[Copilot]

Simplified bash configuration to allow all commands with "*" wildcard. (ca08fc1)

[pelikhan]

@copilot use copilot agent engine

[Copilot]

@copilot use copilot agent engine

Changed engine from codex to copilot. (6d47491)