Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions .github/workflows/daily-multi-device-docs-tester.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

29 changes: 19 additions & 10 deletions .github/workflows/daily-multi-device-docs-tester.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,17 @@ permissions:
tracker-id: daily-multi-device-docs-tester
engine:
id: claude
max-turns: 50 # Prevent runaway token usage (10 devices × ~3 turns + build/setup/report)
max-turns: 80 # 10 devices × ~5 turns each + setup/report overhead
strict: true
timeout-minutes: 30
tools:
timeout: 120 # Playwright navigation on Astro dev server can take >60s; increase to 120s
playwright:
version: "v1.56.1"
bash:
- "npm install*"
- "npm run build*"
- "npm run preview*"
- "npm run dev*"
- "npx astro*"
- "npx playwright*"
- "curl*"
- "kill*"
Expand Down Expand Up @@ -70,20 +71,19 @@ You are a documentation testing specialist. Your task is to comprehensively test

## Your Mission

Build the documentation site locally, serve it, and perform comprehensive multi-device testing. Test layout responsiveness, accessibility, interactive elements, and visual rendering across all device types. Use a single Playwright browser instance for efficiency.
Start the documentation development server and perform comprehensive multi-device testing. Test layout responsiveness, accessibility, interactive elements, and visual rendering across all device types. Use a single Playwright browser instance for efficiency.

## Step 1: Build and Serve
## Step 1: Install Dependencies and Start Server

Navigate to the docs folder and build the site:
Navigate to the docs folder and install dependencies:

```bash
cd ${{ github.workspace }}/docs
npm install
npm run build
```

Follow the shared **Documentation Server Lifecycle Management** instructions:
1. Start the preview server (section "Starting the Documentation Preview Server")
1. Start the dev server (section "Starting the Documentation Preview Server")
2. Wait for server readiness (section "Waiting for Server Readiness")

## Step 2: Device Configuration
Expand All @@ -100,10 +100,18 @@ Test these device types based on input `${{ inputs.devices }}`:

Playwright is provided through an MCP server interface, **NOT** as an npm package. You must use the MCP Playwright tools:

- ✅ **Correct**: Use MCP tools like `mcp__playwright__browser_navigate`, `mcp__playwright__browser_run_code`, etc.
- ✅ **Correct**: Use `mcp__playwright__browser_run_code` with `page.goto(..., { waitUntil: 'domcontentloaded' })`
- ❌ **Incorrect**: Do NOT try to `require('playwright')` or create standalone Node.js scripts
- ❌ **Incorrect**: Do NOT install playwright via npm - it's already available through MCP

**⚠️ CRITICAL: Navigation Timeout Prevention**

The Astro development server uses Vite, which loads many JavaScript modules per page. Using the default `waitUntil: 'load'` or `waitForLoadState('networkidle')` will cause 60s timeouts because the browser waits for all modules to finish. **Always use `waitUntil: 'domcontentloaded'`** for navigation:

- ✅ **Correct**: `page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 })`
- ❌ **Never use**: `page.waitForLoadState('networkidle')` — causes guaranteed timeouts
- ❌ **Never use**: `mcp__playwright__browser_navigate` for first load — it uses default 'load' wait which times out

**Example Usage:**

```bash
Expand All @@ -117,10 +125,11 @@ echo "Playwright server URL: http://${SERVER_IP}:4321/gh-aw/"
// Use browser_run_code to execute Playwright commands.
// IMPORTANT: Replace 172.30.0.20 below with the actual SERVER_IP from the bash command above.
// Do NOT use "localhost" — Playwright runs with --network host so its localhost differs.
// ALWAYS use waitUntil: 'domcontentloaded' to prevent timeout on the Vite dev server.
mcp__playwright__browser_run_code({
code: `async (page) => {
await page.setViewportSize({ width: 390, height: 844 });
await page.goto('http://172.30.0.20:4321/gh-aw/'); // substitute actual SERVER_IP
await page.goto('http://172.30.0.20:4321/gh-aw/', { waitUntil: 'domcontentloaded', timeout: 30000 }); // substitute actual SERVER_IP
return { url: page.url(), title: await page.title() };
}`
})
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/docs-noob-tester.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 19 additions & 15 deletions .github/workflows/shared/docs-server-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,49 +2,52 @@
# Documentation Server Lifecycle Management
#
# This shared workflow provides instructions for starting, waiting for readiness,
# and cleaning up the Astro Starlight documentation preview server.
# and cleaning up the Astro Starlight documentation dev server.
#
# Prerequisites:
# - Documentation must be built first (npm run build in docs/ directory)
# - Bash permissions: npm *, curl *, kill *, echo *, sleep *
# - npm install must have been run in docs/ directory
# - Bash permissions: npm *, npx *, curl *, kill *, echo *, sleep *
# - Working directory should be in repository root
---

## Starting the Documentation Preview Server

**Context**: The documentation has been pre-built using `npm run build`. Use the preview server to serve the static build.

Navigate to the docs directory and start the preview server in the background, binding to all network interfaces on a fixed port:
Navigate to the docs directory and start the development server in the background, binding to all network interfaces on a fixed port:

```bash
cd docs
npm run preview -- --host 0.0.0.0 --port 4321 > /tmp/preview.log 2>&1 &
npm run dev -- --host 0.0.0.0 --port 4321 > /tmp/preview.log 2>&1 &
echo $! > /tmp/server.pid
Comment on lines 13 to 20
```

This will:
- Start the preview server on port 4321, bound to all interfaces (`0.0.0.0`)
- Start the Astro development server on port 4321, bound to all interfaces (`0.0.0.0`)
- Redirect output to `/tmp/preview.log`
- Save the process ID to `/tmp/server.pid` for later cleanup

**Why `npm run dev` instead of `npm run preview`:**
The `npm run preview` command serves the pre-built static output. However, Astro's Starlight documentation site uses hybrid routing which requires the development server (`astro dev`) to correctly serve all pages at the `/gh-aw/` base URL. Using `npm run preview` returns 404 for `/gh-aw/` paths.

**Why `--host 0.0.0.0 --port 4321` is required:**
The agent runs inside a Docker container. Playwright also runs in its own container with `--network host`, meaning its `localhost` is the Docker host — not the agent container. Binding to `0.0.0.0` makes the server accessible on the agent container's bridge IP (e.g. `172.30.x.x`). The `--port 4321` flag prevents port conflicts if a previous server instance is still shutting down.

## Waiting for Server Readiness

Poll the server with curl to ensure it's ready before use:
Poll the server with curl until the `/gh-aw/` path returns HTTP 200:

```bash
for i in {1..30}; do
curl -s http://localhost:4321 > /dev/null && echo "Server ready!" && break
echo "Waiting for server... ($i/30)" && sleep 2
for i in {1..45}; do
STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:4321/gh-aw/)
[ "$STATUS" = "200" ] && echo "Server ready at http://localhost:4321/gh-aw/!" && break
echo "Waiting for server... ($i/45) (status: $STATUS)" && sleep 3
done
Comment on lines +41 to 43
```

This will:
- Attempt to connect up to 30 times (60 seconds total)
- Wait 2 seconds between attempts
- Exit successfully when server responds
- Attempt to connect up to 45 times (135 seconds total) to allow for Astro dev server startup
- Check that `/gh-aw/` specifically returns HTTP 200 (not just that the port is open)
- Wait 3 seconds between attempts
- Exit successfully when the docs site is fully accessible

## Playwright Browser Access

Expand Down Expand Up @@ -94,3 +97,4 @@ This will:
- For Playwright browser tools, use the container bridge IP (see "Playwright Browser Access" section above)
- Always clean up the server when done to avoid orphan processes
- If the server fails to start, check `/tmp/preview.log` for errors
- No `npm run build` step is required before starting the dev server
2 changes: 1 addition & 1 deletion .github/workflows/unbloat-docs.lock.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading