Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -602,7 +602,7 @@ Supported runtime commands today:

`wordpress.core-phpunit` **requires the mounted `wordpress-develop` checkout to already have its Composer dev dependencies installed** before you mount it. WordPress core's `tests/phpunit/includes/bootstrap.php` hard-requires the test toolchain (PHPUnit plus the Yoast PHPUnit Polyfills at `vendor/yoast/phpunit-polyfills/phpunitpolyfills-autoload.php`) and `die()`s if it is absent — a freshly cloned `wordpress-develop` tree has **no `vendor/`**. Run `composer install` (or `composer update -W`) inside the checkout first, or mount a checkout that already has `vendor/`. WP Codebox does **not** silently fetch these dependencies for you (sandbox network downloads remain gated behind `WP_CODEBOX_ALLOW_NETWORK_DOWNLOADS=1`). When the toolchain is missing, the command now fails with a clear, structured error naming the missing paths instead of crashing with an opaque "crashed before producing a structured response" — the pre-flight check runs before core's bootstrap, and a mid-`require` `die()` is captured via output buffering + a shutdown handler so diagnostics always reach `files/core-phpunit/.pg-test-result.txt`.

`wordpress.browser-probe` accepts `wait-for=domcontentloaded|load|networkidle|selector:<selector>|duration`, `duration=<n>s`, and `capture=console,errors,html,network,performance,memory,screenshot`. It records machine-readable evidence refs such as `files/browser/console.jsonl`, `files/browser/errors.jsonl`, `files/browser/network.jsonl`, `files/browser/performance.json`, `files/browser/memory.json`, `files/browser/checkpoints.jsonl`, `files/browser/snapshot.html`, `files/browser/screenshot.png`, and `files/browser/summary.json` when those captures are enabled. The summary includes requested/final URLs, viewport/device metadata, HTML and screenshot hashes, network event counts, optional final/peak browser memory and performance summaries, and a generic `artifact-backed|partial|diagnostic-only` replayability classification. Performance and memory captures use generic browser/CDP data only: JS heap when available, CDP `Performance.getMetrics`, CDP DOM counters, DOM/resource counts and byte totals, and long task counts/duration. Probe scripts may call `window.__wpCodeboxProbeCheckpoint(name, metadata)` when `performance` or `memory` capture is enabled to record named generic checkpoint snapshots. WP Codebox intentionally keeps these browser evidence fields generic; consumers such as eval harnesses may interpret them without WP Codebox adding scoring, grading, or benchmark semantics.
`wordpress.browser-probe` accepts `wait-for=domcontentloaded|load|networkidle|selector:<selector>|duration`, `duration=<n>s`, `viewport=<width>x<height>` (for example `viewport=390x844`), and `capture=console,errors,html,network,performance,memory,screenshot`. It records machine-readable evidence refs such as `files/browser/console.jsonl`, `files/browser/errors.jsonl`, `files/browser/network.jsonl`, `files/browser/performance.json`, `files/browser/memory.json`, `files/browser/checkpoints.jsonl`, `files/browser/snapshot.html`, `files/browser/screenshot.png`, and `files/browser/summary.json` when those captures are enabled. The summary includes requested/final URLs, effective viewport/device metadata, HTML and screenshot hashes, network event counts, optional final/peak browser memory and performance summaries, and a generic `artifact-backed|partial|diagnostic-only` replayability classification. Performance and memory captures use generic browser/CDP data only: JS heap when available, CDP `Performance.getMetrics`, CDP DOM counters, DOM/resource counts and byte totals, and long task counts/duration. Probe scripts may call `window.__wpCodeboxProbeCheckpoint(name, metadata)` when `performance` or `memory` capture is enabled to record named generic checkpoint snapshots. WP Codebox intentionally keeps these browser evidence fields generic; consumers such as eval harnesses may interpret them without WP Codebox adding scoring, grading, or benchmark semantics.

`wordpress.browser-actions` drives the preview with an ordered interaction script so Codebox can prove a plugin still *works* under interaction, not just that it renders. Pass the script as `steps-json=<array>` (inline JSON, or `@<path>` to read it from a file); the legacy `actions-json=<array>` shape is still accepted and normalized to steps. Each step is a thin, stable mapping over a Playwright locator action — this is not a test-runner DSL.

Expand Down
24 changes: 24 additions & 0 deletions packages/runtime-playground/src/browser-command-runners.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ export async function runBrowserProbeCommand({

const waitFor = argValue(args, "wait-for")?.trim() || "domcontentloaded"
const durationMs = durationArg(args, "duration", 0)
const requestedViewport = viewportArg(args, "viewport")
const script = argValue(args, "script")
const capturesBrowserMetrics = capture.has("performance") || capture.has("memory")
const targetUrl = resolveBrowserProbeUrl(urlArg, server.serverUrl)
Expand Down Expand Up @@ -96,6 +97,9 @@ export async function runBrowserProbeCommand({

try {
page = await browser.newPage()
if (requestedViewport) {
await page.setViewportSize(requestedViewport)
}
if (capturesBrowserMetrics) {
await page.addInitScript(BROWSER_PROBE_PERFORMANCE_INIT_SCRIPT)
}
Expand Down Expand Up @@ -1203,3 +1207,23 @@ function durationArg(args: string[], name: string, fallbackMs: number): number {
const value = Number.parseFloat(match[1])
return Math.max(0, Math.round(match[2] === "ms" ? value : value * 1000))
}

function viewportArg(args: string[], name: string): { width: number; height: number } | undefined {
const raw = argValue(args, name)?.trim()
if (!raw) {
return undefined
}

const match = raw.match(/^(\d+)x(\d+)$/i)
if (!match) {
throw new Error(`${name} must use <width>x<height>, for example 390x844: ${raw}`)
}

const width = Number.parseInt(match[1] ?? "", 10)
const height = Number.parseInt(match[2] ?? "", 10)
if (!Number.isFinite(width) || !Number.isFinite(height) || width <= 0 || height <= 0) {
throw new Error(`${name} width and height must be positive integers: ${raw}`)
}

return { width, height }
}
9 changes: 6 additions & 3 deletions scripts/browser-probe-artifact-smoke.ts
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ await writeFile(recipePath, `${JSON.stringify({
"url=/",
"wait-for=load",
"duration=1s",
"viewport=390x844",
"capture=console,errors,html,network,performance,memory,screenshot",
"script=window.__wpCodeboxProbeCheckpoint('fixture-before-return', { source: 'smoke' }); console.info('wp-codebox fixture browser script'); return { title: document.title, hasBody: !!document.body };",
],
Expand Down Expand Up @@ -151,6 +152,8 @@ assert.equal(summary.files.network, "files/browser/network.jsonl")
assert.equal(summary.files.performance, "files/browser/performance.json")
assert.match(summary.hashes.html?.value ?? "", /^[a-f0-9]{64}$/)
assert.match(summary.hashes.screenshot?.value ?? "", /^[a-f0-9]{64}$/)
assert.equal(summary.viewport.width, 390, "summary should record requested viewport width")
assert.equal(summary.viewport.height, 844, "summary should record requested viewport height")
assert.equal(summary.summary.replayability, "artifact-backed")
assert.equal(summary.summary.htmlSnapshot, true)
assert.equal(summary.summary.scriptResult?.title, "My WordPress Website")
Expand All @@ -159,8 +162,6 @@ assert.ok(summary.summary.networkEvents >= 1, "summary should count network even
assert.ok((summary.summary.memory?.domNodes.final ?? 0) > 0, "summary should include memory DOM node counts")
assert.ok((summary.summary.performance?.resources ?? 0) >= 1, "summary should include performance resource counts")
assert.ok((summary.summary.performance?.domNodes.final ?? 0) > 0, "summary should include performance DOM node counts")
assert.ok(summary.viewport.width > 0, "summary should include viewport width")
assert.ok(summary.viewport.height > 0, "summary should include viewport height")
assert.ok(summary.viewport.userAgent.length > 0, "summary should include user agent")

const manifest = JSON.parse(await readFile(manifestPath, "utf8")) as { files: Array<{ path: string; kind: string }> }
Expand All @@ -173,7 +174,7 @@ assert.ok(manifest.files.some((file) => file.path === "files/browser/network.jso
assert.ok(manifest.files.some((file) => file.path === "files/browser/performance.json" && file.kind === "browser-performance"))
assert.ok(manifest.files.some((file) => file.path === "files/browser/screenshot.png" && file.kind === "browser-screenshot"))

const review = JSON.parse(await readFile(reviewPath, "utf8")) as { browser?: { probes?: Array<{ consoleMessages: number; errors: number; checkpoints?: string; html?: string; memory?: string; network?: string; performance?: string; finalUrl?: string; replayability?: string }> } }
const review = JSON.parse(await readFile(reviewPath, "utf8")) as { browser?: { probes?: Array<{ consoleMessages: number; errors: number; checkpoints?: string; html?: string; memory?: string; network?: string; performance?: string; finalUrl?: string; replayability?: string; viewport?: { width: number; height: number } }> } }
assert.ok(review.browser?.probes?.[0], "review should include browser probe summary")
assert.ok(review.browser.probes[0].consoleMessages >= 1, "review should count console messages")
assert.ok(review.browser.probes[0].errors >= 1, "review should count browser errors")
Expand All @@ -183,6 +184,8 @@ assert.equal(review.browser.probes[0].memory, "files/browser/memory.json")
assert.equal(review.browser.probes[0].network, "files/browser/network.jsonl")
assert.equal(review.browser.probes[0].performance, "files/browser/performance.json")
assert.equal(review.browser.probes[0].replayability, "artifact-backed")
assert.equal(review.browser.probes[0].viewport?.width, 390, "review should record requested viewport width")
assert.equal(review.browser.probes[0].viewport?.height, 844, "review should record requested viewport height")
assert.equal(review.browser.probes[0].finalUrl?.endsWith("/"), true, "review should include final URL")

console.log(`Browser probe artifact smoke passed: ${artifactDirectory}`)
Expand Down