Skip to content

feat: interaction-capable browser probe — drive WordPress, not just observe it (#310)#311

Merged
chubes4 merged 1 commit into
mainfrom
feat-310-interaction-probe
May 30, 2026
Merged

feat: interaction-capable browser probe — drive WordPress, not just observe it (#310)#311
chubes4 merged 1 commit into
mainfrom
feat-310-interaction-probe

Conversation

@chubes4
Copy link
Copy Markdown
Collaborator

@chubes4 chubes4 commented May 30, 2026

Summary

Closes #310. Upgrades wordpress.browser-actions from the partial navigate/click/fill/press action shape into the full ordered interaction-script contract, so WP Codebox can drive a real WordPress UI and assert browser behavior — not just prove it renders. Every consumer (Homeboy trace, CI, agent stacks, wp-gym, mobile/desktop tools) inherits a portable E2E capability against a disposable WordPress Playground for free.

Design decisions (the architectural fork)

Extend the existing sibling, don't add a third command. A wordpress.browser-actions command already existed (commit 93c3c12) with navigate/click/fill/press/wait/capture, but it fell short of #310: no schema in runtime-core, no evaluate/drag/hover/select/expect/type, no machine-readable assertions, no steps.jsonl, no per-step/total timeouts. Rather than add a third overlapping wordpress.browser-scenario command (premature proliferation), I upgraded browser-actions in place to the full contract. This satisfies every acceptance criterion while keeping one interaction command.

How steps are passed. ExecutionSpec.args is a flat string[] of key=value, so a multi-step script can't live in flat args. It's passed as a structured steps-json=<inline-json | @path> payload — the same *-json convention recipes already use (actions-json, env-json, workloads-json) resolved via the existing JSON-arg helpers. The legacy actions-json shape is still accepted and normalized to steps, so existing recipes keep working.

Where the schema lives (layer purity). The backend-agnostic step contract — kinds, validateBrowserInteractionScript(), browserInteractionScriptUsesEvaluate() — is declared in runtime-core. The Playwright executor lives in runtime-playground. runtime-core knows nothing about Playwright.

Evaluate policy gate. The arbitrary-JS evaluate step is gated separately behind a new wordpress.browser-actions.evaluate policy capability (mirroring wordpress.run-php's assertRuntimeCommandAllowed posture). Non-JS interaction steps are allowed whenever wordpress.browser-actions is allowed; a consumer can permit UI driving while forbidding arbitrary page JS. Recipes auto-grant the capability when a step opts into an evaluate step.

Step kinds

navigate, click, fill, type, press, drag, hover, select, waitFor, evaluate, expect, screenshot, capture — each a 1:1 map onto a stable Playwright locator action (getByRole via role= selectors, dragTo, selectOption, locator state checks for expect, etc.).

Artifacts & determinism

  • Per-step results append to files/browser/steps.jsonl (index, kind, selector, ok/fail, durationMs, any named screenshot).
  • expect/evaluate assertions produce a machine-readable assertions block (total/passed/failed + each result) in action-summary.json and the review summary, so any consumer can gate on it.
  • Named screenshots (screenshot step with name) write files/browser/screenshot-<name>.png.
  • Per-step timeout plus global step-timeout and total timeout budgets; the run stops cleanly on the first failing step with a probe-error-style record — no silent partial success.
  • Manifest (browser-steps kind), review.json, and artifact redaction all include steps.jsonl.

Example steps script

[
  { "kind": "click",      "selector": "role=button[name='Social']" },
  { "kind": "waitFor",    "selector": ".reactEasyCrop_Container" },
  { "kind": "drag",       "from": ".reactEasyCrop_CropArea", "to": { "x": 40, "y": 40 } },
  { "kind": "fill",       "selector": "#caption", "value": "smoke test" },
  { "kind": "evaluate",   "expression": "document.querySelector('.crop').isConnected", "assert": true },
  { "kind": "expect",     "selector": ".crop-confirm", "state": "visible" },
  { "kind": "screenshot", "name": "after-crop" }
]

Verification

  • npm run build green across all three packages.
  • New browser-interaction-script-validation-smoke (unit-level, no browser) covers every kind, evaluate detection, and per-kind required-field validation.
  • Extended browser-actions-artifact-smoke runs a real multi-step script (fill → click → waitFor → expect → evaluate(assert) → named screenshot) against live Playground+Playwright and asserts steps.jsonl, the assertions block, manifest, and review.json. Passes.
  • command-registry-smoke, discovery-command-smoke, recipe-dry-run-smoke, recipe-browser-smoke, browser-probe-artifact-smoke, runtime-action-adapter-smoke, and package-distribution-smoke all pass — navigate-only and legacy actions-json paths unchanged.

Notes

  • Did not touch CHANGELOG.md or version strings (homeboy owns those).
  • The wordpress.browser-actions.evaluate entry is a policy-only capability (recipe: false, not a runnable Playground handler); it documents the gate and is excluded from the recipe JSON schema.

Upgrade wordpress.browser-actions from the partial navigate/click/fill/press
action shape into the full ordered interaction-script contract from issue #310,
turning Codebox into a portable WordPress E2E sandbox that can drive the UI and
assert browser behavior, not just observe a render.

- Declare the backend-agnostic step schema (kinds, validation, evaluate
  detection) in runtime-core; implement the executor in runtime-playground,
  keeping the layer boundary clean.
- Step kinds: navigate, click, fill, type, press, drag, hover, select, waitFor,
  evaluate, expect, screenshot, capture — each a 1:1 map onto a stable locator
  action. Pass the script via steps-json (inline or @path); actions-json is kept
  as a normalized back-compat alias.
- Gate the arbitrary-JS evaluate step behind a dedicated
  wordpress.browser-actions.evaluate policy capability, separate from the non-JS
  interaction steps, mirroring the wordpress.run-php posture.
- Emit per-step results to files/browser/steps.jsonl (index, kind, selector,
  ok/fail, timing) and machine-readable expect/evaluate assertions into
  action-summary.json; support named screenshots; enforce per-step and
  total-script timeouts with a clean first-failure record.
- Recipe-mountable: recipe policy auto-grants the evaluate capability when a
  browser-actions step opts into an evaluate step.
- Document the contract in the README and CLI command catalog; add a step-schema
  validation unit smoke and extend the browser-actions artifact smoke to a
  multi-step click/expect/evaluate/screenshot script. Navigate-only behavior is
  unchanged.

Closes #310
@chubes4 chubes4 merged commit 2472115 into main May 30, 2026
chubes4 added a commit that referenced this pull request May 30, 2026
Reference example for the wordpress.browser-actions interaction probe
(#311). Mounts a tiny vendor-neutral @wordpress/element demo plugin (no
build step) and drives it under WordPress trunk / React 19: clicks a
counter, asserts a threshold message, moves a slider, asserts the bound
value. Proves the probe verifies a component WORKS under interaction,
not just that it renders. A green run reports 6/6 assertions passed, 0
errors.

The plugin-specific consumer (driving the data-machine-socials
react-easy-crop modal) lives in that plugin's own repo, next to the
code it guards, to keep this generic cookbook example vendor-neutral.
chubes4 added a commit that referenced this pull request May 30, 2026
Reference example for the wordpress.browser-actions interaction probe
(#311). Mounts a tiny vendor-neutral @wordpress/element demo plugin (no
build step) and drives it under WordPress trunk / React 19: clicks a
counter, asserts a threshold message, moves a slider, asserts the bound
value. Proves the probe verifies a component WORKS under interaction,
not just that it renders. A green run reports 6/6 assertions passed, 0
errors.

The plugin-specific consumer (driving the data-machine-socials
react-easy-crop modal) lives in that plugin's own repo, next to the
code it guards, to keep this generic cookbook example vendor-neutral.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: interaction-capable browser probe — make Codebox the portable WordPress E2E sandbox, not just a render-smoke probe

1 participant