Declarative, in-browser UI automation using serialisable JSON steps.
DOMStepRunner lets you walk through a UI to reliably achieve a goal — by describing the journey as simple, declarative steps.
It runs entirely in the browser. Perfect for Chrome extensions, dynamic control flows, scraping flows, and LLM-driven UI control.
DOMStepRunner executes a sequence of JSON-defined DOM steps:
- Wait for element
- Click element
- Optionally skip based on state
- Fail cleanly if something never appears
You define what should happen. The runner handles the timing, polling, resilience, and state transitions.
Think of it as:
A lightweight, declarative UI walkthrough engine that runs directly in the browser.
No headless browser. No Puppeteer. No background server. Works directly in:
- Chrome extensions
- Content scripts
- Injected scripts
- In-page automation
Steps are pure serialisable objects.
This means:
- You can fetch updated steps from a server if the DOM changes
- You can let an LLM generate or repair steps dynamically
- You can store flows in a database
- You can diff and version them
Useful for brittle UIs like Gmail that change frequently.
It:
- Waits for elements instead of sleeping blindly
- Skips intelligently if the UI is already in the desired state
- Handles transient UI states
- Fails with precise diagnostics
Example: YouTube transcript extraction.
To scrape a transcript you must:
- Click “More”
- Click “Show transcript”
- Wait for transcript panel
- Then extract text
DOMStepRunner handles the UI manipulation phase cleanly before your scraping logic runs.
Example: Gmail automation.
You might want to:
- Click compose
- Wait for modal
- Click formatting options
- Wait for dropdown
- Insert content
Instead of imperative brittle code, define the flow declaratively.
Especially useful in Chrome Extensions where you can't update the code quickly if the DOM changes, but you can download fresh JSON to get the latest steps.
The steps are a codified language for working with a page, so an LLM could define the steps and fix them when the DOM changes.
An AI can:
- Read the page
- Set a goal
- Identify necessary UI steps
- Output a
DOMStep[] - Run it
- Detect failures
- Repair and retry
npm install @andyrmitchell/dom-step-runnerEach step describes:
- What element to target
- What action to perform
- Optional timeout
- Optional delay
- Optional skip condition
Example:
const steps = [
{
name: "Open transcript menu",
target: "button[aria-label='More actions']",
action: "click"
},
{
name: "Click show transcript",
target: "ytd-menu-service-item-renderer",
action: "click"
},
{
name: "Wait for transcript panel",
target: "#transcript",
action: "wait_for"
}
];import { DOMStepRunner } from "@andyrmitchell/dom-step-runner";
const runner = new DOMStepRunner(true, 5000);
const result = await runner.run(steps);
if (!result.success) {
console.error("Failed at:", result.failedStepTarget);
}You get structured failure data:
{
success: false,
message: "Timeout waiting for '#transcript'",
failedStepIndex: 2,
failedStepTarget: "#transcript"
}Use the feedback for:
- Retry logic
- LLM repair loops
- Telemetry
- Debugging UI breakages
This section is intended for developers and LLM agents generating flows.
export interface DOMStep {
name: string;
/** CSS selector of the DOM element to interact with */
target: string;
/** Action to perform */
action: "click" | "wait_for";
/**
* Skip condition.
* 'next_available' skips this step if the NEXT step's target
* already exists in the DOM.
*/
skipOn?: { condition: "next_available" };
/** Override default timeout for this step (ms) */
timeoutMs?: number;
/** Delay before action execution (ms) */
delayActionMs?: number;
}export interface DOMStepRunResult {
success: boolean;
message: string;
failedStepIndex?: number;
failedStepTarget?: string;
}constructor(verbose: boolean = true, defaultTimeoutMs: number = 5000)verbose— Logs detailed step progression to consoledefaultTimeoutMs— Default wait time for element polling
Executes steps sequentially.
Behavior:
- Checks skip conditions
- Waits for target element
- Applies optional delay
- Re-checks skip
- Executes action
- Fails gracefully on timeout
Returns structured result object.
Dismiss cookie banner only if present.
const steps = [
{
name: "Dismiss cookie banner",
target: "#accept-cookies",
action: "click",
skipOn: { condition: "next_available" } // If `#app-root` already exists, the click is skipped.
},
{
name: "Wait for main app",
target: "#app-root",
action: "wait_for"
}
];const steps = [
{
name: "Click Compose",
target: "div[gh='cm']",
action: "click"
},
{
name: "Wait for compose modal",
target: "div[role='dialog']",
action: "wait_for"
}
];An LLM might output:
[
{
"name": "Open menu",
"target": ".menu-button",
"action": "click"
},
{
"name": "Wait for dropdown",
"target": ".dropdown",
"action": "wait_for"
}
]Your system simply:
await runner.run(llmGeneratedSteps);If it fails:
- Feed failure back to the LLM
- Regenerate corrected steps
- Retry
Use DOMStepRunner when:
- You need reliable in-browser automation
- You’re building Chrome extensions around changing DOMs (code is slow to update, JSON can be instant)
- The DOM changes frequently
- You want server-driven automation definitions
- You’re building AI agents that manipulate UIs
- You want deterministic, debuggable flows
Do not use it if:
- You need full browser automation across pages (use Puppeteer)
- You need cross-origin control
MIT