Skip to content

Releases: lodado/playwright-spec-for-AI-Agent

playwright-spec-for-ai-agent@2.7.0

Choose a tag to compare

Minor Changes

  • 07b5505: Adaptive (AI-native) execution can now perform @qa-fixture file uploads. An UPLOAD interaction
    becomes an upload_observed_element adaptive action, offered only when the scenario declares an
    upload milestone with a designated @qa-fixture. The agent chooses the target element; the file is
    always the author-designated fixture, resolved strictly inside the project root (no symlink escape,
    size cap) before setInputFiles. Uploads whose fixture is undeclared stay blocked. The QA IR
    milestone gains an optional fixture field and the vocabulary gains upload_observed_element (both
    additive).
  • 07b5505: qa-native execute now defaults to AI-native execution (--provider=hermes --mode=adaptive).
    Passing --mode=strict (or --provider=playwright) still selects the deterministic read-only
    provider; either flag alone infers the matching pair. Runs that previously relied on the implicit
    strict default now run adaptively unless they pass --mode=strict.

Patch Changes

  • 6da2c92: Under --allow-partial, adaptive execution now skips scenarios that compile cleanly but use an
    expectation or step kind the adaptive runtime cannot build (instead of failing the whole run), emits
    a SCENARIO_UNRUNNABLE diagnostic, and narrows the written QA IR to the scenarios that actually ran.
    Also hardens the Playwright AST parser against a variable declaration with no initializer.

playwright-spec-for-ai-agent@2.6.0

Choose a tag to compare

Minor Changes

  • 8a94e84: qa-native execute now takes a spec source that is either an explicit --spec=<file> or a
    --page=<name>. Page mode reads the project config (hermes-qa.config.mjs /
    playwright-spec-for-ai-agent.config.mjs) and runs only the specs it designates for the page: from
    the page's __tests__ directory, the specs whose @qa-scenario matches the page's
    expectedSubscriptionStatus (case-insensitive; page config, then the staging default), plus any
    // @qa-always-run: true, minus any // @qa-live-skip: true. When no status is configured the whole
    directory is designated. Navigation uses the config's per-page targetPath, and --base-url
    defaults to batch.defaultBaseUrl. An explicit --spec always wins; giving both or neither is an
    error. This keeps a run tied to the specs designated for a page's state instead of an ad-hoc plan.
  • 19b7203: Strict execution now performs @qa-fixture file uploads. An UPLOAD interaction (a
    setInputFiles("name") call) replays the file declared by // @qa-fixture: name=path into the
    target input via Playwright's setInputFiles. The fixture path is repo-relative and resolved
    strictly inside the project root (no symlink escape, 32 MB cap) before the browser touches it; an
    upload whose argument names no declared fixture is blocked (UPLOAD_FIXTURE_UNRESOLVED) rather than
    run. Upload stays a strict-mode-only exception — the adaptive/AI provider never uploads. The QA IR
    scenario gains an optional fixtures map and execution-plan interaction nodes an optional value
    (both additive; no schemaVersion bump).

Patch Changes

  • cd2bb5d: Collapse duplicated adaptive-protocol rules into single sources. The action vocabulary is now
    defined once as ACTION_SPECS in contracts — lease building, safe-recovery, milestone semantics,
    the gateway guard, parameter-key validation, element-bound actions, the audit artifact shape, and
    the Hermes execution prompt version all derive from it instead of keeping their own copies. Audit
    artifact shape ("five snapshots plus report_blocked's VISIBLE_TEXT") is defined once as
    auditArtifactShape and consumed by both the provider seal and the evidence validator. The
    observation-settle wait is defined once as observationSettleBudget in core and shared by strict
    observation and adaptive snapshot capture. Behaviour-invariant: the sealed lease order and
    completion semantics are unchanged (equivalence tests lock them); the Hermes execution prompt
    version string changes because it now hashes ACTION_SPECS.
  • babeb28: Adaptive gateway now settles the page after every navigation (document fully loaded plus a 300ms DOM-mutation quiet window, capped at 5s and bounded by the remaining run budget) before sealing snapshot evidence, so testids attached only after client hydration reach the judge instead of the pre-hydration SSR markup.
  • cd2bb5d: CLI internal errors no longer collapse to an opaque "command failed". stderr now names the failure
    category — a CliError's message, or an internal error's stable .code or class name — so a failure
    is never silently swallowed. The raw message and stack, which may embed sensitive data such as
    evidence bytes, stay behind QA_NATIVE_DEBUG.

playwright-spec-for-ai-agent@2.5.1

Choose a tag to compare

@github-actions github-actions released this 30 Jul 08:30
86511d0

Patch Changes

  • 6f0403e: Extract the Hermes CLI transport into a shared private package @persona-runtime/hermes-transport so personaut no longer deep-imports a file inside the sibling app (../../playwright-spec-for-ai-agent/scripts/hermes-runner.mjs) with no declared dependency edge. personaut now imports the package and bundles it. qa-native ships raw .mjs, so its scripts/hermes-runner.mjs re-exports the package in the monorepo and a prepack step inlines the package back into that path for publish, keeping the published artifact self-contained. No runtime behavior changes for either package.

playwright-spec-for-ai-agent@2.3.0

Choose a tag to compare

Minor Changes

  • Move AI reasoning into the execution path so non-deterministic live behavior is judged, not literal-matched. Adaptive scenario failures and budget exhaustion now record ERROR/BLOCKED outcomes and keep their sealed evidence instead of throwing and deleting it; @qa-live-policy: mock-judgment scenarios route to observe-only milestones and carry a SEMANTIC hint so the judge rules on structural equivalence; a report_blocked terminal action lets the agent surrender an unreachable milestone with sealed, judge-verified evidence; element observation waits for visibility instead of snapshotting; mock-judgment contradictions classify as TEST_DATA; and execute gains --budget-* flags while judge gains a verdict summary and --fail-on exit codes.

@lodado/personaut@0.5.0

Choose a tag to compare

@github-actions github-actions released this 30 Jul 08:30
86511d0

Minor Changes

  • 33618b0: Realign the deferred fail-loud design decisions per the maintainer's plan: budget exhaustion (action/time/no-progress) now terminates as a clean ABANDONED with the budget code preserved instead of runtime_error, so it counts toward abandonment and is no longer excluded from variant comparison as infrastructure; one session's evidence-seal failure no longer aborts the whole run (the in-memory manifest is returned and that session degrades to runtime_error while siblings still complete and report); variant comparison reports insufficient_evidence below 5 comparable pairs per arm rather than declaring a winner off a single flipped session; and the hyperactivity heuristic only applies when the action budget is large enough for its ">=80%" ratio to be meaningful.

Patch Changes

  • 6f0403e: Extract the Hermes CLI transport into a shared private package @persona-runtime/hermes-transport so personaut no longer deep-imports a file inside the sibling app (../../playwright-spec-for-ai-agent/scripts/hermes-runner.mjs) with no declared dependency edge. personaut now imports the package and bundles it. qa-native ships raw .mjs, so its scripts/hermes-runner.mjs re-exports the package in the monorepo and a prepack step inlines the package back into that path for publish, keeping the published artifact self-contained. No runtime behavior changes for either package.

playwright-spec-for-ai-agent@2.2.0

Choose a tag to compare

Minor Changes

  • Make execute --allow-partial work on real specs and surface run outcomes.

    • Attribute AST-level compile diagnostics (opaque/dynamic assertion and action targets) to the owning test by source range, so a single unparseable scenario blocks only itself instead of failing the whole file closed.
    • Record live-policy blocked scenarios (skip, auth-mock, subscription-mutation) as blocked so --allow-partial prunes them, instead of exploding createExecutionPlan on a NAVIGATE step under a blocked policy.
    • Surface internal failure detail (message + stack) under QA_NATIVE_DEBUG; the default stays a secret-safe opaque message.
    • Emit a one-line success summary (executed vs. skipped scenarios and the artifact path) instead of running silently.

playwright-spec-for-ai-agent@2.1.0

Choose a tag to compare

@github-actions github-actions released this 29 Jul 08:46
f64f2ba

Minor Changes

  • 6917cc4: Compile Playwright specs per scenario instead of failing the whole file closed. Opaque/dynamic steps now block only the scenario that owns them; the adapter records blocked scenario ids in qaIr.extensions.blockedScenarioIds, and qa-native execute --allow-partial runs the statically compilable scenarios while printing skipped-scenario diagnostics to stderr.

Patch Changes

  • 677463d: Compile role-only Playwright locators (e.g. getByRole("dialog")) instead of throwing: role alone is a valid semantic identity, so accessibleName is now attached only when the locator carries an explicit name.

playwright-spec-for-ai-agent@2.0.0

Choose a tag to compare

@github-actions github-actions released this 29 Jul 08:22
bb1d12c

Major Changes

  • Remove the legacy page-QA CLI. The playwright-spec-for-ai-agent <spec|abstract-ai|judge|review|slack|nightly> bin, its scripts, and the ./cli export are deleted; the evidence-driven qa-native CLI is now the only entry point. Config files keep only the remediation section (loadProjectConfig); page/staging/paths config no longer drives a command.

playwright-spec-for-ai-agent@1.2.0

Choose a tag to compare

@github-actions github-actions released this 29 Jul 07:50
0a03a66

Minor Changes

  • Enable one-shot QA Native: multi-scenario adaptive execution into one sealed manifest, same-origin read-after-click network policy, run envelope v0.2, Hermes runner protocol probe, and auto-discovered shared login at .private/storage-state.json.

playwright-spec-for-ai-agent@1.1.2

Choose a tag to compare

Patch Changes

  • Complete authenticated adaptive one-shot QA with safer bounded action handling.