Skip to content

playwright-eir 0.3.0

Choose a tag to compare

View all tags
@Someone-anon-coder Someone-anon-coder released this 16 Jul 14:07
· 37 commits to main since this release
v0.3.0
38baa60
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

playwright-eir 0.3.0

Self-healing locator engine for Playwright. This release adds CI integration, an opt-in (and measured-not-helpful, disabled-by-default) Gemini fallback, and a full hardening/documentation pass — see the README for the complete picture.

Install

npm i -D playwright-eir
// before
import { test, expect } from "@playwright/test";
// after
import { test, expect } from "playwright-eir";

Results — measured, seed 42, pnpm bench:all

Mutation class Heal rate False-heal rate Suggestion rate Miss rate
id-rename 75.0% 0.0% 25.0% 0.0%
text-change 87.5% 0.0% 12.5% 0.0%
tag-swap 100.0% 0.0% 0.0% 0.0%
class-shuffle 25.0% 0.0% 75.0% 0.0%
sibling-reorder 0.0% 0.0% 0.0% 100.0%
wrapper-inject 100.0% 0.0% 0.0% 0.0%
near-duplicate-sibling-swap 25.0% 0.0% 75.0% 0.0%
compound-release 50.0% 0.0% 25.0% 25.0%

False-heal rate: 0.0% in every class, every measured run. class-shuffle and sibling-reorder are documented structural ceilings (fingerprint schema never captures an element's own class tokens; a position-anchored selector after a reorder doesn't throw, so triage never runs) — not tuning gaps. Full failure-mode analysis in the README; full tuning history in docs/tuning-log.md.

What's new since 0.2.0

Added

  • packages/ci-action — auto-updating PR comments with heal/suggestion diffs, confidence scores, linked screenshots.
  • Opt-in Gemini LLM fallback, structurally suggestion-capped (can never promote a row to healed). Measured across 74 real invocations: no accuracy benefit on any of the 8 mutation classes, and a real free-tier reliability ceiling (77% degraded to no-verdict on rate-limiting). Shipped disabled by default on that evidence — see docs/hybrid-comparison.md.
  • RetryOutcome/ReportRow now distinguish a heal genuinely verified against a recorded post-condition from one accepted because no baseline existed to check against.
  • Full README, a real-timed demo path (demo/README.md, ~20s clone-to-suggestion), and complete docs/ coverage.

Fixed

  • classifyFailureSpecies now recognizes a Playwright test-level timeout message, not only a bounded action-timeout one — closes a gap where a suite without actionTimeout configured would silently never engage Eir's triage on a real broken selector.
  • Benchmark evidence CLIs refuse to overwrite an existing report without --force.

Proven live, not just unit-tested

  • Mechanism A (post-condition verification) demonstrated catching a real, live false heal produced end-to-end by the actual (unmocked) matching engine — note005RealFalseHeal.test.ts.

Documented (known limitations, not fixed this release)

  • EirLocator passed where Playwright expects a real Locator (.and(), .or(), .dragTo(), .locator(sel, {has})) is a real, confirmed bug — characterized with a committed test, fix tracked for a future release.
  • An element's own class tokens are never captured in its fingerprint (the class-shuffle ceiling) — a fingerprint schema-v2 candidate.
  • Full list in the README's "Known limitations" section.

Full changelog: packages/eir/CHANGELOG.md

Assets 2
Loading