Skip to content

Releases: dmoka/quicke2e

QuickE2E 0.5.0

Choose a tag to compare

@dmoka dmoka released this 07 Oct 16:39

Faster runs

  • Early decisions. With a hosted engine, QuickE2E sends the next decision before the 150 ms settle wait. It uses the answer only when the request built from the settled page is byte-identical, so the decisions do not change. Discarded early calls are counted in cost. QUICKE2E_SPECULATE=0 turns this off; it is off for --engine local.
  • One start-page load, engine preconnect, Node compile cache.

Launch task (buy 2 tickets with a discount code from the TicketBay home page, M2 Max, 5 runs per arm):

pass median cost / run
QuickE2E, local engine, Shisa DE-1 5/5 2.94 s $0
QuickE2E, hosted Jev 5/5 3.74 s $0.00038
Claude Code + Playwright MCP, Sonnet 5 5/5 25.29 s $0.0896

Launch testing: 13 testers, 2 rounds

Before release, 13 parallel testers used the packed build as new users on Next.js, Vite, Nuxt, SvelteKit, Angular, MUI, Ant Design, Radix, Mantine, Headless UI, Chakra, data grids, WordPress, Django admin, Gitea, OWASP Juice Shop, OrangeHRM, ParaBank, Windows and Linux, and as an attacker. Fixed in this release:

Verdicts. 10 false-pass patterns: text a page loads by itself, invisible text (scale(0), transparent, clipped, 0x0 panes), a pass with no executed action (now WEAK_ASSERTION), invisible toasts and <script> text in expectSeen, a load attack that kept clicking (now NOT_REFUSED), a whitespace-only attack value that was never typed. Text assertions ignore case. A same-origin HTTP 5xx from fetch/XHR fails the run (allowServerErrors: true opts out).

Security. No secrets or basic-auth credentials in stdout, --json, traces or emitted specs; redact text stays out of emitted locators; neverClick covers every action type, the element actually clicked, map routes and invisible characters; fallback locators act only on a unique match; a watchdog ends a run on a hung page (TIMEOUT); terminal control characters are stripped from output.

Apps. Ant Design and Mantine Select, ARIA sliders, Radix tabs, multi-selects, table pagers, data-grid sorting and row selection, calendar days, cookie banners and modals, Enter to submit, hidden and multi-file uploads, unlinked and table-header labels, icon-only buttons, hidden checkboxes, SvelteKit pages, forms in modals, long forms.

New. expectGone (assert a delete), outcomes ENGINE_ERROR and NOT_REFUSED, exit code 3 when every failure is an engine failure, failure reasons (page says: …, unused spec keys, fields with no spec value, a select value that matches no option, failed requests), spec validation with exit code 2, QUICKE2E_<KEY> shared secrets for emitted specs, --only a,b.

Platforms. Tested on Windows, Ubuntu and macOS with Node 20 and 22. --json is no longer cut at 64 KB on a pipe.

Docs. AGENTS.md and llms.txt ship in the package; Windows/PowerShell lines; a CI example; a Limits section with every pattern that does not work yet.

Gate: fixtures 74/74 (Jev), 71/74 (Shisa DE-1, unchanged from 0.4.0); tests 49/49.

npm i -D quicke2e@0.5.0

QuickE2E 0.4.0

Choose a tag to compare

@dmoka dmoka released this 05 Oct 09:20
npm i -D quicke2e@0.4.0 @playwright/test && npx playwright install chromium

Emitted Playwright specs replay

  • Locators verified on the live page before each action: Playwright's own role and name; scoped to the row or card when several elements share a name (getByRole("listitem").filter({ hasText: "Sauce Labs Bike Light" })); then label, placeholder, test id, position. A candidate wins only when it matches exactly one element, the one the run acted on.
  • Assertions read the page with the same text function as the run; select assertions read the chosen option.
  • storageState (a path), native dialogs, Escape/Tab presses, hover and drag replay. A missing secret skips only its own test.
  • saucedemo.com: 9 of 9 emitted specs replay (0.3.0: 3 of 9).

More apps

  • Unlabelled controls take the caption before them; identical buttons get their card as context ("Add to cart · Sauce Labs Backpack"); a spec key that exactly names a field is that field's key only.
  • Autocomplete inputs can be typed into; an open suggestion list is treated as a popup.
  • Native dialogs accepted by default (dialog: "dismiss"); a prompt takes the spec value its message names.
  • Escape and retry when a popup blocks an action; Tab after typing into a field that opened a date picker.
  • HOVER (hidden captions, tooltips) and DRAG (draggables onto drop zones); range inputs are sliders.
  • Timing: WAIT does not use a step; navTimeout/--nav-timeout; maxTimeMs → TIMEOUT; slow actions flagged.
  • AUTH_REQUIRED when a storageState session is missing or expired; run --reset "<cmd>" before every run.
  • demoqa practice form (date, autocomplete, state/city) passes in 8.5 s (0.3.0: LOOP after 82 s). demoqa select menu, slider, droppable, confirm and the-internet confirm, prompt, hover, dynamic loading pass.

Also in this release: PR #2 (--map keeps a concrete start page).

Fixtures 74/74 on hosted Jev; tests 44/44.

QuickE2E 0.3.0

Choose a tag to compare

@dmoka dmoka released this 04 Oct 17:07
npm i -D quicke2e@0.3.0 @playwright/test && npx playwright install chromium

Safety

  • Unknown options and commands are errors. A typo such as --bse used to fall back to localhost:3000.
  • neverClick: elements the engine is never offered ("Pay*", /delete/i). Every refusal and attack case should block its success action.
  • Minimum confidence (default 0.3, minConfidence / --min-confidence): a pick below it is not executed. In the measured suites no passing action was below 0.34.

Attack cases (PR #1)

  • expectAbsent: text only success shows. If it appears, the run fails with ABSENT_SEEN.
  • control: a load attack (another user's order by URL) checks its assertion against a page where the app says yes.
  • SERVER_ERROR: a 5xx answer to a navigation fails the run. kind: "attack" is shown in the output.

Every failure says why

  • Under each FAIL line: assertions not met, a submit held back for an empty field (field + spec key), LOOP (the same action 3x on an unchanged page), required fields still empty, failed actions. The same in --json (assertions, heldBack, loop, emptyRequired).
  • --json also lists flows that never ran (UNREACHABLE, WEAK_ASSERTION, ABSENT_ON_START). Traces list the options offered per step.

First run

  • One-line fixes for a missing spec file, Chromium, API key, or local engine. Real --help, per-command help, --version.

Fixed

  • expectSeen missed text that a single-page app inserts as one large block with split text nodes (React). Reproduced on saucedemo.com.

Fixtures 67/67 on hosted Jev; tests 38/38.

QuickE2E 0.2.0

Choose a tag to compare

@dmoka dmoka released this 04 Oct 13:13

QuickE2E 0.2.0 — the first release under the new name (formerly jevtester).

npm install -D quicke2e && npx playwright install chromium

Changes

  • Renamed to QuickE2E: npm package and CLI quicke2e, skill skill/quicke2e/, default files quicke2e.spec.mjs / quicke2e.map.json. jevtester on npm is deprecated and points here.
  • run --video <dir> records the browser per run; with --trace, each step gets its start time, decision time and the box of the element it acted on.
  • Clearer CLI errors: an unreachable start page prints UNREACHABLE … Is the app running? and exits 1; an unknown --engine fails with the list of valid engines.
  • Docs for AI agents: AGENTS.md (spec rules, field formats, exit codes, the --json run record) and llms.txt.
  • README rebuilt: launch demo, proof table, reproduce commands, FAQ.

Launch task (from the TicketBay home page, buy 2 tickets with a discount code; median of 5 runs, all passed)

median wall time cost / run
QuickE2E, local Shisa DE-1 3.23 s $0
QuickE2E, hosted Jev 4.79 s $0.00030
Claude Code + Playwright MCP (Sonnet 5) 20.89 s $0.0888

Raw runs: bench/results/launch-task.json.