fix(browser): __piloRefMap fallback when data-pilo-ref is stripped#450
fix(browser): __piloRefMap fallback when data-pilo-ref is stripped#450lmorchard wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves element-ref validation for React-heavy pages by recovering from cases where data-pilo-ref gets stripped between snapshot generation and an action, using the page-side globalThis.__piloRefMap as a fallback while keeping existing failure behavior for true invalid refs.
Changes:
- Core (Playwright):
validateElementReffalls back to__piloRefMaponly when the selector finds 0 matches, reattachesdata-pilo-ref, then proceeds via the selector path again. - Core tests: adds coverage for the recovery path and the fallback miss cases; updates existing mocks to include
evaluatewhere needed. - Extension: adds equivalent recovery logic inside the
scripting.executeScriptpage-context function.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| packages/core/src/browser/playwrightBrowser.ts | Adds __piloRefMap fallback + attribute reattachment when [data-pilo-ref=...] lookup returns 0. |
| packages/core/test/playwrightBrowser.test.ts | Adds unit tests for recovery/miss paths and updates mocks for the new evaluate call. |
| packages/extension/src/background/ExtensionBrowser.ts | Adds __piloRefMap fallback in the injected action executor when data-pilo-ref is missing. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| }, ref); | ||
|
|
||
| if (!reattached) throw new InvalidRefException(ref); | ||
| return this.page.locator(`[data-pilo-ref="${ref}"]`); |
…is stripped When React-style DOM reconciliation strips the data-pilo-ref attribute between snapshot generation and click execution, the attribute-selector lookup in validateElementRef returned 0 matches and threw InvalidRefException — even though the Element reference held in globalThis.__piloRefMap was still valid. The agent then logged the failure as 'Invalid element reference', mixing it together with genuine LLM hallucinations in failure traces. Adds a fallback path in both validateElementRef (playwrightBrowser.ts) and the injected lookup function (ExtensionBrowser.ts): when the attribute selector returns 0, consult globalThis.__piloRefMap, verify the mapped Element is connected to the DOM and belongs to the current frame's document, re-attach the attribute, and proceed. Post-recovery the Playwright path re-runs count() on the new locator and throws InvalidRefException for 0 or >1 — preserving the same validation semantics as the non-fallback path against a reconciliation race that strips the attribute again, or against an unexpected duplicate attribute landing. Genuine hallucinations (ref absent from the map) and genuine DOM removals (Element.isConnected === false) still fail loudly via the existing InvalidRefException path. The ownerDocument check excludes same-origin iframe elements that the snapshot may have walked inline — those can't be acted on from the main-document context anyway, and recovering them would push the failure to click time as an opaque 30s locator timeout. With the guard, iframe refs continue to fail with the same immediate InvalidRefException as before. Closes #449
b9c128b to
15571c6
Compare
|
Addressed Copilot's review comment on the post-recovery validation. After re-attaching the attribute, |
Summary
data-pilo-refattribute between snapshot and click, the attribute-selector lookup invalidateElementRefreturned 0 matches and threwInvalidRefExceptioneven though theElementreference held inglobalThis.__piloRefMapwas still valid.stale_element_refs-classified failures in eval logs a mix of distinct root causes — and unnecessarily aborting agent tasks on React-heavy sites (Booking, Google Flights, Apple).Design Decisions
__piloRefMaponly on the 0-match path. Preserves the happy-path cost (one selector lookup) when the attribute is intact (common case) and the loud-failure contract on the>1match path.ElementHandle. KeepsvalidateElementRef's return type stable asLocatorand helps any subsequent lookup in the same iteration find the element via the existing selector path.Element.isConnectedandownerDocument === documentbefore recovering. Detached elements still fail loudly; same-origin iframe elements (which the snapshot walks inline but the main-frame locator can't reach) continue to fail with immediateInvalidRefExceptionrather than recovering and then timing out at click time with an opaque error.InvalidRefExceptionshape. Recovery is silent; callers see either a working locator or the same exception as today.Changes
packages/core/src/browser/playwrightBrowser.ts—validateElementReffalls back topage.evaluateconsultingglobalThis.__piloRefMapwhen the attribute selector returns 0. Re-attaches and re-queries via the locator on success.packages/core/test/playwrightBrowser.test.ts— three new tests (recovery success path, no-map-entry miss, detached-element miss). Two pre-existing tests updated to includeevaluatein their mock page, since the new code path consults it on the 0-match branch.packages/extension/src/background/ExtensionBrowser.ts— same recovery logic in the injectedscripting.executeScriptfunction. Simpler than the Playwright case because the injected code already runs in the page context (directMap.get+setAttribute, noevaluateround-trip).Test Plan
pnpm --filter pilo-core run testpasses — 672/672, including 3 new testspnpm --filter pilo-extension run testpasses — 266/266pnpm run checkpasses cleanly across all four packagesgitleaks detectcleanstale_element_refsclassifications from attribute-strip cases. Not a merge blocker; will run via the standard eval pipeline.Why no automated test for the extension change
The injected function runs in the Chrome page context via
scripting.executeScriptand isn't exercised by the extension package's Vitest unit suite. Refactoring to extract a separately-testable helper was explicitly out of scope per the spec. Thepnpm --filter pilo-extension run test:e2eharness covers the live execution path.References
Elementreference held in__piloRefMapwas still live.🤖 Generated with Claude Code