Repository navigation
v1.11.1
A security release over 1.11.0, with one advisory: GHSA-wrg7-3hw9-f5wj (a page could choose which
element an element_id acted on). Update the controller (its image, or your checkout of
controller/ if you run it from source). The client, LangChain and MCP bridge packages change only
in version; updating them alone does not deploy the fix.
Upgrade notes:
- Observe again after updating. Ids from observations made before 1.11.1 (
op-and up to 8
characters) are refused asinvalid_element_id, and so are theirselector_hints passed as a
selector. browser.find_elementsandbrowser.wait_for_selectorrefuse aselector_hint
(selector_hint_not_a_selector). Pass theelement_idto the action tools instead, or a CSS
selector of your own.- Some actions that used to run are now refused, below: a click or hover by
element_idwhose
point lands on an embedded frame inside the element, and a hover byelement_idon an element
something covers (1.11.0 moved the pointer there anyway), bothelement_covered; a selection by a
<select>'s ownelement_idwhile it never stops moving (element_not_stable); and a drag to or
from a CSS end with no box (drag_end_not_visible). A click on an input that a label for it
covers, such as the hidden input of a custom checkbox, failed on 1.11.0 too, after
ACTION_TIMEOUT_MS; it is now refused at once, with the way out. - New error codes a caller may see:
invalid_element_id,stale_element_id,
unknown_element_id,element_not_visible,element_disabled,element_not_stable,
element_not_in_view,element_covered,option_not_found,option_disabled,
element_not_a_select,element_not_a_file_input,element_target_moved,change_unconfirmed,
page_not_responding(HTTP 504 over REST),drag_ends_not_in_view,drag_page_scrolled,
drag_end_not_visibleandselector_hint_not_a_selector. Of these, the refusals from
element_not_visibletoelement_not_a_file_input, andstale_element_idand
unknown_element_id, are marked retryable: observe again before retrying. The rest are not. - An upload whose file went in is reported as done. By
element_id, readtarget.deliveryand
verification.verifiedto see whether the input was confirmed to hold the file, rather than
retrying; by selector,verification.verifiedmeans only that the transfer returned.
Security
- A page can no longer choose which element an
element_idacts on. Observe wrote each id into the page
as adata-operator-idattribute, kept any such id the page had set itself, and an action found its element
again with an unescaped[data-operator-id="<id>"]selector, as did aselector_hintpassed as a selector.
A page could give a hidden "Pay now" button the id of a visible "Cancel", copy an id once observe had
written it, or be named by a crafted id carrying selectors of its own, and the action ran on an element the
agent never chose. Every release from 0.1.0 through 1.11.0 is affected. The ids now live in an isolated
JavaScript world the controller creates for each page, which page scripts can neither read nor change, and
observe no longer writes element-id attributes. Anelement_idmust be in the form observe issues (op-
and 24 lowercase hex digits) or it is refused (invalid_element_id). An id from an earlier document (after
a navigation to a new document, or a reload), or for an element the page removed or moved into another
document (a same-origin frame's), is refused asstale_element_id, and one the tab was never given as
unknown_element_id; past the last 10,000 ids a tab was issued, telling the two apart is best-effort.
Observe again and use an id from the new observation. Ids from observations made before the update no longer
work. - An action by
element_idacts on that element, not on one the page points it at. Click, hover, type and
select now find and check the registered element itself in the isolated world, and aim the mouse or the keys
at it; they add no attribute or node to the page. The element is scrolled into view and waited on until it
is visible and stable (the same box in two consecutive animation frames, as Playwright requires; on a page
that draws no frame at all for 1 s, such as a hidden tab, two checks 50 ms apart count instead, and a wait
whose frames stop partway is refused aselement_not_stableat its deadline), for click, hover and type
until it is inside the viewport, and for click, type and select until it is enabled: up to
ACTION_TIMEOUT_MS, except that type waits up to 1 s and then focuses the field directly. The pointer goes
to within 2.5 px, in each direction, of the centre of the element's first client rect inside the viewport
(Playwright's clickable point; a hover goes to the centre itself), and an element covered there is scrolled
again with another alignment, as Playwright retries; the hit test right before the mouse is pressed (for a
hover, while it waits) accepts only the element or something inside it, so a label for it, an embedded frame
in the document or its open shadow roots (inside the element too), or anything over it, is refused
(element_covered, naming what covers it) rather than clicked or hovered (type focuses a covered field
directly instead); keyboard focus is checked on the element before the text goes out, between select-all and
delete, and after the clear (focus_lost; focus in a frame inside the element, its open shadow roots
included, does not count, since keys would go to the frame's document). A selection follows Playwright's
matching rules, and, as Playwright's does, one by theelement_idof a label, or of an element in a label
that is not a control itself (Playwright's 'follow-label' rule), is made in the<select>the label is for,
the one found when the action starts, which is the one checked as well. An element that never becomes ready
is refused withelement_not_visible,element_disabled,element_not_stable,element_not_in_viewor
element_covered; a select with no matching or only a disabled option withoption_not_foundor
option_disabled. Upload still uses Playwright's file transfer. It marks the file input (the element, or,
as Playwright's transfer does, the control of a label it names by the same rule; anything else is refused as
element_not_a_file_input) with a one-time attribute for that one action, checks before the upload that the
marked input is the only element marked (element_target_movedotherwise), and removes the marker in the
document and its open shadow roots, and from the input itself. A change sent to the isolated world (a focus,
a selection, the upload's marker) is never sent twice: when its answer is lost after it was sent, the action
fails aschange_unconfirmed(not retryable), since the change may have been made. - What an
element_idguarantees, and what it does not. The page cannot choose, forge or copy which
element an id names. It can still redirect an action in the gaps between the checks and the input, as it
always could with its own scripts: the few milliseconds between the last hit test and the press, and the
20–80 ms between mouse down and up; focus is not checked between keystrokes, and a keydown handler can move
focus within the key it handles, the clear's Delete included: the check after the clear then reports
focus_lostafter the field it moved to was emptied; hover is not hit-tested once the pointer arrives; a
drag's destination is not hit-tested (drag libraries put the dragged element under the pointer), and its
check of a coordinate end compares scroll positions, so a page can still change what is under that point (by
scrolling a box no element end is drawn inside while stopping its scroll events or inside a shadow root, by
scrolling back to where it was, or by replacing content), and a scroller an element end is slotted into
through a closed shadow root is not seen; the hit test and the focus check do not see into closed shadow
roots, so an embedded frame behind one inside the element takes the click, as it does with Playwright and a
human click, and focus in such a frame reads as focus on the element, so typed text goes to the frame; and
between the check before Playwright resolves the marked file input and the check after it, a page script can
move the marker to another file input of its own document, so that the upload goes there, and then put it
back. The upload's check afterwards is best-effort (below). - No MCP tool or browser action matches a
selector_hintagainst the page any more.browser.drag_drop
takes the registered element for a hint (refusing unknown, stale and malformed ids);browser.find_elements
andbrowser.wait_for_selectorrefuse a hint (selector_hint_not_a_selector) instead of matching an
attribute only the page could have written. A hint passed as aselectorto the action tools still names
its element. - A crashed or hung tab no longer hangs observe's element scan or an action's element checks. Every
message to the isolated world has a deadline:ACTION_TIMEOUT_MS, plus its own wait for a script that waits
on the element. It bounds each message, not the whole action. A page that does not answer in time fails with
page_not_responding(not retryable): HTTP 504 over REST, and an MCP tool error with the same code and
retryable: false. Its CDP session is detached and replaced on the next call, and one whose attach
completes after its deadline, or after its call was cancelled, is detached once it does. Other calls still
wait on the page as before (its title, mouse and keyboard input, and the checks after an action, such as the
bot-challenge probe), so a page that hangs during an action, in its click handler for example, holds the
action until it answers.
Changed
- The hit test before a click now runs after the 30–120 ms pause before the press, for selector targets as
well, so a cover put up during the pause is caught; the press follows the check with no deliberate delay. - A click or selection by
element_idon an elementaria-disableddisables is refused as
element_disabled, by a port of Playwright's rules: on an element whose role takesaria-disabled(a
button, link, checkbox, textbox, option and the like), set on it or on the nearest ancestor that sets it
either way; and, as in Playwright, the check is made on the button a part of it belongs to, or on the
control of the label it is in. A genericdivwitharia-disabled="true"is still clicked, as in
Playwright. 1.11.0 refused the same cases through Playwright, as a retryablebrowser_action_failedafter
the same wait. - A custom checkbox whose visible part is a
<label>over a hidden input is refused at once when clicked by
the input'selement_id(element_covered), where 1.11.0 failed afterACTION_TIMEOUT_MSas a retryable
browser_action_failed(Playwright found that the label intercepts pointer events). The refusal names the
label (or the part of it at the pointer) and the way out: click the label by coordinates, or with the
selectorlabel[for="…"]it gives when the label has aforand the whole selector fits in the advice. An
element whose centre is an embedded frame inside it, which 1.11.0 clicked into, is refused with the advice
to click a part of it outside the frame by coordinates. Such covers, which waiting cannot remove, are
refused after one round of re-scroll alignments, not at the end ofACTION_TIMEOUT_MS(15 s at the
default). A refusal after re-scrolling says the page was scrolled; the page is left where the last re-scroll
put it. The cover a refusal names is bounded (its tag and its id, or its first class, each cut to 40
characters; the description and the advice at most 200 characters each), whatever the page calls its
elements. - Select by
element_idwaits for the named element's box to settle before matching options, as a click
does: a<select>named by its ownelement_idthat never stops moving, which 1.11.0 selected, is now
refused (element_not_stable). For a label, its own box is the one that has to settle. browser.drag_dropwith an end named by itsselector_hint(an element end) reads every other end once
the element ends are in place, since bringing one into view can scroll the page or a scrolling element in
it: an element end again, without scrolling, and a CSS end for the first time, each refused when its point
is outside the viewport (drag_ends_not_in_view); a coordinate end is refused (drag_page_scrolled) when,
afterwards, the window or a box an element end is drawn inside (a web component's shadow scroller included)
is at another scroll position, an element end is drawn inside other boxes than before, or a scrolling
element outside shadow roots sent a scroll event meanwhile, since the point may no longer be where it was.
Both refusals give the way out: a viewport tall enough for both ends (browser.set_viewport), or scrolling
so both are in view. A source named by itsselector_hintis hit-tested after the move and the pause, right
before the button goes down.browser.drag_droprefuses a CSS end whose element has no box (it is not rendered, as underdisplay: none;drag_end_not_visible) before any mouse input, with or without an element end. It used to drag from
or to the top-left corner of the viewport, (0, 0).- An upload is no longer a failure to retry once the file has gone in, by
element_idor by selector. In
1.11.0 a page that closed while the upload settled, or any later step that failed with a browser error, made
it a retryablebrowser_action_failedalthough the file was in, and a retry uploaded it twice. Now the
upload is reported as done, with what failed intarget.after_change. An error that is not retryable,
raised after the transfer (a policy block, a bot challenge), or a cancelled request, still surfaces, and the
approval the upload itself required (REQUIRE_APPROVAL_FOR_UPLOADS) is spent all the same. Byelement_id
the action also recordstarget.delivery:{"outcome": "confirmed"}when the file input holds one file of
that name and size, now or when itsinputorchangeevent fired, else{"outcome": "not_confirmed", "reason": …};verification.verifiedfollows it (signalsupload_confirmed/upload_not_confirmed). The
check is best-effort: a page can give the input a file of its own with the same name and size. By selector
there is notarget.delivery, andverification.verifiedmeans only that Playwright's transfer returned.