Repository navigation
browser actions
Every call an agent makes to a page takes one road, core.recipe.Recipe.run, and ends with the same
report of where the page is. This area holds what the actions share: selectors, waits, the optional
url, clicks and what received them, the pointer, keys, frames, dialogs, single-page apps, scripts,
uploads and the wording of a failure. Each tool's arguments and result fields are generated from its
docstring into the wiki's docs/<tool>.md page and are not restated here.
- An agent clicks a link that exists twice, once hidden: the visible one is clicked and the result
says which. A script waits with
assert, and a flow uses the same action as a step.
ACT-1. The browser a call reaches, its turn and the reopen are in
workspaces-and-sessions.md; the capability table, failure statuses and
the screenshot's MCP shape in surfaces.md; refusals under a person's control in
live-view-and-control.md; typing a secret and the scrub in
secrets-and-scope.md. open_session, save_site_data, print's file and
network_body are in session-settings.md, site-data.md,
workspace-files.md and capture.md.
ACT-2. navigate, interact, drag, write, press_key, extract, execute_script,
assert, outline, screenshot, frame, resize, dialog, upload_file, print and
save_site_data run Recipe.run. In order it checks the selector and the url, takes the session's
turn (WS-39), reattaches, goes to url when one was named and the browser is elsewhere, waits for
the element, runs the action's body, and reads where the page is.
ACT-3. What a caller can get wrong is refused before the browser is touched, so a refusal costs no page load:
- a bad selector or
urlscheme; - an unknown mouse, frame or dialog action, print format or key;
-
send_textwithouttext; - a
dragwith no destination, with both a destination and an offset, or with an offset of zero; - an upload with no source or several;
- an
assertwhosestable_foris not shorter thanwait_timeout.
ACT-4. The result is the body's fields, then matched and index (ACT-11), then the page state
(ACT-12), later keys replacing earlier. The page is read once.
ACT-5. An element that goes stale between the wait and the act makes the wait and the body run once
more, only once, for interact, write and press_key with a selector. A cancelled call, or one
whose browser is ended, stops at its next poll and answers as ended (WS-40).
ACT-6. A selector is one object with exactly one of xpath or css, a non-empty string. A JSON
string is parsed, which a multipart form needs. Id, name, tag, class and link text are reachable
through those two.
One naming both, neither or another key is refused with a sentence that names what was given and shows an example, and so is a value that is not an object. There is never a fallback from one to the other.
Which actions take one:
-
interact,write,extractandupload_filerequire one. -
dragrequires one for the source and takes the destination astoorby_x/by_y. -
press_key,outlineandscreenshottake an optional one. -
frametakes one forswitchas an alternative toindex. - The rest take none.
ACT-7. An action waits for its element in one of three ways, polling every 0.2 s.
| Wait | Satisfied by | Used by |
|---|---|---|
| Clickable | The first match that is displayed and enabled | click, double_click, right_click; drag's source; write; press_key with a selector |
| Shown | The first displayed match, else the first | hover, scroll_to (a clickable wait would refuse the off-screen element it is for); drag's to
|
| Presence | The first match, displayed or not |
upload_file, extract, frame, outline, screenshot of an element |
ACT-8. wait_timeout is seconds, 30 by default (10 for dialog); a value that is not a whole number is
the default. A clickable or shown wait asks about the first 50 matches per poll, since each
is_displayed is a round trip inside the turn, and skips a match that goes stale. When none is
usable a clickable wait keeps waiting and a shown wait settles on the first match.
ACT-9. A timeout reads no clickable element matched '<value>' within <n>s (no element matched for the other waits), then The browser is at '<url>'; if that is not the page you expected, the wait is not the problem.
Then, for a clickable wait only, what the page says:
- with no match:
Nothing on the page matches it.; - with one: that element's reason (ACT-10);
- with several:
<n> elements match it, and none could be used:and the first five as[i] <reason>,and <n> more,Only the first 50 are tried; narrow the selector.past 50, and the move of the first listed match that has one.
The page's answer never replaces the timeout.
ACT-10. The reasons an element cannot be used, tried in this order. One page script gives them to the
timeout, a click's intercept and outline. What is on top is read from the whole stack at the
click point (elementsFromPoint):
ACT-11. When more than one element matched, interact, drag, write and press_key return
matched (how many) and index (which, from 0), and never when one matched. A selector matching
several never fails a call by itself and is never resolved blindly.
ACT-12. Every page action returns the top page's url and title, from inside a frame too. An
open dialog does not fail a call that otherwise succeeded: url and title are null, dialog holds
the message and hint says to answer it with dialog.
ACT-13. write(submit=true) and press_key with Enter or Return wait for a navigation the
keystroke may have started, since sending keys carries no promise to wait: up to 2 s for the focused
element to go stale, then for the new document to stop loading. Expiring is ordinary, because a form
handled in JavaScript never navigates. interact and navigate wait through WebDriver.
ACT-14. A single-page app sets its title after the URL changes. When a call moved the page to
another screen within the same document and the title is still the one the call began with, it is
waited for up to 1 s. A full load, a frame, a dialog, a failed read, a title already changed and a
move that left the screen as it was never wait. A screen is the path and the hash route with its own
query; a change to the URL's query, to an anchor or to a non-route fragment is the same screen, so a
search box mirroring into ?q= stays fast.
ACT-15. interact, drag, write, press_key, extract, execute_script, assert, outline,
screenshot, print, upload_file and save_site_data take an optional url, which is not an
assertion: when the browser is elsewhere it navigates there first, and on that page it does nothing.
navigate's url is required and always loads. frame, resize and dialog take none.
ACT-16. Two URLs are one page when their host, port, path, hash route, query, route query and
userinfo are equal (urls.same_page). The host compares without case, a default port written out
equals none, and a trailing / on the path does not count. A hash route is a fragment starting /
or !/, cut at its own ?, without a trailing /, the ! dropped; any other fragment (#top,
#access_token=...) is not part of a page. The root route counts as no route unless it carries a
query, so /#/ and / are one page. A URL with no page key (about:blank) compares without its
fragment or trailing slash.
ACT-17. A caller may open http, https and about:blank. Any other scheme (data:, file:,
javascript:, view-source:) is refused naming the scheme and never the URL, which may carry
credentials.
ACT-18. Every page a caller names loads through browser.go. A host that does not resolve and a
certificate the browser does not trust are 400s worded as ERR-17 says, in place of the browser's
own text, which quotes the URL.
ACT-19. interact takes click, double_click, right_click, hover or scroll_to. click
is WebDriver's element click, which refuses a covered target; double_click and right_click are
pointer actions that press whatever is at the centre. All but scroll_to move the pointer onto the
element first and act second, so afterwards it is where a person's would be.
ACT-20. For the three that press, a script runs at the click point after the move and before the gesture. The point is the in-view centre of the target's first client rect, which is the point WebDriver clicks.
When the topmost element there is the target or inside it, the result has received_by
{element, depth, painted}, depth 0 being the target. It also has a warning when all of these
hold:
- the receiver is at depth 1 or more;
- it is unpainted or deeper than 2;
- neither it, the target nor anything between is a control.
A selected link or button never warns.
ACT-21. A double_click or right_click whose receiver is outside the target is reported, never
refused: received_by has depth null and a warning says what took it and why, with its checked
selector.
- A click whose receiver is outside is WebDriver's to refuse, and if it lands nothing is reported.
- hover and scroll_to report no receiver.
- A failed probe leaves its fields out and never fails the gesture.
An intercepted click raises the driver's first line, then, when several matched, The selector matched <n>; this was [<i>], the first that could be used., then the reason (ACT-10) of the element
that was clicked, not of the selector's first match.
ACT-22. An element is painted when no ancestor has opacity 0, it is not visibility:hidden, and
one of these holds:
- it is a replaced element (
img,video,canvas,iframe,embed,object,input,textarea,select); - it has a background colour with alpha above 0, a background image, a non-blank text node of its
own, or a
::beforeor::afterthat draws; - it is an SVG element other than
<svg>with a fill or stroke that is notnoneor transparent.
Borders and shadows do not count.
ACT-23. The pointer report has glided when a move was sent, nudged when the pointer was
already inside the target and glide_note when a requested glide was a jump. A move onto a target
the pointer is already inside first steps out to a point near it, so mouseover fires. glide is
false for interact and true for drag: a straight eased path of 6 to 40 steps before WebDriver's
own element move, a jump when the start is not known.
ACT-24. The pointer's position is kept per Grid session id, in memory or in Redis to match the workspace store.
- It is forgotten when a browser opens, when
end_browserruns and when a move failed part way. - A store that fails means an unknown position and fails nothing.
- A move WebDriver will not make (an element taller than the viewport) costs only the position, and the gesture runs as WebDriver does it.
- A gone browser or an unreachable Grid still fails the call.
ACT-25. drag presses at the source's centre, holds 120 ms, travels, holds again and releases;
the approach is a jump. The destination is the first displayed match of to, or the landing point
moved by by_x and by_y, read after the approach because scrolling moves every rect. One outside
the window is clamped to its edge and clamped says where. The result has from, to and glided.
ACT-26. write waits for a clickable element, clears it unless clear is false, types text and
returns value, the field's value read back.
- A password field is never read back, nor is a bound secret's field: with the read turned off it is not made (SEC-22).
- A bound write into an element that hosts another document is refused (SEC-20).
-
submit=truereads the value first, because submitting makes the element stale, then sends Enter and settles (ACT-13).
ACT-27. press_key with a selector waits for a clickable element and sends the key there. With
none it sends to the document's active element, <body> when nothing has focus, because sending to
<body> itself takes focus off the element that had it.
ACT-28. A key is one of:
- one character;
- a name in any spelling (
ArrowLeft,arrow_leftandarrow-leftare one key); - a combination joined with
+such asControl+a, where a+after a+is the plus key. A combination is one sequence, each modifier held for the keys after it.
An unknown name is refused naming it and listing every key name.
ACT-29. frame takes switch, parent or default; switch needs a selector or an index, and
a selector wins when both are given. The frame the session is in is state on the Grid: it lasts across
calls until something switches back, in_frame reports it, and every locator looks in that frame
only. in_frame is window !== window.top, never window.self, which a page can forge with
self = top.
ACT-30. dialog takes accept (the default), dismiss, read or send_text. It waits up to
wait_timeout for a dialog, reads its message before answering, and returns action and message.
read leaves the dialog open. send_text types text and accepts. resize sets the window from
width and height, keeping one left out, and returns the size the window has; it is the one action
that changes what the workspace stores, so a reopened browser has that size.
ACT-31. extract waits for presence and returns text, what a person sees, and html, the element's
innerHTML. No password input's value attribute is in html, in a <template> too (SEC-65).
ACT-32. outline returns elements, count and total.
- By default the elements are the actionable ones: links, buttons, form controls,
summary,label,iframe, and anything with anonclick, atabindex,contenteditable, an ARIA state or a widget role.interactive=falselists every element. -
textkeeps those whose name contains it, ignoring case. -
limit(default 50, never negative) cuts the list whiletotalstill counts what matched.
ACT-33. Content comes before navigation, banner, footer and sidebar chrome, each entry carrying
its region. An entry has role, name, visible and a selector checked to match exactly that
element. The selector is the first of these that fits:
-
#id; -
data-testid,data-test,data-qa,name,aria-label,placeholder,titleorhref, when unique; -
normalize-space()XPath on its own text; - a full CSS path.
A not-visible entry has reason (ACT-10), blocked_by when the reason names something, and for a
hidden one with a found trigger revealed_by and open_with. expanded appears with
aria-expanded.
ACT-34. A name comes from aria-label, aria-labelledby, an image's alt, a form field's label,
placeholder, button value or name, and, for the roles ARIA names from content (button, link, tab,
option and the like), its text. Anything else is named only from text it owns, where at most one
child at each level carries any, so siblings are never run together. A name is cut to 80 characters
ending in …, after the scrub.
ACT-35. screenshot returns image (base64 PNG), width, height and bytes.
-
widthandheightresize the window first and leave it so;full_pageresizes to the document and restores. An element is found after the window is sized. - Unless
saveis false the PNG is kept in the workspace asfile, and a failed save says why infile_errorand never costs the image (FILE-16, FILE-17).
print makes a PDF (with landscape and background, off by default) or an HTML file (the
document as it stands), keeps it in the workspace's Files and returns file, format and bytes
(FILE-18).
ACT-36. execute_script runs the script with no arguments and returns result. One that runs past the
session's script timeout is a 400 worded as ERR-17 says, since Selenium's bare timeout reads as a
failed element wait.
ACT-37. assert runs a script that must return a boolean; any other value is refused naming its
shape (null, a string of 3 characters) and never its content. It asks again every 0.2 s until
the answer is true or wait_timeout passes, wait_timeout=0 asking once, and an answer arriving
after the deadline does not pass.
ACT-38. stable_for (seconds, 0 by default) requires the answer to stay true that long, and the
clock restarts each time it turns false. A value that is not a number is refused, never read as
zero, which would remove the guard. The result has asserted, script, the page as it
was when the answer held, and stable_for when one was asked.
ACT-39. A failed assertion says the caller's message, or assertion failed after <n>s on '<url>'. Give the step a message to say what should have been true., or became true on '<url>' but did not hold for <n>s when it had been true. The script is never echoed, since it can carry a
literal.
ACT-40. upload_file takes exactly one source: text, content (base64) or file, a
workspace://files or session://downloads address. FILE-20 and FILE-21 say how the source is
checked and how the bytes reach the browser, and FILE-12 to FILE-14 how the name the page sees is
made.
ACT-41. A failure is one line a model can act on (ERR-14). It names the fix when there is one: the
selector shape, the key names, dialog for an open dialog, insecure=true for an untrusted
certificate. A selector that never matched, an intercepted click, a refused argument, a failed
assertion, a script timeout, an unresolved host and an untrusted certificate are the caller's, a
400 (SURF-41). A Grid node that stops answering mid-call is NodeLost (ERR-17).
ACT-42. Each action is one row of core.capabilities.CAPABILITIES (SURF-2) and one method of
core.actions.Actions, named alike except assert (assert_) and print (print_). A tool, a
route and a flow step end in that method through Workspaces.act, so all three get the same answer.
The page scripts are files in js/, concatenated into one string because execute_script has no
module loader. A click's receiver, a timeout's reason and outline share one helper file, so the
error and the map cannot disagree about one element.
Not built. The reasoning, and the questions still open, are in the multiplayer design.
ACT-43. (proposed) Every session can switch between its tabs and follow the popup windows its pages open, whether or not another agent shares it.
ACT-44. (proposed) An action reaches the browser over one of two transports, classic WebDriver
or BiDi, chosen as auto, classic or bidi. Classic stays for every action, and BiDi is used
where it is needed. Where the choice is set, beyond run_flow (FLOW-50), and what auto picks for
each action are open; the BiDi action layer has a design of its own.
The action pages are generated from the server's tool schemas, so they describe the version that shipped.
selenium-flow · MIT
Start here
Guides
Lifecycle
Going places
Doing things
Getting things out
Site data
Console and network