Repository navigation
capture
A browser opened with capture=true keeps what its pages log and fetch. This area covers what
capture hears over BiDi and when it listens, the rows it builds, where it keeps them and for how
long, how an agent or an admin reads them, the hint a call's result carries, the masking of secrets
in everything captured, and the fetch of one response body.
- An agent opens a browser with
capture=true, submits a form, and seespagein the result witherrors: 1and a URI. It reads that URI and finds the uncaught exception and where it was thrown. - An agent lists
workspace://network?filter=failedto find the request that returned a 500, readsworkspace://network/3.7for its headers, and callsnetwork_bodyfor the error text. - An agent reads
workspace://consoleafter the browser was reaped; the entries are still there. - An admin opens a workspace's console or network, reads what was kept, or watches rows arrive live.
CAP-1. Browsers, reaps and the monitor's watch are in
workspaces-and-sessions.md, and capture as an open_session option
is in session-settings.md. The scrub of every result and resource, and the
secrets catalogue, are in secrets-and-scope.md. Resources and tools as
surfaces are in surfaces.md, and signed links and the admin door are in
auth-and-roles.md. The spare tab is in site-data.md. The
Console and Network tabs are in admin-ui.md; this file holds the routes behind them.
Recordings and files are in workspace-files.md.
CAP-2. open_session(capture=true) turns capture on for the browser it opens. The flag is
off unless a call passes it: no config key, header or default sets it.
CAP-3. The workspace stores the flag with its other settings, and a silent reopen after a reap replays
it. An explicit open_session that does not pass capture=true does not inherit it, and discards
the workspace's captured rows.
CAP-4. A server whose monitor or capture store was not built refuses capture=true with a 400,
Capture is not set up on this server. A reopen on such a server replays the settings without
capture.
CAP-5. Capture needs the browser's BiDi channel through the Grid's /se/bidi route. Every
browser opens with webSocketUrl set so that the channel exists. Chrome and Firefox both capture.
CAP-6. Capture attaches after the browser exists and before its first page, and the open waits at most 15 seconds for it. The connection is open, subscribed and collecting by then, so the first page's events are heard.
CAP-7. When the attach fails or times out, the open fails with a 503 saying the Grid's BiDi route did not answer, the browser is quit, and the capture hold is released. No browser is left running that the caller was not handed.
CAP-9. Capture subscribes to ten BiDi events on the session's one connection: log.entryAdded;
network.beforeRequestSent, network.responseCompleted and network.fetchError; and
browsingContext.contextCreated, contextDestroyed, navigationStarted, navigationCommitted,
historyUpdated and fragmentNavigated.
CAP-10. The connection also adds a BiDi data collector for response bodies, with maxEncodedDataSize
10,000,000 and collectorType blob. The browser, not this server, keeps each body.
CAP-11. The monitor holds the connection while capture is owed and the watch's deadline has not passed. The deadline is the last activity plus the node's session timeout (300 seconds when the node did not say). A call in flight keeps the connection past it.
CAP-12. At the deadline the monitor closes the connection clean, unsubscribing and removing the
collector first. Capture is then paused, and capturing reads false.
CAP-13. The next call on the session holds the connection again before it runs, so the call that comes back from a pause is captured. A hold that fails backs off 1, 2, 4 and up to 30 seconds, and a call in the back-off does not try. It costs that call its capture and never fails it.
CAP-14. What the browser does while capture is paused is not heard. After a hold again on Chrome, the replay of the last 100 console entries per browsing context is aligned with what was kept: only the entries past the overlap are added. Identical entries are real entries, so alignment compares the sequence, never a set. Firefox's late subscribers get no replay, and nothing is aligned.
CAP-17. Nothing runs on the connection's read loop. Each subscriber has its own queue, at most
1000 events and 32 MiB of frames. An event that finds the queue full is dropped and counted in
dropped.
CAP-18. The spare tab that site data uses is invisible to capture for its whole life: its browsing context is ignored by id, so its requests and logs are neither kept nor tailed.
CAP-19. A console entry has seq, nav, at (epoch seconds), level, type, text,
source ({"url", "line"} or null) and context. line is one-based. text is masked and then
cut to 4096 characters. The entry's arguments are not kept.
CAP-20. A request is one row assembled from its events, with id, seq, nav, at, method,
url, status, status_text, type, mime, size, took_ms, failed, error, context and
navigation (true for a top-level page load). Its request_headers, response_headers and
headers_dropped appear only in the request's own view (CAP-44).
CAP-21. A request's id is <nav>.<n>: the page's number and the request's order in that page,
from 1. The id is this server's. A side index maps it to the browser's own request id and keeps the
browser's session id, MIME type and size.
CAP-22. A request's type is document for a navigation, else from the fetch destination (document,
stylesheet, image, font, script, media), else from the initiator (fetch, xhr), else
from the MIME type, else other.
CAP-23. size is bodySize, else bytesReceived, else null. It is never content.size.
CAP-24. took_ms is responseEnd less requestTime from the request's timings, on both
browsers. Without a responseEnd, it is the end event's timestamp less the start's. A result below
zero or over an hour is null.
CAP-27. A failed request has failed true and error set to the masked errorText, and has
no status.
CAP-28. A page is a navigation of the top-level browsing context that was present at attach.
navigationStarted of that context starts the next page: nav goes up by one and a mark is added.
Navigations of other contexts, frames and other top-level tabs mark nothing, and their rows are
kept under the current page.
CAP-29. A mark is {"nav", "url", "at"}. navigationCommitted, historyUpdated and
fragmentNavigated of the top context update the current mark's URL and start no page. Rows before
the first navigation belong to page 0.
CAP-30. The buffer marks the last 5 pages. A reader asks for the last pages of them, 1 to 5, default
1.
CAP-31. A URL keeps its query string and fragment only while its page is the newest. When the next page
starts, the earlier page's rows, console sources, mark and the headers referer, location and
content-location lose them.
CAP-32. Captured rows live in the server's memory, in one buffer per workspace. Nothing is written to disk or to the workspace store, so a restart loses them.
CAP-33. A buffer holds at most 1000 console entries and 1000 requests. When one is full, the oldest row goes, and a request takes its index entry with it. Requests in flight are capped at 1000 the same way.
CAP-34. Not kept, only counted per page: console entries of level debug (not_kept.debug),
and requests of type stylesheet, image, font, script or media that answered 2xx or 3xx
(not_kept.static). A failed or 4xx or 5xx static request is kept. The live tail sees both (CAP-50).
CAP-35. The server remembers at most 256 workspaces' buffers. Past it, the least recently used goes.
CAP-36. A buffer expires capture.ttl seconds after the last row that landed in it. The key
capture.ttl defaults to 10800 and is at least 60. Reading never extends it.
CAP-37. Captured rows belong to the workspace and outlive the browser. After end_browser, an
admin's End or a reap, workspace://console and workspace://network still answer until
capture.ttl passes with nothing new, or until an explicit open_session without capture=true.
CAP-39. Capture keeps a record of its own per browser: its Grid id, its top context, its masker and what Chrome's replay is aligned against. It outlives an expired buffer, so a page that logs after hours of quiet is captured again with its secrets still masked. At most 256 are remembered, and one whose connection is held is never evicted. An evicted idle one is parked, and a hold again takes it back whole.
CAP-41. The workspace has three resources: workspace://console, workspace://network and
workspace://network/{id}. The first two take a query: ?level= and ?pages= for the console,
?filter= and ?pages= for the network. Reading never opens a browser.
CAP-42. The query values read like this:
-
levelisall,errororwarn.errorselects entries of levelerrorand uncaught exceptions (typejavascript), andwarnselects levelwarn. -
filterisall,failedorxhr.failedselects a failed request or a status of 400 or more, andxhrselects typexhrorfetch. - Both names are trimmed and read without case, and omitted is
all. An unknownlevelorfilteris a 400. -
pagesaccepts"3"as 3, and omitted is 1. A boolean or a number outside 1 to 5 is a 400.
CAP-43. A console or network view is {"capture": true, "pages", "count", "total", "tally", "not_kept", ...} plus entries or requests, newest first.
-
pagesis the marks in view, newest first. -
countis the rows shown, andtotalthe rows of the pages in view before the level or filter. -
tallycountserrorsandwarningsin a console view, andfailedandxhrin a network one. -
not_kepthasdebugorstatic.
CAP-44. A request's own view is the row with request_headers, response_headers,
headers_dropped and its uri, workspace://network/{id}. An id that is not in the buffer is a
400 naming workspace://network. Rows in a list carry none of the header fields.
CAP-45. A view carries dropped when the browser sent events capture never heard (CAP-17, and
the waits in CAP-60), and unmasked when a credential was too long to learn (CAP-62). Each appears
only when non-zero and counts since the buffer began, not per page.
CAP-46. A workspace that does not capture, or whose record is gone, answers every capture
resource with {"capture": false, "how": "open_session(capture=true)"}. That is an answer, not an
error. A workspace that captures and has heard nothing answers an empty view. Every view carries
capturing, true while the connection is held right now and false while it is paused or being held
again, so a quiet buffer is not mistaken for a quiet page.
CAP-47. See SEC-58: a resource's view passes the workspace scrub like any other resource.
CAP-48. workspace://current carries capture, capturing and, with capture on, captured:
what was kept since the last navigation (console, network, errors, failed_requests), and
dropped and unmasked when present. A workspace with no buffer yet reports zeros.
CAP-49. The admin page may listen to rows as they land. A subscription exists only while a page
listens: with no listener, nothing is queued. Its queue holds 500 rows, and an overflow is counted.
Before the next row, the stream says {"dropped": <total>} whenever that total moved.
CAP-50. A tail row is the same masked row the buffer holds, without header fields, plus kept.
kept is false for a debug entry and for a successful static request, which the buffer counts and
does not keep. The server stores no tail.
CAP-51. With capture on, a call's result carries page when the page reported a failure while
the call ran: {"errors", "failed_requests", "see"}. It is absent when both counts are zero.
Both surfaces carry it, and it is declared in every capability's response schema.
CAP-52. errors counts console entries of level error and uncaught exceptions that landed
during the call. failed_requests counts documents, fetches and XHRs that failed or answered 4xx or
5xx. Warnings and failed images, stylesheets, fonts, scripts and media are not counted
(merge.counts_in_hint).
CAP-53. see is workspace://console?level=error when errors is non-zero, else
workspace://network?filter=failed.
CAP-54. The call's window is its turn on the session's lock, so another call's events do not land in
it. The call reads the buffer's running totals when it begins and subtracts them when it ends.
Before the count, it does one session.status round trip on the held connection, so the events
the browser sent before it land first.
CAP-55. A call that has no buffer at its start, because none was created or it expired, has no hint.
CAP-56. A flow run's report carries the sum of its steps' hints. A call that raised carries the hint as
one note on its error: This call caused N page error(s) and M failed request(s): read <see>.
CAP-58. Masking happens once, at capture, so the buffer and the tail hold the same masked rows. One masker per browser masks every text a page or a server chose: method, URL, console text, source URL, status text, MIME type, error text, header names and values.
CAP-59. The masker knows three kinds of value:
- the forms of the secrets bound in this workspace, which
write(secret=...)hands to capture before the first keystroke; - the forms of the secrets in scope on the origins the browser reaches (CAP-60);
- the credentials requests carry, learned as they pass: an
AuthorizationorProxy-Authorizationtoken (after its scheme) and the values of cookies from the BiDicookies,CookieandSet-Cookie.
Each is masked wherever it appears, in every spelling an echo gives it. A catalogued secret shows
its marker [secret:<name>/<key>] (SEC-51), and anything else shows ••• (SEC-64).
CAP-60. With secrets configured, a context's events wait, raw and unbuilt, while the secrets in scope on its new origin are read, then replay masked through the same handlers.
- At most 200 events wait per context and about 32 MiB per browser, oldest first out.
- An event older than 10 seconds is dropped when the next arrives, since nothing runs a timer.
- A read that fails is retried once as every usable secret (SEC-38), and the events land masked with it. When that fails too, what waited is dropped.
Each drop is counted in dropped, and none is kept raw.
CAP-61. The headers authorization, proxy-authorization, cookie and set-cookie (any case)
show ••• whatever their value. A password in a URL's userinfo shows •••, in the URL and in a
referer, location or content-location value.
CAP-62. A value is learned as a credential only when it can be a secret:
- a token of 8 or more characters, or a cookie of 16 or more;
- not only letters and not only digits, with no NUL;
- at most 4096 characters, or 16,384 for a token spelt as RFC 6750's b64token.
A value too long is not learned and is counted once in unmasked.
CAP-63. Learned values are bounded per browser in two tiers of 32,768 characters of forms each.
- Tokens have 16 forms, and the least recently seen goes first.
- Cookies have 48, and a full tier holds the highest ranked. A session-like name ranks over any other and a tracker's under all, then by distinct characters, length and value.
- A cookie not sent for 64 events ranks below every one that was, each cookie domain is sure of half the tier, and one event learns at most 8 new cookies.
CAP-65. A request's headers on each side keep at most 32 headers while their names and values fit
16,384 bytes. Names are cut to 256 characters and values to 4096. The rest are counted in
headers_dropped. Every header is read for credentials before any is shown, kept or not.
CAP-66. The goog:postData field and a request's body are never read, so they reach no row.
CAP-67. network_body(id) returns one response body asked of the browser that made the request.
It is a tool and POST /browser/network-body, takes id (1 to 64 characters) and works while a
person holds the browser.
CAP-68. The call is a 400 for an id that names no request in the workspace's buffer. With no session open, a request the buffer knows is a 404 saying the body went with the browser, and any other id is the usual 400. A request made by an earlier browser is a 404 naming that.
CAP-69. A body whose row's MIME type is not text is not fetched. The answer is binary: true, text: null, and the row's mime and size. Text is text/*, or a subtype json, xml, javascript,
ecmascript (also their x- forms) or x-www-form-urlencoded, with a +json or +xml suffix
allowed. An empty MIME type is asked for.
CAP-70. A text body whose row says it is over 8,000,000 bytes is refused with a 400 before
anything is asked, naming workspace://network/{id}.
CAP-71. The body is asked with network.getData on a connection of its own, never the one
capture holds, because Chrome answers in one frame and a frame over 16 MiB closes its socket. A
frame too large is a 400. The browser's error for a body it does not hold, or whose collector is
gone, is a 404 in this server's words. Any other error is a fixed 500.
CAP-72. A body that the browser returns as base64 is binary, with its decoded size. Otherwise
the answer is {"id", "uri", "mime", "size", "binary": false, "text", "truncated"}.
CAP-73. The text is masked whole with the browser's masker, then, through the tool, scrubbed with every
leashed secret (SEC-67), then cut to 100,000 bytes with truncated saying whether it was. The cut
comes last so no value is cut in half. size is the byte length the browser sent, before the
cut.
CAP-75. The admin door guards these routes under /admin/workspaces/{key}:
-
GET /console?level=&pages=,GET /network?filter=&pages=andGET /network/{id}answer the same views as the resources, through the code the resources use, so the tab and the model cannot disagree; -
POST /network/{id}/bodyanswers the body of CAP-67; -
GET /tail?kind=console|networkstreams the live tail.
CAP-76. The console and network views carry tail_url, a signed link to the tail minted only for a
caller the admin door admitted, when capture is on. The tail opens on that link (AUTH-29), since an
EventSource cannot send a header. A kind that is not console or network is a 400.
CAP-78. The admin's body route reads the body of the workspace's live browser and never reopens one: with none alive it is a 404 saying the body went with the browser. Its text is masked by the browser's masker and cut to 100,000 bytes, and it does not pass the workspace scrub.
CAP-80. Capture does not keep:
- request bodies, and response bodies (the browser keeps those, CAP-10);
- WebSocket or EventSource frames, since capture subscribes to no event for them;
- console arguments, beyond the text the browser renders;
- debug entries and successful static requests, beyond their counts (CAP-34);
- what the spare tab did (CAP-18);
- anything the browser did while capture was paused (CAP-14), except Chrome's console replay;
- events dropped by a full queue or an expired wait (CAP-17, CAP-60);
- cookie values or their names as rows, beyond the masked
CookieandSet-Cookieheaders; - anything from a browser opened without
capture=true.
Not built. The reasoning, and the questions still open, are in the multiplayer design.
CAP-81. (proposed) What a session's pages logged and requested is that session's, and a reader of the console or network sees one session's rows. Replaces, when built, CAP-32's "one buffer per workspace" and CAP-37's "Captured rows belong to the workspace". How rows are keyed, and whether they outlive the session's browser as CAP-37 says today, are open.
The action pages are generated from the server's tool schemas, so they describe the version that shipped.
selenium-flow · MIT
Start here
Guides
Lifecycle
Going places
Doing things
Getting things out
Site data
Console and network