Skip to content

v0.11.0

Choose a tag to compare

@github-actions github-actions released this 20 Aug 20:53
· 86 commits to main since this release
8e0f14e

Breaking

  • Unified control action vocabulary: FillValue is removed. fill and completeForm intents now use a single ControlAction enum shared with the control_action MCP tool. Old fill vocabulary replaced with new unified kinds:
old (fill) new
{"kind":"text","text":X,"clearFirst":false} {"kind":"setText","value":X,"clearFirst":false} (clearFirst now defaults true = replace; pass false to append)
{"kind":"select","option":X} {"kind":"selectOne","value":X}
{"kind":"checked","checked":X} {"kind":"setChecked","checked":X}
{"kind":"files","paths":X} {"kind":"setFiles","paths":X}
  • New operations available in fill: selectMany (multi-select), clear (clear field).
  • activate is rejected in fill; it remains control_action-only.
  • control_action setText accepts clearFirst (default true, replace behavior; unchanged from prior hard-coded behavior in worker-pool and firefox-companion).
  • type_text remains unchanged; its clearFirst still defaults false.
  • Interface version bumped to 2026-08-19; HTTP clients must send the new x-interface-version.

Added

  • Qwen3.5-27B-4bit (mlx-community/Qwen3.5-27B-4bit) is the MLX vision default in vision-proxy, bobby vision connect, and the bobby install model list, where it is the recommended first choice. The previous Qwen2.5-VL-3B default could not drive the solve loop.
  • VISION_COORD_SPACE=normalized|absolute overrides the mlx-vlm provider's per-model coordinate-space detection.
  • network_log reports networkRecordingStarted on the call that attaches the collector, and its description states that recording begins at the first call on a page. A first call previously returned an empty HAR with nothing naming the cause.
  • bobby doctor's companion-port check names the listening process's pid and command, read from lsof under a 2s cap on unix. It is omitted rather than blocking on any lookup failure or non-unix platform.
  • click carries dialogOpened evidence when alert/confirm/prompt opens during the click.
  • type_text carries typedControlKind evidence, which verification uses to accept a checkable's or a select's post-state.
  • form_snapshot collects [role=button] elements and names a button from its own text when it carries no label, so buttons are targetable straight from the snapshot.
  • The DOM candidate collector maps implicit roles for headings, lists, list items, images, tables, rows, and cells, so those elements resolve by role with no explicit role attribute.

Changed

  • Interface error messages carry the allowlisted diagnostic and the repair action, capped at 1024 bytes with the diagnostic truncated first so the repair action survives intact. A non-allowlisted raw message is still never included.
  • The candidate limit applies to the matching set after filtering instead of every gathered candidate, and an explicit ordinal skips the bound because it picks one match deterministically. A large page no longer fails a target that matches once.
  • type_text splits its value into runs the US keyboard layout can key-press and runs that need a caret-level Input.insertText, so Unicode outside that layout and newlines are typed rather than key-pressed.
  • The browser-launch repair and the engineUnreachable diagnostic name the Firefox companion bind (default 127.0.0.1:9876) and the other runtime that may hold it, alongside the BiDi endpoint.
  • visionAssistDenied leads with the deterministic stuck reason (targetNotFound, targetAmbiguous, obstructionSuspected) and then names the closed gate. The code is unchanged; the message read as a policy wall to an agent that never asked for vision.
  • A duplicate in-flight JSON-RPC request id is rejected with a diagnostic and a repair instead of a bare Invalid Request.
  • The advertised executionPath.reason enum carries a description: ineligibleCommand means the command class runs in the browser and is not a failure.
  • Every failing command outcome is logged at WARN with command, session, page, outcome, code, retryable, and message.
  • The targetNotFound re-collect loop backs off 25, 50, 100, 200, 400 ms, capped at 500 ms, instead of re-scanning a large page every 25 ms for the whole deadline.
  • A page opened by page_open or an agent workflow reports its navigated URL and an interactive ready state, and is listed on its session, instead of the pre-navigation blank state.

Fixed

  • A click that opens alert/confirm/prompt returns instead of hanging. The renderer blocks while the dialog is up, so the click's own CDP round trip may never answer; the click races a per-page dialog listener and reports dialogOpened. dialog consumes a dialog that opened before it was called rather than waiting on a future event that already fired.
  • click and keyboard dispatch bring the target page to front first. On a background target in headless Chrome the click path never resolved. A failed raise is logged and the dispatch proceeds.
  • type_text against a non-editable target — disabled, readonly, a fieldset[disabled] ancestor, a non-typeable input type, or a non-form element — fails with invalidRequest instead of dispatching key events, which pinned headless Chrome at 100% CPU on macOS.
  • A screenshot clip is bounded before it reaches CDP: a negative origin, a non-finite value, or a dimension over max_screenshot_dimension is refused with invalidRequest. An unbounded clip could kill the browser process.
  • a11y_snapshot reserves a node's budget slot before descending into its children, so a deep subtree of leaves cannot consume the whole budget and return zero nodes.
  • Page operations clone a browser handle and release the worker mutex instead of holding it across a CDP await, so one page's slow or hung call no longer serializes or stalls every other page in the session.
  • A dead browser target rewritten by the targeting layer triggers the same revive path as a raw CDP receiver-is-gone error, instead of wedging the page.
  • A navigation the browser aborted (net::ERR_ABORTED — a download response or a cancelled navigation) fails as non-retryable and points at download_url and click_and_wait_for_download, instead of advising a retry that repeats the abort. The frame's navigation slot is freed.
  • Driver failures are retryable by code: a malformed request, a target that is not there, and a policy refusal are not retried; transport, launch, internal, and timeout codes are.
  • httpResponseTooLarge is a plain failed rather than needsReconciliation. The body is dropped mid-stream with nothing written, so no effect can have landed.
  • An iframe hop resolves by DOM identity — DOM.getFrameOwner's backend node id against the candidate's own — instead of name and src, so unnamed sibling iframes are told apart. An unnamed hop's stamped target validates and resolves, so a snapshot target passes back verbatim.
  • Role matching treats img and image as one role. Chrome's a11y tree emits image while the DOM collector emits img, so a snapshot role fed back as a target resolves either way.
  • Accessible names are trimmed on both sides of the comparison, so a name carrying surrounding whitespace matches.
  • type_text verification accepts an append (clearFirst: false), a checkable's checked state, and a select's committed option, instead of requiring the post-action read to equal the typed value exactly.
  • The mlx-vlm provider rescales normalized [0, 1000) click coordinates onto the screenshot frame for the families that emit them (Qwen3-VL, Qwen3.5), and unwraps list-typed coordinates such as {"x": [566]} and {"coordinate": [[x, y]]}.
  • The mlx-vlm provider builds a system+user chat template when the processor supports it. The single-message template made Qwen3.5 emit a click with no y key.
  • Formula/bobby-browser.rb carries the v0.10.0 asset digests.