Repository navigation
Foxhand helper v0.5.4
The tools are reworked so agents can read and act on real pages on the first
try, without working around them. Every change below was checked on live
sites (Spiegel, GitHub, YouTube, Wikipedia, TodoMVC, react-select, MUI,
Lexical, ProseMirror, Slate, CKEditor) as well as in the tests. The names now
follow one pattern, <thing>_<verb>. Agents and prompts that call tools by
their old names need updating.
Buttons the snapshot used to miss
- Buttons built from plain elements. A
<div>the page made clickable
with script shows as roleclickable, from a pointer cursor, a click
attribute, Google'sjsaction, or a React or Vue click handler. - Icon-only buttons have names, from their tooltip, SVG title, the icon's
own name (fa-trashbecomes "icon: trash"), a test id or a class. - Closed shadow roots are read and acted on, not only open ones.
- Controls that only show on hover, like the ✕ on a to-do item, are listed
(marked[on hover]) and can be clicked, although an add-on can't produce
CSS:hover. - Styled checkboxes and radios (a hidden input under a styled label) show
as the checkbox, with its state. - Unlabeled fields are named by the text beside them or, for a custom
dropdown, the value it shows.
iframes from other sites
- Cookie banners, sign-in buttons, payment fields and embeds in cross-origin
iframes are part of the snapshot. Their refs work like any other; a name or
selector that isn't on the page is looked for in its frames; a click at a
point over a frame lands inside it.page_textandpage_findinclude their
text. - A consent dialog that closes itself after "Accept" is reported as done,
not as a failure the agent would retry.
Snapshots with a shape
- Controls are nested under what they belong to: the landmark, dialog,
form, list item, table row or card. The agent can tell which "Delete" goes
with which row. Lines that add nothing are folded away. - An open modal, or an overlay covering the screen, is listed first and
flagged in the header, so a long page can't push it out of the snapshot. - Repeated buttons say which card they're in: each "Add to cart" on a shop
page is labeled with its product, each "More actions" with its video, even
when the cards are plain<div>s.
Typing like a person
- Every character is a key press: keydown, keypress, beforeinput, the edit,
input and keyup. Autocomplete boxes open, input masks (phone numbers, cards)
format, and one-box-per-digit code fields follow the focus the page moves. - Rich editors keep the text: Lexical, Slate, ProseMirror, CKEditor and
editors in same-origin frames. If an editor rejects the text, the result says
so instead of forcing text in and corrupting it. - Clicks land on the element under the pointer, such as the icon inside a
button, which is often where the page listens. - Submitting a form presses its real submit button. A "Cancel" or "Back"
button placed first in the form is skipped (before, a checkout form could be
cancelled instead of continued).
Renamed and merged
| Before | Now |
|---|---|
tabs_list, tabs_query |
tab_query (query: "*" lists every tab) |
active_tab |
tab_check_active |
nav_back, nav_forward, nav_reload |
tab_navigate with go: "back", "forward" or "reload" |
tab_duplicate |
tab_create with duplicateOf |
tab_pin, tab_unpin |
tab_pin with pinned: false to unpin |
focus_tab |
tab_focus |
window_list |
window_query |
snapshot_ax |
page_snapshot |
screenshot |
page_screenshot |
act_click, act_type, act_select, act_hover, act_scroll, act_key, act_wait |
page_click, page_type, page_select, page_hover, page_scroll, page_key, page_wait |
act_fill_form |
page_fill |
act_find |
page_find |
bookmarks_search, bookmarks_create, bookmarks_remove |
bookmark_query, bookmark_create, bookmark_remove |
history_search |
history_query |
downloads_list |
download_query |
cookies_for_tab |
cookie_query |
sessions_recently_closed, sessions_restore |
session_query, session_restore |
wait_for_tab_event |
event_wait |
cursor_note |
thought |
tab_move is removed. The thought argument on every tool is gone too: the
thought tool shows a note, and it stays on the next page action within
30 seconds. Otherwise the cursor describes each action itself.
Acting on pages
- Find elements by what they say. Every page tool takes
name(visible
text or label), with optionalroleandnth, as well asrefand
selector. When several elements match, the error lists them. - Refs last longer. A ref keeps pointing at its element until the page
navigates, so one snapshot serves several actions. - Results say what changed. Every action reports the elements that
appeared, text that appeared or went away (an error under a field, "Added to
cart"), what opened, closed, got checked or selected, new alert and status
messages, and the new URL or title. The agent rarely needs a new snapshot
after each step. After typing into a search box it waits briefly for the
suggestions. - A name that isn't found lists the closest ones, including common
synonyms ("Sign in" finds "Log in"). The name written exactly as given wins
over looser matches. - Clicks at a point.
page_clicktakesx,y, for maps, editors and games
drawn on a canvas. page_typefinds fields by label or placeholder and handles rich editors,
checkboxes and selects.page_selectopens custom dropdowns (comboboxes and listboxes) as well as
<select>.page_keyperforms the default action when the page doesn't: Tab moves
focus, Enter submits, and text keys edit.page_scrollfinds the inner scroller of chat apps and can scroll an
element into view.page_waitwaits for things to go (gone: true, for spinners), for the
page to settle, or for the address to change (urlContains, after a login
or checkout redirect).- Pages that never finish loading (long-polling, endless trackers) no
longer cost the full wait:tab_createandtab_navigatereturn once the
new page is up, withready: true. - Modifier keys in any spelling:
ctrl,Control,cmd,Option. - New:
page_dragfor sortable lists, sliders and map panning, and
page_uploadto attach files the agent supplies.
Reading pages
page_textcomes in chunks. Long pages no longer flood the agent's
context: follownextOffsetfor more.withinreads one region.page_textwithformat: "markdown"keeps headings, lists, tables,
emphasis and link URLs, and shows form fields.page_snapshotcan read one region (within) or only what's on screen
(viewportOnly), and includes shadow DOM.page_screenshotcaptures one element or the whole page, as PNG or
smaller JPEG, at any scale.page_findreturns the ref to click for each match and whether it's off
screen, and finds phrases split across elements ("Add to cart").
Browser data
download_querysays where files landed: path on disk, size, progress,
whether the file still exists and why a download failed.- New:
download_startdownloads a URL with the browser's cookies and
returns the path. - Cookies can be written. New
cookie_setandcookie_remove(needs
confirm:true). - New: a notebook for the agent.
storage_get,storage_setand
storage_removekeep JSON values in the add-on across pages and restarts.
Witharea: "local"or"session"they read and write the page's own
storage.
Install
Helper app for the Foxhand add-on (formerly WebMCP Controller).
Windows: open PowerShell and paste:
irm https://hamb1y.github.io/foxhand/ps | iexmacOS / Linux: open Terminal and paste:
curl -fsSL https://hamb1y.github.io/foxhand/sh | shAlready installed? Run the same command again to update: Firefox switches to the
new helper by itself within a few seconds.
Then click Retry in the add-on popup. Manual: download the file for your
system below and run it with install (Windows: double-click the .exe; macOS: run
xattr -c on it first). The webmcp-host-* files are the same programs under their old names.