Repository navigation
Releases
0.11.0
shotbox mutate runs a browser test against small changes to its page,
to find a test that breaks on one that shouldn't matter or passes on
one that should. wait node waits for a node's state, value or count,
and wait gone for one to go; wait idle waits until each program
answers, free on its main thread. capture --tree saves the tree
beside a picture, so compare says what each change is in;
--ignore-node leaves out a clock by its name, and compare --update
takes a reviewed change in one command. a11y diff says what changed
between two trees, and shoot waits for a node or an announcement.
record --sound keeps picture and sound in step on X11. A session's
end now stops what it started that left its process group, and a11y lint and contrast fail, rather than pass, when a program doesn't
answer the accessibility bus: see the changes to check.
Changes to check when upgrading
- A session's end stops what it started that left its process
group, assetsiddoes (a daemon, a test runner's helper, a
browser's own launcher), as a session killed outright already did.
That coversshotbox run,exec,shoot,stop, MCP's
stop_session, Python'sSessionand each run ofshotbox mutate.
It also stops a lasting session started from inside it (shotbox startwithinshotbox run). Such a program kept running after the
session ended. A recording's keeper finishing its file is spared, and
so is a command put in the session from outside with--session. - A program that doesn't answer the accessibility bus (busy, or hung)
fails a check rather than passing it.a11y lintanda11y contrastwithout--windowfail with status 2,not all checked
(Python's raise, MCP's say so as an error), rather than finding
nothing wrong. Everya11yandwait nodecommand that gives up adds
; program 1234 didn't answer: busy, or hungto its error, which
alone read as a window, node or focus not being there. Each gives
such a program its--timeout(15 s at most) to answer, not
libatspi's 15 s:treeandlinttook 16 s with--timeout 2, and
witha11y's default--timeoutof 10, a program slower than that to
answer now counts as not answering.
Added
shotbox wait node RE --state STATE(enabled, focused, checked,
expanded, any the tree says, or not-STATE),--value V(a field's
text, a progress bar's percent) and--count Nwait for a node in
that state, with that value, or that many of them;shotbox wait gone REuntil no showing node matches: a spinner, a dialog after OK. Each
says what it saw last when it gives up. Python'swait_node(states=, value=, count=)andwait_gone, MCP'swait_fornode and gone.--ignore-node NAMEleaves out a node by its name, where there's a
tree:compare's from the tree beside either picture,wait matchand
wait change's from the live one as the wait starts; a clock or a
spinner needn't be given as coordinates that drift with the layout.
Python'swait_matchandwait_changetakeignore_node, MCP's
wait_forignore_node.wait match --click [X,Y]clicks where the picture matched, at its
middle or at X,Y in it, for a canvas or a game with no tree to click
by; Python'swait_match(click=), MCP'swait_formatch withclick.shotbox capture --tree(with--a11y) andshoot --treewrite the
accessible tree beside the picture,OUT.json, its bounds in the
picture's pixels;compare, given pictures with one, says what each
part that differs is in, on--diff's page too:412 pixels, in 80x24 at 12,40: button "Save", and a change in text GTK 4 draws partly
outside its widget is named by that widget, not the panel round it.
--updatetakes the tree with the picture, and removes the
reference's own if the new picture has none. Python'scapturetakes
tree=True.- A recipe, in docs/recipes.md: a web app in Chrome, served
once in a lasting session and pictured as many times as wanted, with
the flags it needs and why, and what it doesn't. shotbox compare REFDIR NEWDIR --updatemakes each new picture that
fails the reference, and adds each with no reference yet, leaving those
that pass untouched: a reviewed change is one command.--diff DIR
also writesindex.html, each failure's reference, new picture and
diff side by side, with copies of both sides, so the directory alone is
a CI artifact.shotbox wait stable --crop Gwatches just that part, and--ignore G(repeatable) leaves out a box, so a page with a clock or a live
chart can hold still round it;shoot --wait stablewatches its
--crop, and takes--ignoretoo. Python'swait_stableand MCP's
wait_forstable takecropandignore.shotbox shoot --wait node:REand--wait announce:REwait, in
order with the other waits, for a showing node with RE in its name or
text, or for something said to a screen reader, aswait nodeand
wait announcedo: a one-shot picture of a page once a line says
it's drawn. Either gives the session--a11y.shotbox a11y diff BEFORE [AFTER]says what changed from a tree
saved witha11y tree --jsonto another, or to the live one, as
MCP's actions withchangessay it: status 1 if anything did, 0 if
nothing, 2 if it couldn't tell.shotbox wait idle [--window RE]waits until each program with a
window shown has answered just now, three times in a row: free on its
main thread, whichwait stablecan't tell, as a busy program holds
as still as an idle one. It asks by X11's_NET_WM_PING, and with
--a11yGTK programs on the accessibility bus too, on Wayland as well;
a browser's page, and a window nothing can ask, are said on stderr not
to have been checked. Python hasScreen.wait_idle(), and MCP's
wait_foridle.shotbox mutateruns a browser test against small changes to the page
it tests, each run in a session of its own, to find a test that fails
on a change that shouldn't matter (BRITTLE: the page at another size,
a button moved 40 px) or passes on one that should (BLIND: a button
hidden or dead, a total a cent off). The test opts in withawait mutated(context)from the npm package; each change to a target is
proven by Playwright looking for it as the test would, and a run whose
change can't be proven is UNAPPLIED, with why. Python has
shotbox.mutate().
Fixed
record --soundon X11 drifted from the sound on a busy machine, a
tone up to a tenth of a second or more from its flash, and by
different amounts within one file: ffmpeg's x11grab took the frames.
shotbox takes them itself now, straight from the X server, with the
pointer drawn on for--pointer, and hands them to ffmpeg raw, as
wf-recorder's are on Wayland.record --soundon Wayland lost the first second or so of sound:
ffmpeg opened the sound only once it had looked over the first frames.- With
--usage, a recording still finishing its file as its session
ended was cut off, leaving an empty or unfinished file and no error:
the session's slice was stopped with the recording's keeper in it.
The keeper, and what it started, now go to a scope of their own
first, and finish as they do without--usage. - With
XDG_RUNTIME_DIRset to a directory that isn't there, as after a
logout, or one you can't make a directory in, as undersu,start,
stopandsessionsended in a traceback. They now say so:
XDG_RUNTIME_DIR is /run/user/1000, which isn't a directory: unset it, or point it at your runtime directory, orcan't make /run/user/1000/shotbox: Permission denied. a11y clickcouldn't click a Firefox check box inside its<label>
by name, with the pointer or--action: Firefox puts a label of the
same name round it, and each ruled the other out. Nor a Chromium field
named by a<label for>, whose text in the label matched too. The
field is clicked.- The focus renamed just as its window came up ("Check" to "Checked")
was sometimes not announced under load, sowait announcefor it
gave up: the recorder looked at the focus again after the rename but
before its event, and took the new name for the one it already had.
A focus first looked at that late can still be missed. - With
--failedand without--a11y,wait announceand Python's
wait_focus, giving up, said the screen twice (the screen then: F; the screen then: F) and took the--failedpicture twice. Once now. - MCP's actions with
changes, andshotbox.a11y.changes(), said
nothing of a program that didn't answer as the tree was read before,
only after: what changed in its windows was left out unsaid, and
could read as "Nothing changed". It's said either way.
0.10.0
Contrast measures web pages' text run by run, in Firefox as in
Chromium, on lines that say where in the DOM each is. It pictures the
screen six times, so text that fades or blinks is said with its range;
it fails GTK 4 text that matches in two places and fails in both, and,
with --window, faint GTK 4 text it can't find where nothing there
could pass; and it says what's behind a modal dialog apart. A click by
name waits until its node is ready, and --action reaches one a click
can't; on Wayland the session holds one pointer, as on X11. wait node
waits on the accessible tree, and pictures match with boxes left out
and within a margin. tabwalk --document and a Firefox recipe help
with web apps; MCP gains zoom, numbered labels and tree changes; a
seeded home's GSettings are kept. Bounds given a window are now the
window's, in MCP and Python: see the changes to check.
Changes to check when upgrading
a11y contrastpictures the screen six times, 0.37 s apart, and
says text whose ratio changes across what it needs as changing, with
its range:PATH: text changing: 2.50:1 to 11.24:1 (at least 4.5),
a note, which fails only with--strict. A fading or blinking prompt
passed or failed by when it was pictured. Changing text that fails at
its best fails,text 3.10:1 at best, changing from 1.40:1, …, and
that passes at its worst passes,text passes at 5.20:1 at worst, changing to 9.80:1, …; a control or image that changes is judged the
same way.--json, Python'scontrast()and MCP'scheck_contrast
give each itsrange, and the note itswhy. The pictures are taken
while measuring goes on, of only what's measured, so a check with
anything to measure takes at least 1.85 s: up to that much more where
measuring took less (a still GTK 3 window, 2.3 s, not 0.5; about 1 s
more on a Chromium page; nothing on a GTK 4 window, which takes
longer, but for its text not found that changed, looked for again in
each picture: 10 to 14 s more for a label first caught blinked off).- On Wayland the pointer stays where a command leaves it, as on X11,
hovering there, until a command moves it;parkand--parkmove it
to the bottom-right corner, where it starts. Each command made a
pointer of its own, which went when it ended, and the hover with it.
The pointer is drawn as nothing, in pictures and recordings: the
session gives sway and its programs a cursor theme whose every cursor
is clear (XCURSOR_THEME, GTK'sgtk-cursor-theme-nameand GSettings'
cursor-theme), since sway 1.9 (Ubuntu 24.04) puts the cursor in a
picture. a11y lintanda11y contrastlines on a web page's node end in
where it is in the page's DOM, the element's tag and id, its text
and the nearest id round it, as Chromium and Firefox give them:
PATH: text 2.84:1, #999999 on #ffffff (at least 4.5) <- p "Faint words" in div#intro. A path alone,.../document web[3]/section[0]/section[2]/static[0], was hard to find in the
source.--json, Python'sa11y_lint()andcontrast()give each
such nodedom(with its element'sclasstoo) andtext, and MCP's
check_accessible_namesandcheck_contrastlines end the same way.
--ignorematches the ending too:--ignore 'div#intro'. A script
that reads the lines asPATH: WHATgets the end in WHAT.a11y tabwalksays a web page's controls Tab never reaches in
Chromium, so--reach-allcan fail where it passed: Tab lands on
the page's document coming in from the toolbar, and that counted as
reaching all that's in it, so a button withtabindex="-1"passed.
Leave out what's meant to be off the Tab order with--ignore.- A click or scroll by name waits until the node is placed, enabled,
still and uncovered, and fails, saying which it isn't, when its
timeout runs out:gave up after 10s: button "Save" [focusable, disabled] isn't enabled,... has no position yet,... is off-screen,... is outside its window(the next three failed at once before,
the first as "can't tell where", which a browser's node just added
was),... is moving,... is covered by label "Cover"(in a GTK 4
window),... is covered by window "Dialog"(a dialog of its window,
on X11). That'sa11y clickanda11y scroll, Python'sclick_node
andscroll_node, and MCP'sclick_elementandscroll_wheel. A
click on a disabled button passed, doing nothing, and one under a
dialog or mid-move went elsewhere; to click anyway, as a check that a
disabled control does nothing, pass--force(force=True, MCP's
force), which still waits for it to be placed. A ready node costs
the click about 50 ms, to see it's still. Screen.wait_matchreturns aMatch(seconds, x, y), not the
seconds as a float: x, y is where the picture matched, the crop's
corner (0, 0 without one), or wheremarginfound it. A caller that
used the float takes.seconds. MCP'swait_formatch says where
too:It matches ref.png, after 0.412 s, at 103,58.a11y contrast --windowfails GTK 4's text it doesn't find, where
nothing there could pass:text not found, but nothing where it can be has the contrast it needs: the best anything there reaches is 1.65:1 (at least 4.5), with itswhy. It was a
note, which doesn't fail, however faint, so a run that passed can
fail now. Stock libadwaita has some: an emptyAdwEntryRow's title
(1.98:1), Files' preferences group description (3.21:1), and Settings
› Sound's "No Output Devices" (3.23:1). Leave them out with
--ignoreif they're not yours. Text dimmed behind an open dialog
doesn't fail: see the next.a11y contrastsays what's behind a modal dialog apart, and
doesn't fail it, with--stricteither:PATH: text behind a modal dialog "Settings", after what wasn't measured, and underbehindin
--json,Screen.contrast()and MCP'scheck_contrast. That's
what's outside an aria-modal dialog in its page (one on the screen),
a modal window's program's other windows, covered or not, and what's
under a libadwaita dialog in its window, which is always modal; not
what's under a modal popover. It was measured where the dialog dims
or covers it, and failed (an aria-modal dialog's backdrop took a
#999paragraph to 2.14:1) or passed on what covered it (a
libadwaita dialog's own text), so a run that failed can pass now,
and its lines are new on stdout. Check what's behind with the dialog
closed.- MCP's
take_screenshotsays where a point on a picture is for
click_elementas a sum,100 + x/0.8122, 50 + y/0.812, in place
of "divide an x,y on it by that, then add the crop's x,y": a client
that reads the old sentence needs the new one. Itscrop, in a burst
too, isWIDTHxHEIGHT+X+Yonly, asrecord_video's andwait_for's
are, not any ImageMagick geometry (100x100,50%x50%+0+0). - MCP's bounds, given a window, are the window's: from its top left
corner, asclick_element's x,y,cropandzoomwith it are, in
get_accessible_tree,check_accessible_names,check_contrastand
a click or scroll by name, and the nodes it lists when more than one
matches. They were the screen's, so a node's bounds given to a click
in the window were off by where the window is. An agent that added
the window's place back no longer should. - The Python API's bounds, given a window, are the window's, as MCP's
are: from its top left corner, asclick(x, y, window=)'s x,y,
scroll's andcapture'scropwith it are, ina11y_tree,find,
focused,wait_focus,a11y_lint,contrast, the nodeclick_node
andscroll_nodegive back, and the nodes they list when more than one
matches. They were the screen's, so a node's bounds given to a click
in the window were off by where the window is. Without a window
they're the screen's, asa11y --json's still are. - A seeded home's GSettings keyfile
(.config/glib-2.0/settings/keyfile) is read, with the session's
interface keys merged into it, assettings.ini's are; it no longer
takes--env GSETTINGS_BACKEND=keyfile. Since 0.9.0 the session's
replaced it, losing the settings of seeds that read it that way
before.
Added
- The Node package's
accessibleis a page's options too:
browser.newPage({ ...accessible })emulates
prefers-reduced-motion: reduce, so a page that honors it holds
still fora11y contrast.reducedMotion: 'no-preference'after it
checks the motion. a11y click NAME --action ACTIONdoes the node's AT-SPI action ACTION
(matched as it is, else in any case) in place of a pointer click, for
a node a click by name won't reach: a control given no size, which
never gets a position, or a drop-down's option, which isn't in the
tree, or isn't showing, until the drop-down is opened:a11y click Pick --action open, thena11y click Beta --action select. An
action's name and what it does are the toolkit's, so it's named;
without one, it fails listing the node's actions. It waits for the
node to be enabled, unless--force, but not placed, still or
uncovered, and says the action:button "Send" [focusable], by its action "press"; a node without it fails, listing the actions it has,
as does one whose program refuses it. A click by name that gives up on
a node with no position lists its actions:... has no position yet; its actions: click (--action NAME does one); on a match that isn't
showing, it says that, aswait nodedoes. Python's
click_node(action="NAME")and MCP'sclick_elementwithaction
have it too.wait node REwaits for a node showing in the accessible tree whose
name or text RE is found in,--window REfor one window's: a status
line sayin...
0.9.0
GTK 4.14 and later's text measured for contrast again, found where it's
drawn rather than where GTK says, at its color even when thin, in a
theme's fonts, and with a reason for what isn't; --passes and JSON
that keeps notes apart from failures. Sessions sealed from the
machine's system bus and printers by default. a11y speak for what
Orca says; announcements of alerts as they appear; one keyboard for a
Wayland session, so the focus is always there to read; blinking carets
and smoothly scrolled text no longer taken for flicker.
Changes to check when upgrading
a11y contrast --jsonkeeps what wasn't measured apart: it's an
object, not a list:problems, each node with itsproblem, and
notes, each with itsnoteandwhy, for what wasn't measured and
doesn't fail. They came in one list, told apart byfails, so a
script counting entries counted the notes as failures. Nofailskey
now.Screen.contrast()gives the same dict, its lists of nodes, and
has nonotesargument: the notes are always there.a11y lint --jsonis as it was.- A session has a stand-in system bus and no printers, unless
--machine-bus(Session(machine_bus=True)), which replaces
--system-bus(system_bus=True), now refused. Programs reached the
machine's system bus and CUPS, so GNOME Settings under--desktop
showed its Wi-Fi networks, and a print dialog its printers. Under
bubblewrap, the machine's bus and CUPS sockets aren't there either;
CUPS on localhost:631 still answers a program that asks there, unless
--net loopback. A program that needs the machine's services
(NetworkManager, logind, polkit, CUPS) wants--machine-bus: check
pictures taken with it before sharing them. MCP's sessions have the
stand-in.
Added
a11y contrast --passessays what passes too, after what fails, a line
each with its ratio and colors (PATH: text passes at 10.84:1, #2e2e32 on #e6e6e7 (at least 4.5)), and for GTK 4's text, how far from where
GTK says its text is it was found: to compare one version with
another, or check where text was found, without setting each ratio to
30.--jsonlists
them aspasses, each with itscontrast;Screen.contrast(passes=True)
and MCP'scheck_contrastwithpasseshave them too.shotbox a11y speak [--keys KEY...]: what a screen reader says, from
a real Orca started in the session for it, with a home of its own
there and no speech or braille output. It presses Tab until the
focus comes round (stopping before a text view, which would take it as
typing), or the keys given, and prints what Orca said as it started,
then a line a press: the node with the focus, then what Orca said
(button "Save" [focusable, focused] says "Save button"), orsays nothing.Screen.speak()
in Python, and MCP'sget_speech. Needs Orca, whichshotbox doctor
lists; checked with Orca 46.1, 48.0, 50.3 and 51.1. The Action's
speak: trueinstalls Orca, and whata11y: truedoes.- Announcements (
wait announce,announcements, MCP's
list_announcementsandwait_forannouncement) take in what Orca
says as it appears, and not only GTK 4's announcements and live
regions: an alert, notification or info bar shown, or first given the
focus, with its name and its labels' (assertive: Warning Delete notes.md? It can't be undone.), as GTK 3's message dialogs and
GtkInfoBar, Qt'sQMessageBoxand libadwaita'sAdwAlertDialog
(from libadwaita 1.6) are; and the focus renamed (polite: Copied), but not a name much
like the last (a countdown), as Orca leaves out. A GTK 3, Qt or
libadwaita program that says an error this way is heard.
Changed
- On Wayland, a session has one keyboard, made as it starts and shared
by every command until it stops, not one a command, so programs
always have one, as on a desktop and on X11: GTK 4 and GTK 3 say
where the focus is between commands (the tree, MCP'swait_for
focus), libadwaita's alert dialog is heard as the focus goes in, and
a field with the focus draws its caret in pictures.key --down
works there too, and a key a hold killed outright left down stays
down until--up, as on X11;click --downis still refused.a11y tabwalkanda11y speakno longer press Shift first. shotbox a11y tabwalk(andScreen.tab_walk(), MCP's
check_focus_order) reads the tree after each press only as far as
the node with the focus, not all of it: 50 presses through a GTK 4
list of 300 rows take 5 seconds, not 90, and through a GTK 3 one of
400, a second, not 11. Every read of the tree asks each node for its
role once, not three times, and one that isn't showing for nothing
more than its states:a11y tree,lintandcontrastmake a
quarter fewer calls on the accessibility bus, and under half as many
in a list mostly scrolled out of view. What they find is as before.a11y contrastmeasures GTK 4 before 4.14 (Debian 12's 4.8, Ubuntu
22.04's 4.6), which says where a widget's content is, inside its
border and padding, and says it right; not its edges (--edges),
which are outside that, nor what's in a popover, or on a stack's page
no longer shown, a line each. Such a window passed with
nothing measured before (nothing in it was walked); now what fails,
fails. docs/accessibility.md, "GTK 4's positions".a11y contrastmeasures the text of GTK 4.14 and later, which it left
out, where it finds it: it draws the node's name in the font GTK says
and looks for it where GTK's wrong box can be (a stack's place, which
GTK counts twice, taken off; up to 4.14.5 inside the box, from 4.14.6
as far right and down as its containers' padding and borders can
have moved it), and takes it only where it matches and nothing else
there does, nor another node's text (or placeholder), drawn there to
compare, and from 4.14.6 where GTK lays it out as wide as found
(text not all ASCII isn't measured there). A
failure says how far from where GTK says its text is it was found,
which is how far GTK has it wrong, or where GTK doesn't say, from its
box's corner (gnome-calculator's "=", 3.76:1, 14,19 px). Text not
found (over two lines, a name that isn't
drawn), not looked for within 30 seconds, images, icons and edges are
still not measured, a line each.
Such a window exited 2, nothing measured, andScreen.contrast()and
MCP'scheck_contrastfailed on it; now they pass or fail on its
text. Needs pycairo and PangoCairo (python3-gi-cairo,
gir1.2-pango-1.0), which the Action'sa11y: trueinstalls and
shotbox doctorchecks for.
docs/accessibility.md, "GTK 4's positions".a11y contrastfinds more of GTK 4.14 and later's text, and faster:
each size and hinting of a node's name is matched at every place in
its room at once, exactly, not first by a few of its pixels at each,
and rooms up to 600 by 400 pixels (a sidebar's) are looked in, not 600
by 200. Under a theme with a grid drawn behind everything and a large
heading, a sampler had 29 of 36 labels measured in 21 seconds, not 5
in 34 with the rest out of time; gnome-calculator takes 1.2 seconds,
not 2. A match elsewhere in the room for the best-matching shape,
which keeps text from being taken, is always seen, and comparing
other nodes' text keeps to the 30 seconds too.a11y contrastdoesn't look for GTK 4.14 and later's text all outside
its window or a scrolled window it's in (a row scrolled out of view,
which GTK calls showing; not in a popover, which can be drawn outside
both), and says so:text not measured: not shown (scrolled out of view, or hidden), not that it wasn't found. Nor, from 4.22.3, a
single letter, which GTK gives no width. The line for text not looked
for says why (a single letter, over more than one line, not all
ASCII), as its why does. On 4.16 and later, text drawn a quarter of a
pixel off the rows (a line set by its baseline) is found: a theme's
sampler's description, which matched 0.967, matches 0.995.- Chromium's pages (Google Chrome's, Playwright's Chromium's) are read
in an--a11ysession without--force-renderer-accessibility:
shotbox a11yasks a browser window for its attributes, as Orca does,
which has Chromium build the page's tree for the rest of the
browser's life, and waits for it, up to--timeoutfor a big page
(one whose tree isn't built by then is noted, or fails with
--window). A window with no page (a crashed
tab, or a browser started with
--disable-platform-accessibility-integration) is noted on stderr;
the one--windowasks for, with nothing in its tree, fails saying
so and what turns it on, froma11y treeand the other commands and
MCP'sget_accessible_tree, rather than give the window alone.
Electron's windows are asked too, but not waited for. The Node
package'saccessibleno longer has the switch. - Playwright finds its browsers in a session without
PLAYWRIGHT_BROWSERS_PATHgiven: the session sets it to yours
(~/.cache/ms-playwright, or what it is outside), not the scratch
home's empty cache; not with--identity, whose sessions don't have
your name. docs/sealed.md.
Fixed
a11y contrastcrashed on GTK 4.14 and later text whose CSS weight
isn't one Pango names (font-weight: 450), measuring nothing. Its
text is looked for at the nearest named weight.a11y contrastmeasured thin GTK 4 text lighter than its color, and
failed it: a stem thinner than a pixel covers none whole, so a "1" in
#b0b0b0on#353535(5.66:1) measured 3.94:1, and DejaVu Sans's
"l" at 9px is as thin on GTK 4.14 as on 4.24. Text found where it's
drawn, whose glyphs cover fewer than three pixels whole, is now
measured at the color a pixel they covered whole would be, from how
much of each they cover, where the shape found accounts for every
pixel (no shadow, a size it's drawn at); other text, as be...
0.8.0
MCP tools named for what they do (verb_noun: renamed, see below), with
descriptions that say when to use each; MCP revisions 2025-11-25 and
2026-07-28; pictures sized for the model looking at them, bounded
answers, and a program's text marked as its own. Waiting for a spawned
program's window by its name, window waits that say what's there, and
sessions killed outright cleaned up, with sessions --stale and
--clean for what was left before.
Changes to check when upgrading
-
An unknown MCP tool is a protocol error: a tools/call of a tool
shotbox doesn't have is a JSON-RPC error, -32602, as the spec has it,
not anisErroranswer. -
MCP's tools are named verb_noun, so an agent can guess what each
does from its name. A client's permission list that names them
(Claude Code'smcp__shotbox__click, say) needs the new names; a call
by an old one is refused, naming the new one. Their arguments and
answers are as before.Was Now spawnspawn_programprogram_outputget_program_outputaccessible_treeget_accessible_treescreenshottake_screenshotrecordrecord_videoclickclick_elementscrollscroll_wheelgamepaduse_gamepadlintcheck_accessible_namescontrastcheck_contrasttab_walkcheck_focus_orderannouncementslist_announcementssoundrecord_audiostart_session,stop_session,press_keys,type_textand
wait_forkeep theirs. The commands (shotbox click,a11y lint...)
are as they were. -
MCP marks a program's text:
get_program_output's lines, the line
wait_forline andrecord_video'suntil_lineend on, and
list_announcementscome a line each after|, and the instructions
say such lines, and quoted names and titles, are the programs', not
the person's. A client reading
wait_forline's answer as the bare line needs to take the|off.
Added
shotbox --version: which shotbox this is.shotbox sessions --stalelists scratch dirs whose sessions have gone,
with each one's size and the command, pid and time that made it, which
ashotbox.jsonin it now says, and lasting sessions unused for a day;
shotbox sessions --cleanremoves those dirs, with what's left running
of their sessions (an older shotbox's, which say nothing, only with
--unmarked).shotbox doctorsays how many there are, and their
size.- MCP's
wait_fora window takes the name a program was spawned by, and
waits for a window of that program's, whatever its title, on X11 and
Wayland, over one titled the name (^name$still waits for that),
which is taken if the program has none of its own, as a daemon's
window is;spawn_program's answer says so. - MCP revisions 2025-11-25 and 2026-07-28, with
server/discover. - MCP's
take_screenshotandrecord_videosay a picture's size, and
shrink one past what Claude sees unscaled (a 1568 px long edge or 1568
patches of 28 px: 1920x1080 to 1456x819), saying the scale, so an x,y
read off it can be mapped back forclick_element. A burst's and a
clip's sheet lays its pictures out as many to a row as shrinks them
least, two 1920x1080 screens to a row, not always four. - MCP's
start_session, without the system's PyGObject in sight (plain
pipx install, uvx), starts anyway and says which tools won't work
and how to fix it, as it says what it couldn't seal.
Changed
- A session killed outright (SIGKILL) no longer leaves its scratch dir
in$TMPDIR, nor its X server, bus and programs running, nor those of
a session started in it: a watchdog process of its own stops them,
letting a recording finish its file
within 5 seconds, and removes the dir (not one kept
with--keep), asshotbox mcp's does for its sessions, which it now
stops with SIGTERM too, before SIGKILL 5 seconds on. - MCP's answers are bounded:
start_session's screen is at most
7680x4320;get_accessible_tree, the checks andlist_announcements
stop at 500 lines, saying how many more and to narrow it with window;
and a program's line, or a tree line, stops at 500 characters. - docs/mcp.md says that
wait_forline reads any text file under the
allow file's directory, so it belongs in its own or the project's, not
your home, and that allowing a terminal, browser or file manager hands
the agent arbitrary commands throughtype_text. - A command that needs PyGObject, run from uvx or
uv tool install,
whose environments can't see the system's, says to install with
pipx install --system-site-packages shotbox, not to make the
virtualenv with--system-site-packages, which uv's tools can't. - MCP's tool descriptions say when to use each tool and which to use
instead, what its answer holds, and its defaults:get_program_output
reads a program that has exited,list_announcementskeeps what was
said, oldest first, andcheck_contrastraises the window it measures. - MCP's
check_focus_order, when nothing has the focus to start from,
says to click the window withclick_element, not to run
shotbox focus, which an agent can't. - A window wait that gives up (
wait window,Screen.wait_window, a
--windowthat waits, MCP'swait_for) lists the windows there are,
the nearest first, with their app ids on Wayland, as a click by name
lists the nearest names; so does a--windowwith no such window.
0.7.1
MCP tool definitions an agent or a directory can read in full: every
parameter described with its limits, the allowed programs listed, and
hints as the spec reads them.
Changed
- MCP's tools/list describes every parameter, with the limits the server
holds them to (minimum,maximum);spawn's andprogram_output's
programis one of the--allowfile's names. The tools give the
hints clients look for:wait_foris read-only,stop_sessionalone
destructive, and onlyspawnand the tools that drive a program
reach an open world. - MCP's
wait_fortakes atimeoutof 0 as 0 seconds, not 10, and
start_sessionrefuses atext_scaleunder 0.1.
Fixed
flickerdoesn't take text scrolling smoothly past, lines apart, for
flicker: the background between lines comes back, but only every
other look, and a look drawn plain between frames of a scene has to be
plain each time it comes back.flicker's docs say what it can't tell from flicker: text scrolled
by fractions of a pixel (GTK 4's or kinetic scrolling, a browser at a
fractional scale), whose rows repeat, as it's never the same picture
shifted.
0.7.0
Accessibility checks you can gate on: lint and tabwalk agree with Orca
and don't edit what they walk, and contrast says what it can't measure
(GTK 4) rather than guess. Scrolling; names clicked as they are; an MCP
server that tells agents how to use it, answers while it works and
stops what's cancelled; and --clock that works on Ubuntu 24.04.
Changes to check when upgrading
- GTK 4 apps' contrast isn't checked: GTK 4 says where its widgets
are without their containers' padding and borders, and a stack's
child with its place in the stack counted twice, so what was measured
was often a button's fill or a neighbor's text.a11y contrastsays
each GTK 4 node is "not measured", and why on stderr; with nothing
measured at all, as in a GTK 4 window, it exits 2, not 0 (or 1), and
Screen.contrast()raises. GTK 3, Qt and browsers are measured as
before.SHOTBOX_TRUST_GTK4_BOUNDS=1measures GTK 4 at its bounds, for
a GTK whose bounds you've checked. docs/accessibility.md, "GTK 4's
positions". - A name is taken as it is before as a regex:
a11y click, MCP's
click,find()andclick_node()pick the nodes named exactly NAME
over those NAME matches as a regex, among those showing when they
look, so a calculator's.is its button rather than every
one-character one, and(Draft)is itself. A window's title, a role
and a focus wait match a name as it is too, or as a regex:Foo (1)
now finds the window titled that. - MCP's
lintandcontrastanswer with each node asaccessible_tree
has it, with its bounds, rather than its path. An--ignore(ignore)
regex is found in either.
Added
a11y contrastfinds large text, 18pt or 14pt bold all through, by
its font as GTK and Chromium say it, and holds it to WCAG's 3:1 rather
than 4.5:1;--large RATIOsets that ratio.Screen.contrast(large=..., notes=True), the second for what wasn't measured; MCP'scontrast
lists that after the failures, and why.- Scrolling:
shotbox scroll X Y DY [DX] [--window RE]turns the
mouse wheel at a point, DY clicks down and DX right (negative, up and
left), over XTEST's buttons 4 to 7 on X11 and the virtual pointer's
axis_discreteon Wayland;shotbox a11y scroll NAME DY [DX]over a
node found by its name, asa11y clickfinds it. A step inplay
scripts too;Screen.scrollandScreen.scroll_node, and MCP's
scroll, by a node's name or a point. - MCP's server instructions: as it connects,
shotbox mcptells the
agent how its tools go together; clients pass it on, and directories
show it. record --until-line RE FILE [--tail SECS]: a recording that ends on
an event, once a line matching RE is written to FILE, aswait line
waits for one, and SECS after it;--secondsis then the most it
lasts. It prints the line, or says it was stopped or ran its
--secondsfirst.Screen.record(until=(re, path), tail=...), whose
Recording says thelineand whether it wasstopped; MCP'srecord
takesuntil_line,fileandtail, its file in the--allowfile's
directory, aswait_for's line is.SHOTBOX_SANDBOX=offin the environment:--no-sandboxfor every
session, from Python and MCP too, and for one started inside a session,
for a Dockerfile or CI job to set once (falseand0too).shotbox doctorsays whether sessions have a sandbox, and why not.shotbox key shift: a modifier alone.a11y tabwalksays what Tab never reached (not reached by Tab): the
showing, sensitive controls and fields, and tables and lists whose
items take the focus; the last line counts them.--reach-allfails
on them; without it they don't fail the walk, since arrows or
shortcuts may reach them and a window's own buttons are out of the
Tab order.--ignore REleaves out those it's found in.
Screen.tab_walk(reach_all=True, ignore=...); MCP'stab_walksays
them too, and takesignore.
Changed
- A session that can't make a sandbox says so only where it would have
hidden something: not in a container with no sound, input, HID or USB
devices and nothing in/run/user/UIDorXDG_RUNTIME_DIR. Where it
says so, it says how to say it's meant.
Fixed
- shotbox's own commands inside a
--clocksession (wait,key --hold,record...) run on the real clock, rather than under the
session's libfaketime, which in 0.9.10 (Ubuntu 24.04's) makes the
time.sleepof Python 3.11 and later fail; the programs they start for the session, as
term's, keep the session's clock. a11y contrastleaves out a disabled control's label, as the control
is; says text too small to have a core of its color (a ".") is too
small to measure, and passes it; and takes text as the third pixel in
order of contrast, not the strongest color three pixels share, which
a thin stroke seldom draws.- A name, role, window title, announcement or
--ignorethat isn't a
regex (+) says so in a sentence, rather than with a traceback; a name
that isn't one is taken only as it is. a11y clickand MCP'sclick, when more than one node matches, list
them as tree lines with where they are, and how to narrow it, rather
than by their paths; when none does, the nearest names. MCP'sclick
takes atimeout.- MCP's
wait_forwith awhatthere isn't names the choices, rather
than asking for a value. shotbox mcpanswerspingat once during a long call, takes
notifications/cancelled, calling off the request's waits, click,
check or recording and sending no answer to it, calls off the call
running when its stdin closes, rather than finishing it, and sendsnotifications/progressevery 5 seconds to
a call with aprogressToken. docs/mcp.md says which calls run long,
and how to raise a client's timeout.a11y tabwalkno longer presses Tab in a text view, where it typed a
tab into the document: it tries ctrl+Tab, shift+Tab in Qt (GTK's
types a tab), Escape (not in a dialog, which it would close) and F6 to
leave it, and fails as a trap only if none does. One it begins in,
nothing leaving, or leaves backward, it leaves by giving the focus to
what's after it, so the rest is walked. It presses Shift first, as
GTK 4 on Wayland says where the focus is only after a key; with no
node focused even then, it presses nothing, and says so; focus gone
after a press is said as that, not as Tab leaving it where it was.
The docs no longer say ctrl+Tab always gets out: an editor's own
shortcuts can take it. MCP'stab_walkanswers intabwalk's lines
(round again after 6, where it saidRound again after 6.).a11y lintandtreetake a field's label as its name, as a screen
reader does: a GTK 3 field with a mnemonic label passes, and is
text "Name:"in the tree;a11y click Name:(andscroll) takes the
field, not its label. They leave out what isn't showing, as a
revealer's unrevealed bar (gedit's search bar), not only what's
hidden; and lint, what says it has no size (gedit's one document's
hidden tab). Thanks to @jay23606, whose #141 found the same fix.wait linewith a pattern that isn't a regex says so, rather than
with a traceback.record --sound --seconds Nwhose session's sound server stops first,
as a lasting session's does when it's stopped: the picture ends with
the sound, rather than running on for up to a second past it.--audiowith--clock: the sound server runs on the real clock, so
sound plays and records. libfaketime moved the timer it plays by (late
by the offset, or at once, as TIME was behind or ahead of now), and
nothing in the session could read it.recordwhose ffmpeg is still opening what it records 10 seconds in
kills it and fails saying so, rather than waiting on it for good,
Ctrl-C and all.
0.6.0
Sound, and video with it, on X11 and Wayland; flicker, latency and what a
session used, measured; a session's own clock, identity and network;
timed scripts of input; and checks that a picture comes out the same
twice. Pictures of GTK 4 programs change: see the first note below.
Changes to check when upgrading
- GTK 4 draws text with manual font rendering (GTK 4.16 and later),
so text is drawn at its color rather than fainter, by however much
automatic rendering draws it on that GTK release. Every GTK 4 picture
with text changes, its widths too, so layout can shift, anda11y contrastmeasures thin text higher: take committed GTK 4 pictures
again. A seeded home whose GTK 4settings.inisets
gtk-font-renderingkeeps its own. - Under bubblewrap, a program can't gain privileges, its supplementary
groups show asnogroup, and one killed by a signal exits with 128
plus the signal's number.--no-sandboxruns it as before. comparesays where pixels differ, after how many:N pixels differ, in WxH at X,Y, on each picture's line with directories too.- A session with a display wants ImageMagick, and one with X11 or
Xwaylandxwininfoandxprop(x11-utils), before it starts, rather
than at its first wait or picture. Screen.recordreturns a Recording, not the path: itspath, and
with sound what was heard. It stands in for the path where one is
opened (os.fspath) and instr(), but.suffixand the rest of a
Path's are on.path.
Changed
- Waits look faster.
wait stable,matchandchangetake their
pictures straight from the X server (a few milliseconds, whereimport
took tens) or from grim's raw output on Wayland, and compare them in
memory, with no PNG between;matchandchangelook every 10 ms. A
change is seen within about a frame, and the timechangesays is that
close. - Every program a session spawns runs under bubblewrap where it can,
not only with--gamepads:/dev/sndand/run/user/UIDare empty
for it, so it can't play through the machine's speakers or find the
desktop's Wayland, PulseAudio, PipeWire or bus sockets. Without
bubblewrap, the session says so and runs it as before. - A window asked for and not there is said with the titles, or app ids,
there are, and that it's matched in full. - The README is a front page, the reference moved to docs/commands.md,
sealed.md, gamepads.md, python.md and node.md, the a11y commands to
docs/accessibility.md, the Wayland example to docs/wayland.md, and the
tests to CONTRIBUTING.md.
Added
shotbox play SCRIPT: a script of timed input, played in a
session on one clock, with no command started for each step, so a demo
plays out the same way each time it's recorded. A line is a time
(seconds from the start, or+SECSafter the step before) and a
command for inside a session; one ending in&runs on in the
background, a recording say, and${NAME}is the session's variable.
Each line's options are checked before the first runs, and a failure
says its file and line. AlsoScreen.play, which takes the text, or a
Path to the file, andvarsfor${NAME}.wait line RE FILE: wait for a line matching RE to be written to
a file, and print it, for a script that follows a program's log of
what happens rather than sleeping by a clock the program may not
keep to. From the file's end when the wait starts (--from-startfor
the top); a file not there yet is waited for, and one rotated or
rewritten is followed. AlsoScreen.wait_line, and MCP'swait_for,
whose file is in the--allowfile's directory.shoot --twice(andterm --shoot --twice): the picture taken in
two fresh sessions, and a failure if they differ, beyond--fuzzand
--max, saying how many pixels and the box they're in, with
OUT-diff.pngbeside OUT. If they're alike, one is kept as OUT.--identity USER@HOST: who and where a session's programs are,
forrun,shoot,term,start,exec,Session(identity=...)
and MCP'sstart_session:hostname,uname -n,whoami,id -un,getpwuid(),ls -l's owner,$USER,$LOGNAMEand
$HOSTNAMEsay USER and HOST, so a picture doesn't show who took it.
It needs bubblewrap, and not--no-sandbox.--clock TIMEand--tz ZONE: every program a session runs sees
the wall clock start at TIME (ISO 8601, in ZONE or UTC) and run on, a
program spawned later reading the same clock as the first, so dates in
pictures don't change between shoots. Forrun,shoot,term,
start,exec,Session(clock=..., tz=...)and MCP's
start_session; by libfaketime, which doesn't reach static binaries or
Go programs.- What a command needs, checked before it starts anything. A tool
not on PATH,/dev/uinputit can't open for--gamepads, or PyGObject
with VTE (term) or AT-SPI (--a11y) that this Python can't load is
said at once in a line, with the package to install, or, from a
virtualenv that can't see the system's packages, how to make one that
can (for pipx's,pipx install --force --system-site-packages shotbox);Sessionand MCP'sstart_sessionsay the same.
shotbox doctorgoes through every check, a line each, and fails
only if what every session needs is missing. --usage [FILE]: what a session's programs used, together, and
apart, its own processes: peak memory and CPU time, from a systemd
slice of the session's where the user's systemd can make one, else
read from/proc; as JSON to FILE.Session(usage=True)and MCP's
start_sessiontake it too.shotbox latency move X1 Y1 X2 Y2 --watch G: how long what's
drawn takes to follow the pointer, a move at a time, as the median and
95th percentile;Screen.latencyfrom Python.shotbox flicker: a part of the screen that keeps switching
between two looks, or flashes another once, found from pictures taken
as fast as they come, a tile at a time, by its mean color and in time:
its box, how many changes, how often, and both colors; and how many
pictures it took and how many changed, on stdout whatever's found, a
rate too slow said. With--save DIR, a sheet of each one's pictures,
its box outlined and a tile round it magnified, each labeled with when
it showed.
Screen.flickerfrom Python.--net loopback: a network of the session's own, which every
program it runs shares and nothing else is on, forrun,shoot,
start,execand MCP'sstart_session: a server a test starts is
reached by the rest of the session, and from nowhere else.wait port
looks in it.shotbox listen IN.wav: a recording's peak and RMS level, samples
clipped, and gaps, where a program fell behind and left the sink
silent; it fails on any, or on silence.shotbox.sound.hearfrom
Python, and MCP'ssoundtool, which records a session's sound
betweenstartandstopand says the same of it.- A GitHub Action,
uses: mishan/shotbox@vX.Y.Z: installs shotbox
at that version and what it runs on an Ubuntu runner, with each
feature (a11y,wayland,audio,record,term,gamepads)
asked for by name, and lets bubblewrap make its sandbox. shotbox exec -- COMMAND: the session's seal with no display, for
a headless browser or a test suite: its own environment, home and bus,
under bubblewrap, and--audioif it's wanted.--no-sandbox, for a program that needs a setuid helper (sudo, an
AppImage), which can't run under bubblewrap.- What's sealed, in docs/sealed.md: what a session keeps from a program,
and what it doesn't. And, in the README, instructions to give an agent
so it runs graphical programs, browsers and sound in a session. wait match REF.pngandwait change, with--window,--crop,
--fuzzand--max: wait until the screen looks like a picture, or
looks different from when the wait started, and say how long it took.
AlsoScreen.wait_matchandwait_change, and MCP'swait_for,
whose picture to match is a file in the--allowfile's directory.shotbox record OUT.wav: what plays in a session with--audio,
48 kHz 16-bit stereo, bit for bit for what plays in that format, on X11
or Wayland, for--seconds(to the sample) or until stopped.- Recipes, in docs/recipes.md: checking a change in a GUI, visual
regression in CI, an accessibility audit, a game smoke test and an
agent over MCP. - How shotbox compares, in docs/comparison.md, with
linux-x11-harness, agent-workspace-linux, kwin-mcp, agent-desktop,
dogtail, openQA and others. shotbox record OUT.mp4in Wayland sessions, and .webm and .gif:
the same options, the same length for--seconds, and the same ways
to stop, the file finished however it stops. wf-recorder takes the
frames over wlroots' screencopy and ffmpeg encodes them, as on X11.
sway draws no pointer, so--pointeris refused there. The Action's
recordinstalls wf-recorder whenwaylandis on too.shotbox record OUT.mp4 --sound, and .webm: in a session with
--audio, what plays too, in the same file, AAC in an MP4, Opus in a
WebM, by one ffmpeg, the sound in step with the picture from its first
frame, within a frame or so. Both are--secondslong, and it stops
the waysrecorddoes, the file finished. Then it says whatlisten
would of the sound: its level, clipping and dropouts, bylisten's
--gap,--clippedand--quiet, failing for them with--strict.
AlsoScreen.record(..., sound=True), whose Recording has what
shotbox.sound.hearheard, and itsproblems().- MCP's
recordtool: a clip of the screen or a window, up to 60
seconds and 100 MB, saved ...
0.5.0
Holding keys and buttons, pictures as colorblind eyes see them, bursts, videos, and gamepads.
Install
pipx install --system-site-packages shotbox # the command and the Python API
npm install --save-dev shotbox # the Node helpers
The README lists the system tools shotbox drives.
Added
key --hold SECSandclick --hold SECSkeep keys or a button down that long, the chords pressed together and let go last first: for games, and anything that reacts to how long something's held. In the background (&), one is held while other input goes on.key --down/--upandclick --down/--upleave a key or button down between commands, on X11. A Wayland virtual keyboard's keys go up when its connection closes, so there--downis refused, saying to use--hold.shotbox vision IN OUT --as KIND, and--vision KINDoncaptureandshoot: a picture as someone with protanopia, deuteranopia or tritanopia sees it (Machado et al., 2009, at full severity), or in gray.shotbox record OUT: a video of the display or a window, as an MP4, WebM or GIF, for--secondsor until stopped (Ctrl-C,kill, its session's display going), and the file is finished however it stops, even killed outright.--crop,--fps,--pointerand--park; X11 sessions only for now, and it needs ffmpeg.capture --burst N, with--every SECSand--sheet FILE: several pictures, numbered and timed, and a sheet of them, for motion.--gamepads Nandshotbox pad: virtual Xbox 360 controllers, made with uinput, that only the session's programs see: each runs under bubblewrap with a/dev/inputof its own, and the machine's raw HID and USB devices masked. Worked withpad press,stick,triggerandneutral; what's left down stays down between commands. MCP:start_sessiontakesgamepads, and agamepadtool.term --shootpasses--audioon to the session it starts.- MCP:
press_keysandclicktakehold(seconds) andstate(down or up), which last across calls on Wayland too, since a session keeps its input connection;screenshottakesvision, andburstandeveryfor a burst as one sheet.
Fixed
- A
TMPDIRtoo long for a session's sockets is said at the start (Linux allows a socket path 107 bytes), rather than D-Bus failing to start or sway crashing.
The full list is in the changelog.
0.4.3
contrast on GTK 3 and on GTK 4 icons and fields.
Install
pipx install --system-site-packages shotbox # the command and the Python API
npm install --save-dev shotbox # the Node helpers
The README lists the system tools shotbox drives.
Fixed
- GTK 3's switch isn't measured as text. It calls a switch a toggle button, so its off mark was held to text's 4.5:1.
contrast --edgestakes a field's edge from what it fills all round: a text view in a frame, which has no edge of its own.- A GTK 4 icon button is held to graphics' 3:1, not text's 4.5:1. GTK 4 puts a label in a control that draws text, so one with none draws an icon. GTK 3's buttons keep their labels to themselves, and are still held to text's.
- A missing GObject typelib is said in a line, naming the packages, rather than with a traceback from the announcement recorder.
Changed
- A control with nothing drawn says how it measures with the pointer over it, as a tab's close button is drawn only then: still "text not seen", but now one can tell that from text that's never seen.
- Light GTK 3 text that fails says its color too. Light or small text covers no pixel whole, so it measures short of its color, by more on one font and renderer than another, and text drawn see-through looks the same: it's still measured as drawn, but where GTK 3 says the color, and what's drawn is a lighter shade of it, the failure says it.
The full list is in the changelog.
0.4.2
tabwalk --visible on GTK 4's offset nodes.
Install
pipx install --system-site-packages shotbox # the command and the Python API
npm install --save-dev shotbox # the Node helpers
The README lists the system tools shotbox drives.
Fixed
tabwalk --visiblefinds a ring drawn off its node. GTK 4 places a node short of the padding of what it's in, right of and below where it's drawn: a libadwaita tab 6 pixels, a header bar's buttons 7, a tab's close button 10. Only 4 pixels round a node's bounds were counted, and a tab whose top edge already had the ring's color, as a current tab's rail can, was reported as showing no focus. When too little changed at its bounds, the count is now taken at any offset up to 16 pixels each way where what changed looks like a focus shown: along three or four of the node's sides, and little just outside them. The ring the focus left isn't counted where the walk found it, so a sibling right beside it, a radio button or a toolbar icon with a thin ring, still is, and a sibling with no ring of its own isn't credited with its neighbor's.- No EGL warnings from GTK 4 programs on X11. They opened EGL though they draw with cairo, and Mesa warned on stderr that Xvfb has no DRI3. A session now sets
GDK_DISABLE=gl(GTK 4.16 and later); pictures are the same.
The full list is in the changelog.