Skip to content

Choose a tag to compare

@mishan mishan released this 23 Oct 15:45
· 156 commits to main since this release
v0.10.0
2ea4353

Contrast measures web pages' text run by run, in Firefox as in
Chromium, on lines that say where in the DOM each is. It pictures the
screen six times, so text that fades or blinks is said with its range;
it fails GTK 4 text that matches in two places and fails in both, and,
with --window, faint GTK 4 text it can't find where nothing there
could pass; and it says what's behind a modal dialog apart. A click by
name waits until its node is ready, and --action reaches one a click
can't; on Wayland the session holds one pointer, as on X11. wait node
waits on the accessible tree, and pictures match with boxes left out
and within a margin. tabwalk --document and a Firefox recipe help
with web apps; MCP gains zoom, numbered labels and tree changes; a
seeded home's GSettings are kept. Bounds given a window are now the
window's, in MCP and Python: see the changes to check.

Changes to check when upgrading

  • a11y contrast pictures the screen six times, 0.37 s apart, and
    says text whose ratio changes across what it needs as changing, with
    its range
    : PATH: text changing: 2.50:1 to 11.24:1 (at least 4.5),
    a note, which fails only with --strict. A fading or blinking prompt
    passed or failed by when it was pictured. Changing text that fails at
    its best fails, text 3.10:1 at best, changing from 1.40:1, …, and
    that passes at its worst passes, text passes at 5.20:1 at worst, changing to 9.80:1, …; a control or image that changes is judged the
    same way. --json, Python's contrast() and MCP's check_contrast
    give each its range, and the note its why. The pictures are taken
    while measuring goes on, of only what's measured, so a check with
    anything to measure takes at least 1.85 s: up to that much more where
    measuring took less (a still GTK 3 window, 2.3 s, not 0.5; about 1 s
    more on a Chromium page; nothing on a GTK 4 window, which takes
    longer, but for its text not found that changed, looked for again in
    each picture: 10 to 14 s more for a label first caught blinked off).
  • On Wayland the pointer stays where a command leaves it, as on X11,
    hovering there, until a command moves it; park and --park move it
    to the bottom-right corner, where it starts. Each command made a
    pointer of its own, which went when it ended, and the hover with it.
    The pointer is drawn as nothing, in pictures and recordings: the
    session gives sway and its programs a cursor theme whose every cursor
    is clear (XCURSOR_THEME, GTK's gtk-cursor-theme-name and GSettings'
    cursor-theme), since sway 1.9 (Ubuntu 24.04) puts the cursor in a
    picture.
  • a11y lint and a11y contrast lines on a web page's node end in
    where it is in the page's DOM
    , the element's tag and id, its text
    and the nearest id round it, as Chromium and Firefox give them:
    PATH: text 2.84:1, #999999 on #ffffff (at least 4.5) <- p "Faint words" in div#intro. A path alone, .../document web[3]/section[0]/section[2]/static[0], was hard to find in the
    source. --json, Python's a11y_lint() and contrast() give each
    such node dom (with its element's class too) and text, and MCP's
    check_accessible_names and check_contrast lines end the same way.
    --ignore matches the ending too: --ignore 'div#intro'. A script
    that reads the lines as PATH: WHAT gets the end in WHAT.
  • a11y tabwalk says a web page's controls Tab never reaches in
    Chromium
    , so --reach-all can fail where it passed: Tab lands on
    the page's document coming in from the toolbar, and that counted as
    reaching all that's in it, so a button with tabindex="-1" passed.
    Leave out what's meant to be off the Tab order with --ignore.
  • A click or scroll by name waits until the node is placed, enabled,
    still and uncovered
    , and fails, saying which it isn't, when its
    timeout runs out: gave up after 10s: button "Save" [focusable, disabled] isn't enabled, ... has no position yet, ... is off-screen, ... is outside its window (the next three failed at once before,
    the first as "can't tell where", which a browser's node just added
    was), ... is moving, ... is covered by label "Cover" (in a GTK 4
    window), ... is covered by window "Dialog" (a dialog of its window,
    on X11). That's a11y click and a11y scroll, Python's click_node
    and scroll_node, and MCP's click_element and scroll_wheel. A
    click on a disabled button passed, doing nothing, and one under a
    dialog or mid-move went elsewhere; to click anyway, as a check that a
    disabled control does nothing, pass --force (force=True, MCP's
    force), which still waits for it to be placed. A ready node costs
    the click about 50 ms, to see it's still.
  • Screen.wait_match returns a Match(seconds, x, y), not the
    seconds as a float: x, y is where the picture matched, the crop's
    corner (0, 0 without one), or where margin found it. A caller that
    used the float takes .seconds. MCP's wait_for match says where
    too: It matches ref.png, after 0.412 s, at 103,58.
  • a11y contrast --window fails GTK 4's text it doesn't find, where
    nothing there could pass
    : text not found, but nothing where it can be has the contrast it needs: the best anything there reaches is 1.65:1 (at least 4.5), with its why. It was a
    note, which doesn't fail, however faint, so a run that passed can
    fail now. Stock libadwaita has some: an empty AdwEntryRow's title
    (1.98:1), Files' preferences group description (3.21:1), and Settings
    › Sound's "No Output Devices" (3.23:1). Leave them out with
    --ignore if they're not yours. Text dimmed behind an open dialog
    doesn't fail: see the next.
  • a11y contrast says what's behind a modal dialog apart, and
    doesn't fail it
    , with --strict either: PATH: text behind a modal dialog "Settings", after what wasn't measured, and under behind in
    --json, Screen.contrast() and MCP's check_contrast. That's
    what's outside an aria-modal dialog in its page (one on the screen),
    a modal window's program's other windows, covered or not, and what's
    under a libadwaita dialog in its window, which is always modal; not
    what's under a modal popover. It was measured where the dialog dims
    or covers it, and failed (an aria-modal dialog's backdrop took a
    #999 paragraph to 2.14:1) or passed on what covered it (a
    libadwaita dialog's own text), so a run that failed can pass now,
    and its lines are new on stdout. Check what's behind with the dialog
    closed.
  • MCP's take_screenshot says where a point on a picture is for
    click_element as a sum, 100 + x/0.8122, 50 + y/0.812
    , in place
    of "divide an x,y on it by that, then add the crop's x,y": a client
    that reads the old sentence needs the new one. Its crop, in a burst
    too, is WIDTHxHEIGHT+X+Y only, as record_video's and wait_for's
    are, not any ImageMagick geometry (100x100, 50%x50%+0+0).
  • MCP's bounds, given a window, are the window's: from its top left
    corner, as click_element's x,y, crop and zoom with it are, in
    get_accessible_tree, check_accessible_names, check_contrast and
    a click or scroll by name, and the nodes it lists when more than one
    matches. They were the screen's, so a node's bounds given to a click
    in the window were off by where the window is. An agent that added
    the window's place back no longer should.
  • The Python API's bounds, given a window, are the window's, as MCP's
    are: from its top left corner, as click(x, y, window=)'s x,y,
    scroll's and capture's crop with it are, in a11y_tree, find,
    focused, wait_focus, a11y_lint, contrast, the node click_node
    and scroll_node give back, and the nodes they list when more than one
    matches. They were the screen's, so a node's bounds given to a click
    in the window were off by where the window is. Without a window
    they're the screen's, as a11y --json's still are.
  • A seeded home's GSettings keyfile
    (.config/glib-2.0/settings/keyfile) is read, with the session's
    interface keys merged into it
    , as settings.ini's are; it no longer
    takes --env GSETTINGS_BACKEND=keyfile. Since 0.9.0 the session's
    replaced it, losing the settings of seeds that read it that way
    before.

Added

  • The Node package's accessible is a page's options too:
    browser.newPage({ ...accessible }) emulates
    prefers-reduced-motion: reduce, so a page that honors it holds
    still for a11y contrast. reducedMotion: 'no-preference' after it
    checks the motion.
  • a11y click NAME --action ACTION does the node's AT-SPI action ACTION
    (matched as it is, else in any case) in place of a pointer click, for
    a node a click by name won't reach: a control given no size, which
    never gets a position, or a drop-down's option, which isn't in the
    tree, or isn't showing, until the drop-down is opened: a11y click Pick --action open, then a11y click Beta --action select. An
    action's name and what it does are the toolkit's, so it's named;
    without one, it fails listing the node's actions. It waits for the
    node to be enabled, unless --force, but not placed, still or
    uncovered, and says the action: button "Send" [focusable], by its action "press"; a node without it fails, listing the actions it has,
    as does one whose program refuses it. A click by name that gives up on
    a node with no position lists its actions: ... has no position yet; its actions: click (--action NAME does one); on a match that isn't
    showing, it says that, as wait node does. Python's
    click_node(action="NAME") and MCP's click_element with action
    have it too.
  • wait node RE waits for a node showing in the accessible tree whose
    name or text RE is found in, --window RE for one window's: a status
    line saying a page is drawn, where wait stable passes on a WebGL
    canvas still waiting on its server, in place of polling a11y tree | grep. Giving up, it says what came nearest: a match not showing,
    the nearest names and texts, or that nothing comes near.
    Screen.wait_node(re, window=None) and MCP's wait_for node have
    it too, and say the node's text where that matched, cut to 80
    characters round the match.
  • A recipe, in docs/recipes.md: a web app in Firefox, served on the
    session's own network, in a new profile each run, its window sized
    to the screen, pictured once the page says it's drawn.
  • a11y tabwalk --document lists only what's in a browser window's web
    page, and says only its controls Tab never reached: Tab still goes
    through the browser's own (the address bar, Back, the tab strip),
    which were listed as not reached on every page. A window with no web
    document is refused, and a walk that never gets into a page with
    controls fails, even without --reach-all: round again, never in the page.
    Screen.tab_walk(document=True) and MCP's check_focus_order with
    document have it too.
  • compare and wait match and change take --ignore WxH+X+Y
    (repeatable), as flicker does, to leave out a box that changes on
    its own, a clock or a cursor, as openQA's exclude areas and
    Playwright's mask do, rather than a --fuzz loose enough to hide
    real changes. compare's diff shows it magenta, and with directories
    NAME=WxH+X+Y is for one picture. Screen.wait_match(ignore=...),
    wait_change and MCP's wait_for have it too.
  • wait match --crop G --margin PX looks for the picture up to PX
    pixels from the crop's place each way, as openQA searches for a
    needle, and prints where it matched: 0.412s at 103,58. For a
    canvas or a game, where what's drawn moves a little from run to run
    and there's no tree to find it by. compare --margin PX compares the
    second picture where it matches the first best, but for a band PX
    wide round the edge, and says how far it moved, whether it passes or
    fails. Screen.wait_match takes margin; MCP's wait_for match
    takes margin (up to 32).
  • a11y contrast says on its line why GTK 4's text it looked for wasn't
    taken, and its why says what that means: (no match good enough: 0.44 at best), (it matches elsewhere too, in "unchecked"), (another node's text matches there too: "Row 10"), naming the node where it's
    its text, not its name, that matches, or (what matches is drawn wider or narrower than GTK lays it out). All four read "wasn't found
    there alone" before.
  • MCP's take_screenshot has zoom: a region's corners, [x0, y0, x1, y1] in click_element's x,y, much as Anthropic's computer-use tool
    takes a zoom's, shown as large as fits (up to 8 times) to read small
    text. Its answer, and a crop's or a shrunk picture's, says where a
    point on it is for click_element: 100 + x/4.356, 39 + y/4.355,
    rather than leaving the agent to undo a scale and a corner itself.
  • a11y contrast --strict fails on its notes, too, what it couldn't
    measure and text whose contrast changes: for
    CI that wants every node measured, or ignored with --ignore. Those
    lines are failures, under problems in --json, each with its why
    if it has one; Screen.contrast(strict=True) and MCP's
    check_contrast with strict have it too.
  • MCP's click_element, scroll_wheel, press_keys and type_text
    with changes say what they changed in the accessible tree once the
    screen holds still, and when it was read: nodes added, gone, renamed
    or in other states, a tree line each, up to 500, so an agent needn't
    read the whole tree again. shotbox.a11y.changes(before, after) says
    it of two a11y_tree()s.
  • a11y tree says a program that didn't answer (busy, or hung), on a
    line of its own, where its windows were left out without a word; with
    --json, and in Screen.a11y_tree(), a window has its program's
    pid and toolkit.
  • MCP's take_screenshot with labels draws each named node's box and
    a number on the picture, and answers with each number's tree line and
    box, scaled and cropped as the picture is and cut to what it's in, so
    what's seen can be clicked by name.

Fixed

  • On Wayland a click or scroll, or the start of a held click, sometimes
    didn't reach a busy program: between commands sway had no pointer,
    so programs had none, and one slow to take up the next command's
    missed what it sent first. The session's server holds the one
    pointer now, as it holds the keyboard, and every command points
    through it. A button pressed by a command killed outright goes up
    when it dies.
  • a11y contrast looked for every toolkit's label text where it's
    drawn, as it does GTK 4.14 and later's, whose boxes are wrong; GTK 3's,
    Qt's and Chromium's aren't, so their text is measured in them again.
    A faint GTK 3 label was said as not found, "GTK None
    reports where its widgets are wrong".
  • a11y tabwalk takes a web page's <textarea> as the browser does:
    Tab moves on from it. It was taken for a text view that keeps Tab
    and left with ctrl+Tab, which goes to the browser's next tab, Escape
    and F6, which in Chromium goes into the toolbar, where Tab then went
    round: the page's controls after it were said as not reached, failing
    --reach-all, and the browser was left there for the next walk. In a
    web page Tab is pressed first; the rest only if it keeps the focus,
    and where one of them leaves the page and Tab never comes back to
    it, the walk fails, left the page by F6, and Tab never came back to it, and presses Escape to take Chromium back to the page.
  • Firefox from Ubuntu's PPA, which GitHub's runners have, never joined
    an --a11y session's bus, so wait node and a11y gave up on its
    window: it asks whether a screen reader is running, as Chromium does,
    and takes GNOME_ACCESSIBILITY=1 for a yes, which the session now
    sets with ACCESSIBILITY_ENABLED=1.
  • a11y contrast measured nothing on a page in Firefox, which keeps a
    page's text in what it's in (a paragraph, a list item, a table cell,
    a link, the page itself), where Chromium gives each run of it a
    static node named by it: a page of faint text passed. A web node's
    own text, a Chromium static node's too, is now measured run by run,
    each run in one look and on one line, between the objects in it (a
    link, a check box) and emoji, which are drawn in their own colors:
    never by a whole box, which would take a dark link's color for the
    faint words round it, or the dark words round a faint link that
    wraps for the link's. A run the browser can't place, after up to a
    second's wait, is can't tell where it is. A canvas's fallback text
    isn't measured, in Chromium either, since it isn't what the canvas
    draws. And a web page with text, none of it measured, is a note,
    text not measured: none of the page's text was.
  • A read of the accessible tree (a11y tree, lint, a click by name
    and the rest) failed with Object does not exist at path … when a
    window closed while it was read. A window gone is left out, as a node
    gone already was, and so are its nodes when it goes while they're
    walked; one gone before its bounds are read leaves its nodes' bounds
    null, as a node gone then had.
  • compare failed on two pictures with a million or more pixels
    differing, unable to read ImageMagick's count, 1.04e+06; it counts
    them exactly now.
  • a11y contrast didn't take GTK 4's text where another node's shorter
    text inside it matched there too: "unchecked", under a "check" whose
    room took it in, was a note. A shorter text can't match all of a
    longer one, so where it's drawn apart too it doesn't count against it
    now; "check" itself, matching in both places, isn't taken unless it
    fails in both (see below).
  • MCP's take_screenshot said one scale for a thin picture, its
    width's, though each side is rounded on its own: a 10x4320 crop said
    0.4 and was 0.363 down. It says both when they differ. A crop with
    none of what's taken in it is refused, not shown as one pixel, or in
    a burst, as a sheet of nothing.
  • a11y contrast left faint GTK 4.14 and later text over a theme's
    grid unmeasured, which doesn't fail, while text of the same color
    elsewhere failed: the fainter the text, the more of what varies where
    it's drawn are the grid's lines (8% of it under #3a3446 text on
    #0f0d14), and the less it matched its shape. Where the best match
    falls short, it's worked out again there with the lines through the
    text's box taken off: those labels are found, and fail at 1.61:1.
  • a11y contrast didn't take GTK 4's text where another node holding
    the same text has it in its room: a combo row's selected item under
    an entry holding it, or a label over one, was a note however faint.
    Where no other node could have drawn it there, it's measured, and
    only a failure is reported; if it would pass, it's still a note. Text
    matching in two places that either can be its (a faint "check" over a
    faint "unchecked") is measured in both, and fails at the better if it
    fails in both: text 2.50:1 at best, …; it matches in two places and fails in both.
  • record on X11 went by a frame rate ffmpeg guessed from when the
    frames came, not --fps: on a busy machine a 30 frames a second MP4
    was 29.97 (2 seconds lasting 2.002), a WebM a frame or two short, and
    a GIF's frames timed wrong (1 second lasting 1.2). It's at --fps
    now, as on Wayland.
  • wait announce heard nothing of a headed Chromium's live regions
    until a11y tree or another command had read its tree, which is what
    has Chromium take a screen reader to be at work. The recorder asks
    each program that joins the session's bus for its attributes, as Orca
    does, so in an --a11y session a browser's accessibility is on from
    the start, which slows its pages a little.