[Bug]: toBeVisible() ignores screen reader invisibility (aria-hidden=true)

[mleonhard]

Version

1.52.0

Steps to reproduce

Download attached file and then:

% unzip ~/Downloads/playwright-aria-hidden.zip 
Archive:  playwright-aria-hidden.zip
  inflating: playwright-aria-hidden/Example.spec.ts  
  inflating: playwright-aria-hidden/package.json  
  inflating: playwright-aria-hidden/playwright.config.ts  
  inflating: playwright-aria-hidden/server.ts  
  inflating: playwright-aria-hidden/tsconfig.json  
% cd playwright-aria-hidden 
% npm i

added 99 packages, and audited 100 packages in 743ms

14 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
% npx playwright test

Running 1 test using 1 worker

  ✘  1 [firefox] › Example.spec.ts:3:5 › example (443ms)


  1) [firefox] › Example.spec.ts:3:5 › example

    Error: expect(received).resolves.toEqual(expected) // deep equality

    Expected: "false"
    Received: "true"

      5 |     const element1 = page.locator('#element1')
      6 |     await expect(element1).toBeVisible() // <-- Test should stop here.
    > 7 |     await expect(element1.getAttribute('aria-hidden')).resolves.toEqual('false')
        |                                                                 ^
      8 | })
      9 |
        at /Users/user/work/playwright-aria-hidden/Example.spec.ts:7:65

playwright-aria-hidden.zip

Expected behavior

I expect expect(element1).toBeVisible() to fail saying that the element is invisible to screen readers because of attribute aria-hidden="true".

Actual behavior

expect(element1).toBeVisible() succeeds.

Additional context

No response

Environment

System:
    OS: macOS 15.3.2
    CPU: (14) arm64 Apple M4 Pro
    Memory: 965.92 MB / 48.00 GB
  Binaries:
    Node: 22.14.0 - ~/.nvm/versions/node/v22.14.0/bin/node
    npm: 10.9.2 - ~/.nvm/versions/node/v22.14.0/bin/npm
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: 1.52.0 => 1.52.0

[dgozman]

@mleonhard Playwright has a specific definition of the element visibility that does not include aria-hidden. You can use anything aria-related, e.g. getByRole(), toHaveAccessibleName() or toMatchAriaSnapshot() - these take aria-hidden and other aria-defined visibility signals into account. Let me know whether this helps.

Note that we are not going to change our existing visibility definition, because that would be a large breaking change.

[mleonhard]

@dgozman Thanks for your quick reply!

I made a modal that starts out hidden (display:none style and aria-hidden=true attribute), then appears when the user clicks a button. Unfortunately, clicking the button removes display:none but leaves aria-hidden=true, so the modal does not appear for screen-reader users. I wrote a Playwright test that clicks the button and then calls toBeVisible() to check that the modal is visible. I need the test to flag the accessibility problem.

Does Playwright provide any way to check if an element matched by testID or locator is visible to screen readers?

• getByRole() does not support passing a data-testid attribute or matching on a locator. The page https://playwright.dev/docs/locators#locate-by-role suggests that we identify elements by text (using name) but this leads to brittle tests that break whenever text is reworded. At least it supports substring or regexp (docs).
• toHaveAccessibleName() also matches on text, which makes tests brittle. It supports regex, but the docs don't say if it does substring matching (docs).
• toMatchAriaSnapshot() matches on the full text, which is more brittle than substring or regexp.
• Using locator('#modal[aria-hidden="false"], #modal[aria-hidden=""]') is awkward and verbose.
• Using locator('[data-testid="modal"][aria-hidden="false"], [data-testid="modal"][aria-hidden=""]') is even more awkward and verbose.
• toBeVisible() could do this automatically and ergonomically

Does Playwright team ever make breaking changes? Maybe a slow rollout would be acceptable: add a global 'ariaVisibility' option defaulting to "warning", then a year later make it "error". This would give folks time to notice the warning and either opt out by setting the option to "ignore" or update their tests and hopefully find and fix some accessibility problems and set the option to "error" to prevent the class of accessibility regressions.

[dgozman]

@mleonhard It seems like you need some kind of accessibility tests for your page, because everything works in a standard scenario, but some things are not accessible. The tools for that are axe accessibility audits, aria snapshots and various toHaveAccessible...() assertions.

Does Playwright team ever make breaking changes?

I'd say no. Certainly not any big changes like this one.