[css-grid] Masonry layout in CSS Grid 3 has potential to cause accessibility problems with reading order.

[nattarnoff]

Assuming all authors code their masonry grid in a valid reading order, screen reader user who are non-visual will have no problem with it. But once we introduce visual screen reader users, low vision with no assistance, magnification software users, and users with cognitive problems, the layout begins to cause problems in focus order and reading order.

If we have
<div>
<item>The quick</item>
<item>brown fox</item>
<item>jumped</item>
<item>over</item>
<item>the lazy</item>
<item>dog</item>
</div>

The screen reader user will hear "the quick brown fox jumped over the lazy dog." When we add the CSS and visual layout, the reading order could be realistically be:

Image description: 6 blocks laid out using masonry reads, "The quick brown fox jumped dog over the lazy"

If these have active controls, the focus order would match the visual as well. I think using a simple sentence quickly demonstrates the damage and problems that could happen for users with cognitive issues, zoom users, and the dissonance a sighted screen reader user would encounter. This gets even more amplified if the author improperly uses tabindex set to a positive number.

I'm not clear on a solution. But I am clear that while the masonry may initially be targeted for photos and other visual pages, we have to expect it to be used by others for different things. If the individual items are standalone, like articles in a newspaper, then this isn't an issue, but we need to provide a way to control the focus/reading order to make sure the content doesn't become inaccessible.

[Loirooriol]

The screen reader user will hear "the quick brown fox jumped over the lazy dog." When we add the CSS and visual layout, the reading order could be realistically be

But speech order not matching visual order is already happening in Grid level 1. Can you clarify what's different with masonry?

If these have active controls, the focus order would match the visual as well.

Would it? From https://drafts.csswg.org/css-grid-1/#source-independence

Note: The reordering capabilities of grid layout intentionally affect only the visual rendering, leaving speech order and navigation based on the source order. This allows authors to manipulate the visual presentation while leaving the source order intact and optimized for non-CSS UAs and for linear models such as speech and sequential navigation.

[rachelandrew]

But speech order not matching visual order is already happening in Grid level 1. Can you clarify what's different with masonry?

at the moment in Grid, yes you can move things out of order, however (other than dense packing) auto-placement doesn't do that. So where you have moved things out of order you should know you have done it. The fact that people are building grid tools that move things out of order and don't warn the user is a different though related subject.

With masonry we give people a really compelling layout method which is likely to cause this problem without the author actively doing it. It also might be tricky to test as at one breakpoint the reordering might not be too bad, and at another awful.

This was brought up at the CSSWG face to face where masonry was first proposed. It does feel that we are adding more layout methods that create this reordering issue without trying to solve the issue in a way that would help this, but also other reordering problems.

[rachelandrew]

See also : https://rachelandrew.co.uk/archives/2019/06/04/grid-content-re-ordering-and-accessibility/ (predates this masonry spec).

[nattarnoff]

If you look again at my first sentence, you'll see my concern is not how a screen reader will relay the information.

Assuming all authors code their masonry grid in a valid reading order, screen reader user who are non-visual will have no problem with it

I think @rachelandrew nailed my biggest concern. As the layout adjusts to content, viewport, or any other factor, the visual reading flow as well as the tab order are going to create problems for other sets of disabled users.

• Neuro-divergent (cognitive) may get lost on where to focus their attention next
• Low vision, no AT may lose where their focus indicator has moved to, or may need to read content from a block that isn't proximate to where they are looking
• Low vision, screen reader will be hearing one thing while trying to locate it on the page because it doesn't follow the visual flow of the page
• Keyboard only - expect a linear flow of tabstops. If the tabstops follow code order, not visual order, they may get lost trying to track things

Using positive tabindex seems tempting here, but if item two goes from being on the right of item 1 to below it, while item 3 slides into where 2 was (these are pure hypothetical) the tab order now may make no sense. So we don't want to force that with HTML code or create a similar thing for reading order.

We need a method though that will ensure that users will be able to read things in their proper comprehension order and that tab order matches visual order. A thought here could be "item priority" which defaults to code order (or no priority), while other options allow you to specify any number of blocks to accept a value that then makes sure the tab order changes to match the visual.

[MatsPalmgren]

I slightly disagree with the premise of this issue. You really shouldn't use (default) masonry layout in the first place if the item placement matters to the semantic meaning of the content. So I think the original example doesn't make sense, for anyone, sighted or not. (There is however masonry-auto-flow: next that could be used for content where the item order matters, but then you sacrifice optimal packing.)

I agree with the later points though, that taking input both visually and from a screen reader simultaneously could be slightly disorienting since the order may not completely match (they are typically somewhat close for default masonry layout though). This is a general problem with Grid and other layout that can place boxes in arbitrary positions.

I don't think adding more HTML/CSS features and relying on authors to use those features is a tenable solution. For two major reasons: 1, authors rarely take the time to do that extra work, and 2, predicting the visual layout is very hard in responsive designs. Instead, I think it should be solved by browser implementations and AT. For example, a browser could provide the visual box order of a Grid container (masonry or regular) to an AT. This could be used by the AT to offer DOM order or visual order as an option to the user. Visual order might be preferred by users that consume both visual and screen reader information simultaneously.

Browsers need to have a concept of visual order anyway to implement selection in a way that makes sense. For example, selecting text with the mouse from an item in the top-left corner of a grid into its neighboring item on the right should really just select the text in those two items, not the entire DOM range in-between. (We don't implement that in Firefox yet, but we intend to.)

[frivoal]

As far as keyboard navigation is concerned, https://drafts.csswg.org/css-nav-1/ is an attempt to cope with the fact that there is no perfect answer to ordering of content laid out in a 2D space, and therefore navigation should itself be available in 2D.

[nattarnoff]

I want to remind folks that there are people who are disabled yet use no AT, so we need to consider that when writing specifications. Like previously said, we have non-visual screen reader users, visual screen reader users, and ZoomText (and similar magnifiers) users. But we also have neuro-divergent users who may use no AT. We have low-vision users whose "AT" is just reducing the screen or viewport's resolution or increasing font size. And there are people who are keyboard only who actually may have no disability or a physical one which doesn't require AT. There seems to be a glossing over some of these types when trying to address the concern I raised.

You really shouldn't use (default) masonry layout in the first place if the item placement matters to the semantic meaning of the content.

Completely agree, but we have to know authors will use it inappropriately unless something is enacted to prevent or work around these issues. Leaving it to authors will bring interfaces that have a11y problems.

I don't think adding more HTML/CSS features and relying on authors to use those features is a tenable solution.

I can agree with this. I'm looking for solutions that don't really leave the potential problems in the hands of authors anyway. I want them to be able to use these grid features without accidentally making an inaccessible experience just because they misunderstood which property to use when. This happens enough with the HTML we have.

As far as keyboard navigation is concerned, https://drafts.csswg.org/css-nav-1/ is an attempt to cope with the fact that there is no perfect answer to ordering of content laid out in a 2D space, and therefore navigation should itself be available in 2D.

I took some more time reviewing that today as it didn't really sink in yesterday. I do think this can answer the problem, however we are relying on the following:

• People accepting a new navigation paradigm. This will take time to be natural
• If the navigation shortcuts are more than just a single key press, we may be introducing a new problem for people with physical disabilities (this is really a comment for that spec)

I think CSS-Grid in general (and especially for masonry) needs to have a dependency on CSS-Nav where when grid is implemented, the parent grid is the spatial navigation container and default spatial navigation properties are implemented on the objects inside. This could prevent all my concerns, but I realize CSS-Grid is in the wild now and adding a dependency to it could have significant unexpected impacts to those sites using grid. WCAG could add a best practice around this, but now we are depending on authors to do the work to know the the two specs and make sure they follow the guidelines for accessibility. I'm aiming for as close to preventing inaccessible experiences by default as possible.

[aardrian]

Playing catch-up…

@nattarnoff

But I am clear that while the masonry may initially be targeted for photos and other visual pages, we have to expect it to be used by others for different things.

I have seen it used for cards, making the entire box an interactive thing.

@rachelandrew

With masonry we give people a really compelling layout method which is likely to cause this problem without the author actively doing it. It also might be tricky to test as at one breakpoint the reordering might not be too bad, and at another awful.

This is a huge point — the few cases I have seen involve devs testing at up to 3 arbitrary breakpoints (iPhone, iPad, their own desktop).

@MatsPalmgren

I slightly disagree with the premise of this issue. You really shouldn't use (default) masonry layout in the first place if the item placement matters to the semantic meaning of the content.

But people will. We have lots of evidence telling us devs will use whatever achieves their visual layout objective regardless of intent (<i> for icon fonts, display: contents as a CSS reset, …).

@frivoal

As far as keyboard navigation is concerned, https://drafts.csswg.org/css-nav-1/ is an attempt to cope with the fact that there is no perfect answer to ordering of content laid out in a 2D

That draft does not address the challenge today and should not be considered a fix. If it happens, that could be ace since it could address years of issues with floats, absolute position, order, auto-placement, tabled layouts… Stuff that pre-dates this by at least a decade.

@nattarnoff

I want to remind folks that there are people who are disabled yet use no AT, so we need to consider that when writing specifications.

Big +1.

I also agree that throwing more features at this will not solve it.

Here's what we can posit based on experience:

• developers will implement it without completely understanding the specification;
• developers will not test with AT nor users;
• developers will test with limited breakpoints;

Perhaps a place to address this is their tooling.

I know many developers rely on the grid (or shape-outside) features in browser developer tools to help them build layouts.

What do you think about browser developer tools warning devs when their layouts break some basic flow patterns? The simplest approach (IMO) is to draw arrows from box to box, and (for LTR languages) use that visualization to highlight problems when the arrow goes both upward and to the left.

I have images demonstrating these arrows where I first proposed the reading / source order tool:
https://adrianroselli.com/2017/11/feature-request-for-firefox-grid-inspector-source-order.html#Update01

Chromium now has a reading order viewer built in, but it does not use arrows to make breaks in the flow more obvious.

This feature would need to built by the browser as it builds support for masonry to be most effective.

[nattarnoff]

I'm all for tools that help developers build more accessible sites. Your concept is interesting, but I'm not sure it well it will delivery the message.

I think the best thing would be to make sure that we include clear examples and demos of how to do it right along with warnings on the impacts of doing it wrong. With clear instructions, then people working on tools and browsers can build ways to clearly demonstrate this and hopefully we can educate devs.

[aardrian]

@nattarnoff

I think the best thing would be to make sure that we include clear examples and demos of how to do it right along with warnings on the impacts of doing it wrong.

Agreed. I was not clear that my suggestion was in addition to this. Partly based on experience with devs today where they do not read the spec or even MDN (so they cannot get the message), they just read an article at popular dev site. I feel the accessibility challenges need to be raised in more than one place.

Which makes my suggestion outside the scope of the CSSWG, I know.

[QINGCHARLES]

How is this issue solved already by big players in the masonry space, e.g. Pinterest?

[tabatkins]

I think this issue can be closed. Masonry presents no new problems that aren't already present with Grid's arbitrary reordering capability, so any solutions that we come up with for either will fix both. And we have indeed come up with such a solution, via reading-order-items.

Tho also, reading-order-items won't even be necessary by default - if you're just auto-flowing things in a Masonry, you'll get an approximation to the standard reading order anyway, and that's exactly what the property would do. reading-order-items will just fix cases where you're either using order to change up the layout order, or explicitly placing some items in tracks.

(Note that the example in the OP showing a confusing order won't happen by default; it requires "over" to have been explicitly placed there, and possibly "the lazy" too. The default placement will instead make the last row [over][dog][the lazy], and since [dog] is somewhat lower than the other two, it's reasonable for the eyes to scan it after [over] and [the lazy] and get the expected reading order.)