What is considered sufficiently accessible for Gutenberg to be merged into core?

[briandeconinck]

In #10318, @tofumatt proposed an independent accessibility audit for various aspects of the new editing experience. I think that's a great idea, as do many others!

In that issue, the conversation shifted to a discussion of what would be done with the results of the audit --- in particular, whether poor results would delay merging Gutenberg into core. After some back and forth, @tofumatt locked the issue and posted:

Thanks for all the input, but as this discussion has become quite off-topic from its intended scope, I'm locking this issue's conversation for now. I'll leave it open and post updates surrounding the audit.

I think that's fair --- people, including me, were expressing strong feelings. Keeping that issue focused on the particulars of the independent audit makes sense. And Matt wasn't necessarily in a position to answer everyone's concerns.

But the questions people were raising are still important and deserve some answers. So, with minimal grandstanding, I'd like to submit a separate issue asking two narrowly-focused questions about accessibility expectations:

1. If the independent audit determines that Gutenberg fails to meet one or more of WCAG 2.0 AA's success criteria, will WordPress 5.0 be delayed until those issues are resolved?

2. If the answer to (1) is "no" and the Classic Editor is considered a sufficient alternative for users who find Gutenberg inaccessible, will the WordPress accessibility coding standards be updated to reflect this change in policy?

Some definitive answers from project leadership on these two points will help clarify the situation --- and hopefully help reestablish some trust as well.

[tofumatt]

I'm trying to get an answer to these questions, but as this issues were only raised tonight I'm not yet able to answer them. I pinged the WordPress 5.0 release lead (@m) in the other issue and asked the question of the 5.0 focus area release leads. My understanding thus far is that it would not delay release–see @josephahaden's answer in Slack about release date: https://wordpress.slack.com/archives/C02RP4X03/p1539011815000100

[briandeconinck]

Hi @tofumatt! I understand and appreciate that you don't have the answers to these questions --- and I look forward to an answer from folks who are in a position to answer them.

On a more personal note:

I did catch up on the discussion on Slack in both #core-editor and #accessibility, and I do understand that we were putting a lot of pressure on you for answers. I hope you trust that the emotions people are expressing are coming from a sincere desire for Gutenberg to have an accessible experience, and not an arbitrary desire to delay the project. Advocacy and activism are not incompatible with developing a good product.

I do think the two questions I asked in this issue are important for having some transparency and to set realistic expectations for everyone involved. But I also want to make sure you and others involved in the project realize that while WCAG provides success criteria, adhering to WCAG 2.0 AA doesn't guarantee an accessible experience. (The WP coding standards note this as well.)

I guess what I'm saying is, I hope the audit and an assessment of WCAG success criteria are part of a continuing conversation, and not the end of the conversation --- regardless of whether that conversation ought to take place in GitHub issues, in Slack, or elsewhere.

Many of us aren't in a position to contribute code to the project or participate regularly on the core accessibility team. Even so, we depend professionally or personally on WordPress continuing its otherwise strong commitment to accessibility.

[jb510]

2. If the answer to (1) is "no" and the Classic Editor is considered a sufficient alternative for users who find Gutenberg inaccessible,

In these discussions I keep seeing this written this way, “alternative for users who find Gutenberg inaccessible...”. I don’t think it’s correct phrasing because it suggests the classic editor as a fallback for specific users needing to use AD, whereas classic editor would need to be applied site wide since one can’t properly edited content that was originally authored in Gutenberg in the classic editor.

Site owners would need to elect to enable the classic editor site wide if they (morally or legally) cared about being accessible to even one site author now or in the future.

I feel discussing it in “site wide” terms might help people understand how broad the impact of this issue could be, particularly for enterprise, institutional and government use cases.

[postphotos]

Summary: WCAG 2.0 AA is probably what we should set as a milestone for compliance, but we should consider 2.1 moving forward

Many of us aren't in a position to contribute code to the project or participate regularly on the core accessibility team. Even so, we depend professionally or personally on WordPress continuing its otherwise strong commitment to accessibility.

I feel discussing it in “site wide” terms might help people understand how broad the impact of this issue could be, particularly for enterprise, institutional and government use cases.

I agree with both @briandeconinck @jb510 's thoughts here. The current WordPress guidelines say, clearly:

All new or updated code released in WordPress must conform with the WCAG 2.0 guidelines at level AA.

and has done since March of 2016.

This pact has provided a many of us the opportunity to say, Don't worry, WordPress will be stable, safe, reliable and accessible, which led to the flourish of not only a11y themes but institutions adopting WordPress that are required to comply with to operate (e.g. the US's Section 508 requirements). We, as builders of stuff for WordPress, depend on the core to be consistent, even as it grows.

It's the pressure of the default that really leads me to worry. I worry for the IT folks in higher ed or government who aren't at WordCamps who don't get the memo to install Classic editor AND Ramp, and update anyway. If Gutenberg weren't the default experience for all things yet - I'd have a lot less anxiety about what might happen. But if we ship something that isn't truly ready and decide to rescind this promise to site admins and owners, it could have potentially serious repercussions.

If we, the w.org project, clearly intend to ship without addressing the full scope of issues, we should definitely have a good answer; saying "Oh, there's a fallback in Classic" doesn't feel like it suffices the current scope requirements for Core.

Further, if we intend to revise this policy, I have a suggestion.

There are great opportunities in WCAG 2.1 AA to take it steps further. FWIW, W3C, as of June this year, has set this (WCAG 2.1) as the new recommendation. For example, the new recommendations for Target Sizes for buttons is hard to meet when you have too much stuff going on in your interface, and likely sets up a great design problem for us to further refactor and refine what we intend to do. Bigger buttons and more focused interfaces are better for everyone.

In short, I'm really happy we're making progress, and this response is not intended to be negative; I really do want to see Gutenberg soon, but not too soon.

[edent]

In the UK, and throughout the EU, it is a legal requirement that public sector websites are accessible.

Websites must

meet accessibility standards - this means making your website ‘perceivable, operable, understandable and robust’ for all users - you can achieve this by making sure it meets the international accessibility standard, WCAG 2.1 AA or its European equivalent, EN301 549

I'd suggest that if Gutenberg doesn't meet WCAG 2.1 AA, then it is unsuitable for use.

In the UK, 21% of the population (13.3 million people) reported a disability.

[briandeconinck]

@tofumatt's update to #10318 makes moot question (1) in my original issue here:

For at least the time being, Automattic has decided to forgo conducting an Accessibility audit on Gutenberg. The main reasons being:

an audit will not be actionable given our release timeline, because…
the audit will not affect release timing, so...
it would be more prudent to explore an audit on a less rushed timeline in the future

(I am disappointed by this decision, especially since Matt suggested the audit as a way of demonstrating that Gutenberg is more accessible than many who are expressing concerns believe. It strikes me as a missed opportunity to build some trust. A healthy community strikes me as more valuable long-term than an on-time release.)

Further, in today's accessibility meeting discussion of the WP accessibility coding standard, @tofumatt also said:

must [being used in the wording of the coding standard] creates the kind of slides I see people posting about how WordPress 5.0 can’t be released unless WCAG is met, which seems to me an untrue conclusion, given the release lead [Matt Mullenweg] hasn’t given any of us veto power that I know of

This more or less answers both (1) and (2). My interpretation---and please, someone let me know if this is incorrect---is that the WP coding standards are recommendations rather than rules, and that they are to be applied at the discretion of the release lead.

Note that this is my interpretation of @tofumatt's comments. If others involved in the project leadership want to weigh in with a different interpretation, I would be eager to hear it!

But given today's updates, I'd like to ask one new question. I think it still fits within the scope of this issue.

1. When deciding whether the new editor is sufficiently accessible to be merged into core, what criteria is the release lead using to make their decision?

I want to emphasize that I'm not a Guten-skeptic looking to arbitrarily delay the project. I'm a huge fan, and have spoken about why I'm excited at WordCamps and other events.

But there's been a lot of concern expressed about the accessibility of the new editor, by both regular participants in the accessibility team and by the broader community. If the editor is considered "ready" without an accessibility audit, helping people understand how that conclusion was reached might help reassure those of us concerned about potential legal risk (especially if disregarding the coding standards becomes the norm).

[briandeconinck]

(Pressed the wrong button and accidentally closed. Reopened.)

[tofumatt]

For what it's worth, it would be helpful to know of the specific, discrete WCAG 2.0 AA violations in the editor. I know the publish button's colour contrast fails (I just noticed this recently and need to file a bug, but I also need to go to bed…), but there are Accessibility issues in the label that are specific, ones that are general, ones that are discussions, but I don't usually see explicit references to the Accessibility Standards in the issues.

Maybe I'm missing them because I'm just too scattered right now, but if they aren't there, knowing an issue violates a specific WCAG guidelines and the fix for it would be great. If they're in the issues I'm sorry I missed them 😞

If there aren't that many issues with explicit WCAG violations it could be that things are okay by the standards. Which isn't to say we have no issues, but none/few that are dire.

[postphotos]

@tofumatt: Thanks for your reply, and I understand your position and thoughts here. Echoing @mor10, I am so excited to see Gutenberg ship, and have been tinkering with it for the better part of a year and a half, but how things get launched is critical.

If there aren't that many issues with explicit WCAG violations it could be that things are okay by the standards. Which isn't to say we have no issues, but none/few that are dire.
I want to agree with you, but I don't think this has been evaluated yet.

I think the objective of this ticket, beyond discourse, is to accurately describe:

What is considered sufficiently accessible for Gutenberg to be merged into core?

Determining what is the actual requirement here will help address what people's perception is, and hopefully rally together the community toward the right solution.

Thinking not about emotions here, and instead action steps so we can (hopefully) find consensus. I think it's important to state the intent by the WordPress.org project and the Gutenberg team. So with that there are three questions that this group should discuss:

1. So yes, generating discrete issues/PRs is the practical "how we'd address this," but to what lengths are we addressing it in comparison to, and does the existing promise of "all new features to meet WCAG 2.0 AA guidelines" hold true still?

2. Alternately, I think the intent of your ticket Do an accessibility audit and make a blog post about it #10318 (closed earlier today) was to construct that milestone of tasks, tickets and guidelines; I was not able to comment there as you closed the issue to "collaborators only," but I think we could find a reasonable way to move forward on a way that satisfies both that (10318) and this (10537) tickets' intents.
   If a third-party audit is out of the question, what still is available to the community to shake out and suss recommendations? If there we can lean on an agreed-upon set of acceptance criteria (e.g. we promise that a) Gutenberg will have a keyboard-friendly layout b) works with at least JAWS/VoiceOver c) Has color contrast and usability testing, etc.) we can confidently say, "Gutenberg went through a process."
   I'd love to help, but my personal capacity is limited but I have seen a few things start to crop up. You mentioned an issue with the Publish button, and I've found a general issue with tabbing through the interface. In trying a keyboard-only operation to create a post, I found myself caught in several places and unable to proceed further. I haven't been able to gather my thoughts to create anything actionable yet, but plan to later this week. In short, Gutenberg's a11y needs aren't theoretical; I fractured my wrist earlier this year and primarily only used a keyboard (with one hand, mind you) for nearly two months.
   
   The legendary work that @rianrietveld and @afercia have led out thus far, plus the countless contributors across the community, surfaced ~200 issues and PRs inside Gutenberg thus far, and I know there's more issues to be shaken out, but what do we intend to do, and how do we intend to do it?

3. What process surfaces to make that final decision? It's assumed it's the release lead (and perhaps you, @tofumatt, as a11y lead) to rally on what is decided here?
   
   For example, @briandeconinck's comment above is critical:

If the editor is considered "ready" without an accessibility audit, helping people understand how that conclusion was reached might help reassure those of us concerned about potential legal risk (especially if disregarding the coding standards becomes the norm).

If we (w.org) decide to break from the way we've done things, whose responsibility is to explain why, and what website owners, agencies, institutions, etc. should do if they find themselves in the inevitable pickle as a result of this proceeding?

[tofumatt]

Good points in there, and you’re right: I went off-topic in this issue a bit. Sorry about that, I’ll try to keep it better focused! 😅

I meant to say that the standard seems to be WCAG 2.0 AA as per the guidelines, but given my installation as accessibility release lead less than a month before UI freeze, I wouldn’t say that’s an enforceable standard for this release.

Those standards are maintained by the WordPress Accessibility Team but aren’t, as I understand it, a blocker to release (see yesterday’s Accessibility chat on Slack).

It’s possible this relates less to Gutenberg and more to core processes? Is there a sort of meta component in WordPress Trac to discuss that (with a link to this issue?) I don’t want to offload discussion for the sake of it, but core committers with more experience than I might be able to speak to it more than I.

A side note: the standard for WordPress Accessibility is for new and updated code, but has anyone checked to see if it actually applies to all existing WordPress UI? I’m unsure, but just don’t want to persist the narrative that if Gutenberg merged it’d be the only violation of the spirit of that Accessibilty standard. Even if it would be a newer and more visible one.

[swissspidy]

A side note: the standard for WordPress Accessibility is for new and updated code, but has anyone checked to see if it actually applies to all existing WordPress UI?

I hope this is not meant as an excuse or something :-) The answer is: no it doesn't. WordPress is a huge and old project and it would be impossible to fix all existing accessibility issues in one sweep. That's why the goal was set for new code introduced to WordPress— at least that's how I understand it. The rule was similar for coding standards: leave existing code as is, but newer code must adhere to that. Luckily, fixing coding standards is easier (can be mostly automated) than accessibility ones.

[tofumatt]

Didn’t mean it as an excuse, more just something to keep in mind when talking about accessibility issues in the editor when updating client sites–if we shipped an editor with accessibility issues, we wouldn’t be shipping the only piece of WordPress with said issues.

Not an excuse, but some context worth including 😊

[briandeconinck]

@postphotos thank you for saying many of the things I was trying to say, but so much better and clearer.

@tofumatt, you're right that accessibility issues still exist in other parts of WordPress. (I believe the accessibility team was hoping to resolve some of those in v4.9.9 before the change in timelines was announced.)

But I'm sure you'll agree that Gutenberg is a significant change to a user interface that's been almost static for 12 years---and an interface that any user with more than a Subscriber role will use at some point.

There's really no comparison between changes in other recent releases and Gutenberg, at least in terms of accessibility (good or bad).

And so I think it's fair to expect that some process (even an informal one) will assess the impact and make a decision about whether the new editor is sufficiently accessible to be merged into core. Even if other releases haven't gone through a similar assessment, this is no ordinary release.

When I opened this issue, I was assuming that the audit you proposed would be part of that assessment process. I wanted to understand how it fit into that process better, and so I asked my questions. Yesterday, I learned that this assumption was incorrect.

So now my question is: What process is the release lead following to decide if the editor is sufficiently accessible?

It's absolutely fine if you don't have an answer. But it's still an answerable question.

[tofumatt]

Right now my process to determine that is everything in the Milestone: Accessibility milestone on GitHub. It’s not a list set in stone: can remove things after reconsidering; I tried to make it a generous rather than too narrowly scoped list. We can also add to it when they aren’t significant UI changes (UI freeze is tomorrow).

Hope that helps!

[briandeconinck]

I don't mean to be difficult, but no, that doesn't help.

You said yesterday in Slack that the release lead hasn't given you veto power to say "this isn't accessible, it's not ready."

So while your process may be to work through everything in the Milestone: Accessibility milestone, that doesn't provide any real insight into how accessibility is assessed for the decision to release or not release.

[tofumatt]

Ah, sorry, I understand. If critical issues weren't addressed in full by the release date, I would advocate waiting, but I think that's just a suggestion on my part. That said: I think the only critical piece in that milestone that should delay release to account for keyboard navigation is #5694, which is being worked on today.

There are some focus issues in that milestone I'm reevaluating as part of spending my day doing a lot of triaging. I don't think there are any other blockers.

[logicalphase]

With all due respect to you @m, it seems that you've been tossed into the fire and thrown a of can lighter fluid to hold on to. You clearly had a set of specific objectives, but you're not empowered to make decisions, and I feel bad for you in that regard. But the absence of a comments from the one who is empowered, in fact it seems the only one who is on this issue, from @m is evident. He should at least take some time to back you up here.

[photomatt]

@photomatt ? Guys I have no clue what are you discussing here. I understand I been brought into this by mistake. Regards!

[Stephen-Cronin]

@photomatt ? Guys I have no clue what are you discussing here. I understand I been brought into this by mistake. Regards!

Pretty sure he meant @m who goes by 'photomatt' everywhere else (twitter, instagram, wordpress.org, etc).

[logicalphase]

My mistake. I edited the OP.

[afercia]

Discussed during today's accessibility bug-scrub, as there's still the label Needs accessibility feedback. Not sure how much this issue is actionable at this point. Also not sure what kind of accessibility feedback can be given other than pointing to the official WordPress accessibility coding standards: https://make.wordpress.org/core/handbook/best-practices/coding-standards/accessibility-coding-standards/

[briandeconinck]

@afercia I've been following along quietly in today's bug-scrub, and I basically agree.

When I opened this issue, I was ultimately trying to get answers about project governance and community norms. I don't know the right way to raise those questions in an actionable way, but I think it's clear that this issue hasn't been the right avenue.

I'm definitely okay with removing Needs accessibility feedback and don't anticipate any meaningful responses from project leadership. Leaving it open right now, but I'm fine with closing it if there's no further comment.

[afercia]

@briandeconinck Thanks. Sure, we agree to keep the "Accessibility" label and keep it open for now 🙂

[afercia]

Gutenberg is in core now. Closing.