New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do an accessibility audit and make a blog post about it #10318

Open
tofumatt opened this Issue Oct 3, 2018 · 75 comments

Comments

Projects
None yet
@tofumatt
Member

tofumatt commented Oct 3, 2018

Right now there is a perception that Gutenberg is quite inaccessible, both in general and compared to the Classic Editor. Below is more/less a copy+pasted comment I made in a recent Make Post about 5.0:

It should be pointed out that the existing (Classic) editor doesn’t include many components for plugin developers extending the editor. The core editor in WordPress is accessible by virtue of being quite basic (it's essentially a <textarea>), but it’s also at the mercy of plugin developers to ensure their code is accessible. This means the difference between a WordPress editor without and with plugins is wildly different–this should be less true for a Gutenberg install with 5 or 500 blocks.

Core Gutenberg components and blocks are built with accessibility in mind, and block author will be able to use these components in their own blocks and get a lot of accessibility built-in to their blocks for free. This is a big win for accessibility of the editor after it’s been extended at all.

There are certainly aspects (often third-party code like colour or date pickers) that are not accessible, though we’ve been working to improve those when there are better solutions.

The Classic Editor and Gutenberg’s editor aren’t an apples-to-apples comparison; one is extremely basic and one is very full-featured. I expect the latter to require more work to make fully accessible, but as we work toward that (and identify bugs), we can provide all users with a better experience.


I think there's a notion of Gutenberg being inaccessible because of older accessibility audits that identified a lot of issues in the very early versions. Things have changed a lot since the early days, and when the plugin was labeled "1.0" it was hardly a ready-to-ship product. I worry that many of those sentiments haven't been re-examined and updated, so there is a prevailing idea that Gutenberg is not accessible or is entirely less accessible than the Classic Editor.

What I'd venture is that Gutenberg is selectively less accessible, but overall more accessible feature-for-feature. Something like a date picker or a certain interaction being inaccessible does not make the entire editor inaccessible. Feature-for-feature, compared to a classic editor with similar capabilities (eg a bunch of plugins installed), I'd bet* Gutenberg is more accessible.

Anyway, I would like to see about doing a new round of accessibility testing with community a11y folks (and see about accessing some Automattic resources to help with it as well).

After that it'd be cool to do a blog post with data that could show Gutenberg's current state of accessibility. It'd be neat to highlight the built-in accessibility features that third-party blocks get for free as well, to show how blocks beat old editor plugins for built-in accessibility.

Inspired by: https://wordpress.slack.com/archives/C02RQBWTW/p1538599045000200


  • -One coffee bet, signed me.

@tofumatt tofumatt added this to the Merge: Accessibility milestone Oct 3, 2018

@tofumatt tofumatt self-assigned this Oct 3, 2018

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 4, 2018

Member

It would be really cool to know that all blocks that replicate classic editor functionality are accessible in Gutenberg.

Member

tofumatt commented Oct 4, 2018

It would be really cool to know that all blocks that replicate classic editor functionality are accessible in Gutenberg.

@mor10

This comment has been minimized.

Show comment
Hide comment
@mor10

mor10 Oct 4, 2018

The right thing to do here is to get an independent accessibility audit. WordPress has a stated policy of meeting accessibility guidelines in core, and also has a responsibility to its users to meet those guidelines. Only a thorough accessibility audit can show whether this is the case or not. The accessibility team have flagged numerous issues, some of which have been handled, others which have not. At the very least, these must be addressed.

WordPress is an interface between the publisher, a database, and the visitor. How exactly the publisher chooses to input or manipulate the interface is up to that user. WordPress' job is to make the interface accessible for whatever input modality chosen by that user. This is neither new nor controversial: It is the foundation on which the web stands.

mor10 commented Oct 4, 2018

The right thing to do here is to get an independent accessibility audit. WordPress has a stated policy of meeting accessibility guidelines in core, and also has a responsibility to its users to meet those guidelines. Only a thorough accessibility audit can show whether this is the case or not. The accessibility team have flagged numerous issues, some of which have been handled, others which have not. At the very least, these must be addressed.

WordPress is an interface between the publisher, a database, and the visitor. How exactly the publisher chooses to input or manipulate the interface is up to that user. WordPress' job is to make the interface accessible for whatever input modality chosen by that user. This is neither new nor controversial: It is the foundation on which the web stands.

@bamadesigner

This comment has been minimized.

Show comment
Hide comment
@bamadesigner

bamadesigner Oct 4, 2018

I agree with @mor10. Considering how invested Automattic and the core team has been in the project, and are still seeing issues, the only true and valid path forward is to have an independent, objective audit.

bamadesigner commented Oct 4, 2018

I agree with @mor10. Considering how invested Automattic and the core team has been in the project, and are still seeing issues, the only true and valid path forward is to have an independent, objective audit.

@postphotos

This comment has been minimized.

Show comment
Hide comment
@postphotos

postphotos Oct 4, 2018

Contributor

WordPress has a stated policy of meeting accessibility guidelines in core

Yes, exactly. I'd want confidence from the leads in the larger w.org project - such as @afercia and @rianrietveld - in the work here.

In short, I believe Gutenberg CAN be accessible, but I'm concerned it might not be shippable in its current state based on the 117 items open in the project.

Hoping this can happen!

Contributor

postphotos commented Oct 4, 2018

WordPress has a stated policy of meeting accessibility guidelines in core

Yes, exactly. I'd want confidence from the leads in the larger w.org project - such as @afercia and @rianrietveld - in the work here.

In short, I believe Gutenberg CAN be accessible, but I'm concerned it might not be shippable in its current state based on the 117 items open in the project.

Hoping this can happen!

@rianrietveld

This comment has been minimized.

Show comment
Hide comment
@rianrietveld

rianrietveld Oct 4, 2018

Member

I'd fully support an external, independent accessibility audit.

Member

rianrietveld commented Oct 4, 2018

I'd fully support an external, independent accessibility audit.

@nic-bertino

This comment has been minimized.

Show comment
Hide comment
@nic-bertino

nic-bertino Oct 4, 2018

As you mentioned, there are some perception issues that I think are related to the lack of material and information regarding user testing, and specifically, re-testing as development progressed. Instead of chasing user outcomes, it feels like we are chasing down bugs and regressions.

I appreciate your honesty here, and I agree that an external/independent audit would be valuable. But, I do fear that this is retroactively addressing accessibility from a technical standpoint (i.e., checking the boxes for WCAG 2.x/AA) instead of looking to provide a better experience for those that might not be using a mouse and keyboard. For accessibility alone, qualitative and quantitative research of the Gutenberg experience without data relative to the current experience (or, even better, other similar products) might not tell a fair story.

I think transparency around this information would go a long with community support and ease a lot of friction, both from an accessibility standpoint and the larger Gutenberg rollout.

nic-bertino commented Oct 4, 2018

As you mentioned, there are some perception issues that I think are related to the lack of material and information regarding user testing, and specifically, re-testing as development progressed. Instead of chasing user outcomes, it feels like we are chasing down bugs and regressions.

I appreciate your honesty here, and I agree that an external/independent audit would be valuable. But, I do fear that this is retroactively addressing accessibility from a technical standpoint (i.e., checking the boxes for WCAG 2.x/AA) instead of looking to provide a better experience for those that might not be using a mouse and keyboard. For accessibility alone, qualitative and quantitative research of the Gutenberg experience without data relative to the current experience (or, even better, other similar products) might not tell a fair story.

I think transparency around this information would go a long with community support and ease a lot of friction, both from an accessibility standpoint and the larger Gutenberg rollout.

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 4, 2018

Member

I'm not surprised to hear others are in favour of an external audit, which would be great to do. 👍 I can tell you I'd love that too, and will be looking into ways we can facilitate that over the next week.

Are there any accessibility-specialising agencies/companies involved with WordPress that would be able to prioritise their time to run through an accessibility audit of the project (probably as of 4.0, which is a big release)? I think it would be handy to establish a baseline sense of how Gutenberg stacks up against the Classic Editor.

I am sure there will be issues (as mentioned above, we have over 100 accessibility issues) but I'm trying to get a macro view of accessibility in Gutenberg. It should be pointed out we have well over 100 issues labelled "bugs" as well, but this is thanks to us considering no bug too small. We try to log everything, but remember this means 100 accessibility issues, not 100 accessibility blocks. The accessibility blockers (as of right now) are contained in the Merge Milestone this issue is a part of.

Thanks to those who have added their voices to support of an audit. I will provide a list of flows we want to test and ensure work for users as an MVP--when I do I'll post them here but I haven't fully formulated them yet.

Member

tofumatt commented Oct 4, 2018

I'm not surprised to hear others are in favour of an external audit, which would be great to do. 👍 I can tell you I'd love that too, and will be looking into ways we can facilitate that over the next week.

Are there any accessibility-specialising agencies/companies involved with WordPress that would be able to prioritise their time to run through an accessibility audit of the project (probably as of 4.0, which is a big release)? I think it would be handy to establish a baseline sense of how Gutenberg stacks up against the Classic Editor.

I am sure there will be issues (as mentioned above, we have over 100 accessibility issues) but I'm trying to get a macro view of accessibility in Gutenberg. It should be pointed out we have well over 100 issues labelled "bugs" as well, but this is thanks to us considering no bug too small. We try to log everything, but remember this means 100 accessibility issues, not 100 accessibility blocks. The accessibility blockers (as of right now) are contained in the Merge Milestone this issue is a part of.

Thanks to those who have added their voices to support of an audit. I will provide a list of flows we want to test and ensure work for users as an MVP--when I do I'll post them here but I haven't fully formulated them yet.

@Shelob9

This comment has been minimized.

Show comment
Hide comment
@Shelob9

Shelob9 Oct 5, 2018

Contributor

An external audit is a great idea. I would be happy to contribute financially to hiring a qualified firm to perform this audit.

Contributor

Shelob9 commented Oct 5, 2018

An external audit is a great idea. I would be happy to contribute financially to hiring a qualified firm to perform this audit.

@bamadesigner

This comment has been minimized.

Show comment
Hide comment
@bamadesigner

bamadesigner Oct 5, 2018

I would also contribute financially, especially for something of this magnitude and importance.

I would also be willing to lead the way to help raise money from the community.

bamadesigner commented Oct 5, 2018

I would also contribute financially, especially for something of this magnitude and importance.

I would also be willing to lead the way to help raise money from the community.

@aardrian

This comment has been minimized.

Show comment
Hide comment
@aardrian

aardrian Oct 5, 2018

I like this idea. For enterprises that require WCAG compliance in their software, an audit can go a long way to quelling concerns. Generating a VPAT from an audit may also help uptake in organizations that require it as part of their contract process. See Building Accessibility into Technology Vendor Contracts for related info.

aardrian commented Oct 5, 2018

I like this idea. For enterprises that require WCAG compliance in their software, an audit can go a long way to quelling concerns. Generating a VPAT from an audit may also help uptake in organizations that require it as part of their contract process. See Building Accessibility into Technology Vendor Contracts for related info.

@logoscreative

This comment has been minimized.

Show comment
Hide comment
@logoscreative

logoscreative Oct 5, 2018

I also support an external, independent accessibility audit. I'll check into my day job's excellent a11y resources and see what they recommend. :)

logoscreative commented Oct 5, 2018

I also support an external, independent accessibility audit. I'll check into my day job's excellent a11y resources and see what they recommend. :)

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 5, 2018

Member

That’s very kind! But we aren’t looking for financial contributions from community. I’m going to contact some accessibility auditors and see about getting them to run some audits on Gutenberg.

I’ll see what kind of costs and timelines are involved, but if we can get something reasonably priced with a timeline that will work for 5.0 I should be able to get Automattic to cover the cost. 😊

Member

tofumatt commented Oct 5, 2018

That’s very kind! But we aren’t looking for financial contributions from community. I’m going to contact some accessibility auditors and see about getting them to run some audits on Gutenberg.

I’ll see what kind of costs and timelines are involved, but if we can get something reasonably priced with a timeline that will work for 5.0 I should be able to get Automattic to cover the cost. 😊

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 5, 2018

Member

I was a bit unsure of financial support I could provide from Automattic when I posted an earlier comment and I was a bit unclear about expectations from community: sorry about that! I’ve updated the comment and clarified what I wasn’t looking for. Right now I’m going to see about arranging audits, it will take me probably a few days at least to wrangle the quotes! I will report back once I know more 😊

Member

tofumatt commented Oct 5, 2018

I was a bit unsure of financial support I could provide from Automattic when I posted an earlier comment and I was a bit unclear about expectations from community: sorry about that! I’ve updated the comment and clarified what I wasn’t looking for. Right now I’m going to see about arranging audits, it will take me probably a few days at least to wrangle the quotes! I will report back once I know more 😊

@grahamarmfield

This comment has been minimized.

Show comment
Hide comment
@grahamarmfield

grahamarmfield Oct 5, 2018

I know this may not be the place to state this, but my company does accessibility audits. So let me know if you want to know more. Although not so active recently, I have been involved with WordPress for a while and have contributed accessibility ideas to Gutenberg as well as other parts of WP. If I'm not independent enough, I work with a trusted partner who has never used WordPress.

grahamarmfield commented Oct 5, 2018

I know this may not be the place to state this, but my company does accessibility audits. So let me know if you want to know more. Although not so active recently, I have been involved with WordPress for a while and have contributed accessibility ideas to Gutenberg as well as other parts of WP. If I'm not independent enough, I work with a trusted partner who has never used WordPress.

@nic-bertino

This comment has been minimized.

Show comment
Hide comment
@nic-bertino

nic-bertino Oct 5, 2018

@tofumatt I think a request for proposals would be a good way to state exactly what an audit is looking for and give service providers an opportunity to provide a general outline of how they would proceed. Is there a precedent for this type of process at Automattic?

nic-bertino commented Oct 5, 2018

@tofumatt I think a request for proposals would be a good way to state exactly what an audit is looking for and give service providers an opportunity to provide a general outline of how they would proceed. Is there a precedent for this type of process at Automattic?

@CTenan

This comment has been minimized.

Show comment
Hide comment
@CTenan

CTenan Oct 5, 2018

Hello, I am the IT Accessibility Coordinator for NC State University. We use WordPress for our campus websites and as a state institution we are required to follow Section 508 of the Rehabilitation Act of 1973. Because of this, our authoring tools like WordPress have to be accessible for individuals with disabilities.

We have a huge variety of users of WordPress including students, faculty and staff who use WordPress blogs and edit WordPress sites on behalf of their departments of their organizations. All that to say, WordPress must be accessible for these individuals.

As an institution of higher education, we want to stay current with new programs and technologies. It would be detrimental for us to not be able to progress and use Gutenberg or to do a "separate by equal" situation where people with disabilities are limited to using the classic editor while able-bodied people can Gutenberg.

CTenan commented Oct 5, 2018

Hello, I am the IT Accessibility Coordinator for NC State University. We use WordPress for our campus websites and as a state institution we are required to follow Section 508 of the Rehabilitation Act of 1973. Because of this, our authoring tools like WordPress have to be accessible for individuals with disabilities.

We have a huge variety of users of WordPress including students, faculty and staff who use WordPress blogs and edit WordPress sites on behalf of their departments of their organizations. All that to say, WordPress must be accessible for these individuals.

As an institution of higher education, we want to stay current with new programs and technologies. It would be detrimental for us to not be able to progress and use Gutenberg or to do a "separate by equal" situation where people with disabilities are limited to using the classic editor while able-bodied people can Gutenberg.

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 5, 2018

Member

That is so awesome to hear 👍

I totally agree! Of course, I want to set the clear expectation that there may be accessibility issues we discover (or possibly know about) that we are not able to fix before WordPress 5.0 is released.

There are plenty of sites, for plenty of reasons, that will want/need to use the Classic Editor plugin for a limited amount of time while Gutenberg–or outside plugins–are improved. There are probably going to be users who encounter accessibility issues–along with a host of other issues–that necessitate the Classic plugin for a few weeks/months while we improve our software.

I want the future of the WordPress Editor to be awesome and accessible for everyone. My apologies in advance for users who fall through the cracks at first, for whatever reason. But know we will work toward making Gutenberg and WordPress awesome for them too, even if it doesn't happen right away. ❤️

Regarding Audit

Hey all! I'm still a bit new to being a release lead and I'm coming from nearly a decade working on the @mozilla project where we too worked very much in the open, missteps and all. I'll be as transparent as possible about what's happening, including trying to give timely updates (hence my being unsure about being able to pay for accessibility audits until checking with some folks).

The kind of audit I'm looking to do may not be anything too legal/compliance-specific as I am first looking to get a macro sense of the usability of WordPress by users with accessibility needs. If that includes compliance stuff though: awesome.

Right now I've been in touch with some @WordPress Accessibility community members and knowledgable @Automattic employees who have provided me with some folks to contact about an Accessibility audit. Right now I want it to be impartial (somewhat outside of the WordPress community is great to avoid bias/past knowledge/etc.), high-level, actionable, and done somewhat quick. I know I'm asking a lot 😆

I have some folks in mind now and will be reaching out to them I have contacted several companies that specialise in accessibility audits. I'll post updates here, but I won't know more until next week. For now, there's:

  1. no need to offer financial support (but OMG thank you so much to those who have, that's super cool to see your commitment to WordPress)
  2. no need to offer your services (I think we want someone outside the WordPress ecosystem, but if you want to help with regular bugs/accessibility testing please do!)
  3. much ❤️ from me 🙂

Have a great weekend, I'll keep you all posted! 😄

Member

tofumatt commented Oct 5, 2018

That is so awesome to hear 👍

I totally agree! Of course, I want to set the clear expectation that there may be accessibility issues we discover (or possibly know about) that we are not able to fix before WordPress 5.0 is released.

There are plenty of sites, for plenty of reasons, that will want/need to use the Classic Editor plugin for a limited amount of time while Gutenberg–or outside plugins–are improved. There are probably going to be users who encounter accessibility issues–along with a host of other issues–that necessitate the Classic plugin for a few weeks/months while we improve our software.

I want the future of the WordPress Editor to be awesome and accessible for everyone. My apologies in advance for users who fall through the cracks at first, for whatever reason. But know we will work toward making Gutenberg and WordPress awesome for them too, even if it doesn't happen right away. ❤️

Regarding Audit

Hey all! I'm still a bit new to being a release lead and I'm coming from nearly a decade working on the @mozilla project where we too worked very much in the open, missteps and all. I'll be as transparent as possible about what's happening, including trying to give timely updates (hence my being unsure about being able to pay for accessibility audits until checking with some folks).

The kind of audit I'm looking to do may not be anything too legal/compliance-specific as I am first looking to get a macro sense of the usability of WordPress by users with accessibility needs. If that includes compliance stuff though: awesome.

Right now I've been in touch with some @WordPress Accessibility community members and knowledgable @Automattic employees who have provided me with some folks to contact about an Accessibility audit. Right now I want it to be impartial (somewhat outside of the WordPress community is great to avoid bias/past knowledge/etc.), high-level, actionable, and done somewhat quick. I know I'm asking a lot 😆

I have some folks in mind now and will be reaching out to them I have contacted several companies that specialise in accessibility audits. I'll post updates here, but I won't know more until next week. For now, there's:

  1. no need to offer financial support (but OMG thank you so much to those who have, that's super cool to see your commitment to WordPress)
  2. no need to offer your services (I think we want someone outside the WordPress ecosystem, but if you want to help with regular bugs/accessibility testing please do!)
  3. much ❤️ from me 🙂

Have a great weekend, I'll keep you all posted! 😄

@LakenH

This comment has been minimized.

Show comment
Hide comment
@LakenH

LakenH Oct 5, 2018

If looking for further vendors for the audit, I recommend WebAIM at Utah State University, who is generally considered as one of the top web a11y resources in the United States. Not affiliated with them in any way other than reading their amazing web a11y documentation, but I have seen that they do offer audits.

LakenH commented Oct 5, 2018

If looking for further vendors for the audit, I recommend WebAIM at Utah State University, who is generally considered as one of the top web a11y resources in the United States. Not affiliated with them in any way other than reading their amazing web a11y documentation, but I have seen that they do offer audits.

@sbrinley

This comment has been minimized.

Show comment
Hide comment
@sbrinley

sbrinley Oct 5, 2018

Although WordPress itself is open source and still has lots of volunteer labor, it's also hit the "big time," as of were, and can no longer afford to fall back on the, "But we're just open source," crutch. I think a full accessibility audit and the commitment to improve where WP is found to be lacking (not just Gutenberg) would be a great source of forward movement when it comes to WP adoption. Plus, of course, democratizing publishing, and all that.

sbrinley commented Oct 5, 2018

Although WordPress itself is open source and still has lots of volunteer labor, it's also hit the "big time," as of were, and can no longer afford to fall back on the, "But we're just open source," crutch. I think a full accessibility audit and the commitment to improve where WP is found to be lacking (not just Gutenberg) would be a great source of forward movement when it comes to WP adoption. Plus, of course, democratizing publishing, and all that.

@davisshaver

This comment has been minimized.

Show comment
Hide comment
@davisshaver

davisshaver Oct 10, 2018

Contributor

There are plenty of sites, for plenty of reasons, that will want/need to use the Classic Editor plugin for a limited amount of time while Gutenberg–or outside plugins–are improved. There are probably going to be users who encounter accessibility issues–along with a host of other issues–that necessitate the Classic plugin for a few weeks/months while we improve our software.

This sentiment is just unacceptable to me and I hope many others who have internalized the WordPress values...

My apologies in advance for users who fall through the cracks at first, for whatever reason. But know we will work toward making Gutenberg and WordPress awesome for them too, even if it doesn't happen right away. ❤️

What is the upside of pushing through an experience that’s not up to the standards of everything else in Core? Deadlines aren’t arbitrary, but that doesn’t mean you cut corners. At least not according to the WordPress values as I understand them. These values are likely distinct from anything Mozilla had, as a newcomer to the community I encourage you to look into them.

In general the sentiment that mostly-working will be enough creates unnecessary risk for the community, and one must wonder who benefits from that. What is the upside to the Gutenberg project or WordPress? It’s not as if we need the help identifying issues.

Contributor

davisshaver commented Oct 10, 2018

There are plenty of sites, for plenty of reasons, that will want/need to use the Classic Editor plugin for a limited amount of time while Gutenberg–or outside plugins–are improved. There are probably going to be users who encounter accessibility issues–along with a host of other issues–that necessitate the Classic plugin for a few weeks/months while we improve our software.

This sentiment is just unacceptable to me and I hope many others who have internalized the WordPress values...

My apologies in advance for users who fall through the cracks at first, for whatever reason. But know we will work toward making Gutenberg and WordPress awesome for them too, even if it doesn't happen right away. ❤️

What is the upside of pushing through an experience that’s not up to the standards of everything else in Core? Deadlines aren’t arbitrary, but that doesn’t mean you cut corners. At least not according to the WordPress values as I understand them. These values are likely distinct from anything Mozilla had, as a newcomer to the community I encourage you to look into them.

In general the sentiment that mostly-working will be enough creates unnecessary risk for the community, and one must wonder who benefits from that. What is the upside to the Gutenberg project or WordPress? It’s not as if we need the help identifying issues.

@davisshaver

This comment has been minimized.

Show comment
Hide comment
@davisshaver

davisshaver Oct 10, 2018

Contributor

Since Automattic will apparently be sponsoring and conceiving parameters for this audit, there’s the potential appearance of bias and that they are scoring their own work. I propose that the greater community proceed with crowdfunding to hire an accessibility specialist for a parallel report from the perspective of actual users. Maybe Automattic can do a technical audit or whatever, but the suggestion that Gutenberg accessibility is “not so bad” has been vehemently disagreed by real experts in this field (to wit, Sina Braham at WordCamp Publishers). Maybe we need something closer to a Mikey test where we get blind users to navigate through the app and say if they’d use it again.

Contributor

davisshaver commented Oct 10, 2018

Since Automattic will apparently be sponsoring and conceiving parameters for this audit, there’s the potential appearance of bias and that they are scoring their own work. I propose that the greater community proceed with crowdfunding to hire an accessibility specialist for a parallel report from the perspective of actual users. Maybe Automattic can do a technical audit or whatever, but the suggestion that Gutenberg accessibility is “not so bad” has been vehemently disagreed by real experts in this field (to wit, Sina Braham at WordCamp Publishers). Maybe we need something closer to a Mikey test where we get blind users to navigate through the app and say if they’d use it again.

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 10, 2018

Member

This is what I've been sending out to companies I've been approaching for this work, by the way. It may help other folks involved in Accessibility, I'm not sure. At the very least I'm posting it here for transparency 🙂


Requirements

A series of accessibility tests to be done using the Gutenberg editor on a WordPress website using WP-Admin. These tests should include, at minimum, the following test flows:

Core Publishing Flow

  • Create a new post
  • Add common content (paragraph, image, list, HTML, and table content; see “Using Blocks” below)
  • Publish a post immediately and post is made as intended
  • Schedule a post for the future using date picker
  • Assign categories and tags to post
  • Assign a featured image to a post
  • Experiment with other Document-level settings, time-permitting

Using Blocks

These are the top-twenty most used blocks on WordPress.com and should be tested to find any accessibility issues not already filed in our list of accessibility issues:

  1. paragraph
  2. image
  3. heading
  4. list
  5. gallery
  6. quote
  7. embed/youtube
  8. html
  9. separator
  10. more
  11. cover-image
  12. shortcode
  13. button
  14. table
  15. spacer
  16. embed
  17. block
  18. embed/twitter
  19. columns
  20. code

These blocks should be tested with assistive technologies to ensure there are no blockers to their usage, and that their block-level settings are accessible.

Classic Editor

We should verify that, compared to the Classic Editor, there are not notable regressions with regard to accessibility. Creating a post with the same kind of content one can create in a barebones Classic Editor context should be doable with Gutenberg. Gutenberg having zero regressions in editing experience compared to the Classic Editor is my ultimate goal for saying we can recommend it as the default editing experience for users of assistive technologies.

Assistive Technologies to Consider

Because of the limited timeline available and the prioritisation of actionability over extreme thoroughness, I would prefer the top two-to-three browser+screenreader combinations to be used for testing. Using more if time allows is absolutely accessible, but I would prefer starting with the most-used combinations. From my understanding this is:

  • JAWS with Internet Explorer/Edge (Windows)
  • NVDA with Firefox (Windows)
  • VoiceOver with Safari (MacOS)
  • ZoomText

Mobile assistive technology is not a priority for this audit. VoiceOver testing on MacOS should hopefully include some crossover with iOS VoiceOver users. Mobile app usage is presumably more of a focus than web site usage.

Post Report and Issues in the Open

Our expectation is that any generated accessibility report should be posted in the open to the WordPress community and should not require me (or any other Automattician) as a gatekeeper. Issues should be posted with useful context to GitHub, with a focus on issue cause and reproducibility. Solutions or approaches for fixes can be posted if there is time in the scope or if the problem is especially complex.

Member

tofumatt commented Oct 10, 2018

This is what I've been sending out to companies I've been approaching for this work, by the way. It may help other folks involved in Accessibility, I'm not sure. At the very least I'm posting it here for transparency 🙂


Requirements

A series of accessibility tests to be done using the Gutenberg editor on a WordPress website using WP-Admin. These tests should include, at minimum, the following test flows:

Core Publishing Flow

  • Create a new post
  • Add common content (paragraph, image, list, HTML, and table content; see “Using Blocks” below)
  • Publish a post immediately and post is made as intended
  • Schedule a post for the future using date picker
  • Assign categories and tags to post
  • Assign a featured image to a post
  • Experiment with other Document-level settings, time-permitting

Using Blocks

These are the top-twenty most used blocks on WordPress.com and should be tested to find any accessibility issues not already filed in our list of accessibility issues:

  1. paragraph
  2. image
  3. heading
  4. list
  5. gallery
  6. quote
  7. embed/youtube
  8. html
  9. separator
  10. more
  11. cover-image
  12. shortcode
  13. button
  14. table
  15. spacer
  16. embed
  17. block
  18. embed/twitter
  19. columns
  20. code

These blocks should be tested with assistive technologies to ensure there are no blockers to their usage, and that their block-level settings are accessible.

Classic Editor

We should verify that, compared to the Classic Editor, there are not notable regressions with regard to accessibility. Creating a post with the same kind of content one can create in a barebones Classic Editor context should be doable with Gutenberg. Gutenberg having zero regressions in editing experience compared to the Classic Editor is my ultimate goal for saying we can recommend it as the default editing experience for users of assistive technologies.

Assistive Technologies to Consider

Because of the limited timeline available and the prioritisation of actionability over extreme thoroughness, I would prefer the top two-to-three browser+screenreader combinations to be used for testing. Using more if time allows is absolutely accessible, but I would prefer starting with the most-used combinations. From my understanding this is:

  • JAWS with Internet Explorer/Edge (Windows)
  • NVDA with Firefox (Windows)
  • VoiceOver with Safari (MacOS)
  • ZoomText

Mobile assistive technology is not a priority for this audit. VoiceOver testing on MacOS should hopefully include some crossover with iOS VoiceOver users. Mobile app usage is presumably more of a focus than web site usage.

Post Report and Issues in the Open

Our expectation is that any generated accessibility report should be posted in the open to the WordPress community and should not require me (or any other Automattician) as a gatekeeper. Issues should be posted with useful context to GitHub, with a focus on issue cause and reproducibility. Solutions or approaches for fixes can be posted if there is time in the scope or if the problem is especially complex.

@LukePettway

This comment has been minimized.

Show comment
Hide comment
@LukePettway

LukePettway Oct 10, 2018

@davisshaver Hello, and I totally agree with you. A thorough and unbiased audit is critical to ensuring that the maximum level of a11y is adhered to.

Not only is it important to make sure the editor is accessible but that there is also a foundation in place to ensure that modifications to the editor can also easily be made accessible. Obviously we can't enforce what people do on their own, but we can at least provide a foundation for people to implement that is accessible.

@tofumatt Also happy to assist in any way possible. I may have access to some pretty good resources for testing, specifically others with React and a11y experience. There are also some people on my team who specifically work with a11y in their day to day.

LukePettway commented Oct 10, 2018

@davisshaver Hello, and I totally agree with you. A thorough and unbiased audit is critical to ensuring that the maximum level of a11y is adhered to.

Not only is it important to make sure the editor is accessible but that there is also a foundation in place to ensure that modifications to the editor can also easily be made accessible. Obviously we can't enforce what people do on their own, but we can at least provide a foundation for people to implement that is accessible.

@tofumatt Also happy to assist in any way possible. I may have access to some pretty good resources for testing, specifically others with React and a11y experience. There are also some people on my team who specifically work with a11y in their day to day.

@bamadesigner

This comment has been minimized.

Show comment
Hide comment
@bamadesigner

bamadesigner Oct 10, 2018

@tofumatt Thank you for the update. I have a few follow up questions:

  1. Is the plan for the audit to be completed before 5.0 is released?
  2. If the audit returns issues, what is the plan for addressing them? Will the 5.0 release be delayed until they are addressed?
  3. Are there plans being made for how the project will test for accessibility going forward?

bamadesigner commented Oct 10, 2018

@tofumatt Thank you for the update. I have a few follow up questions:

  1. Is the plan for the audit to be completed before 5.0 is released?
  2. If the audit returns issues, what is the plan for addressing them? Will the 5.0 release be delayed until they are addressed?
  3. Are there plans being made for how the project will test for accessibility going forward?
@grahamarmfield

This comment has been minimized.

Show comment
Hide comment
@grahamarmfield

grahamarmfield Oct 10, 2018

@tofumatt If you want some testing by users with a variety of disabilities/impairments, I've done some work with a company called Hassell Inclusion, and they can organise that sort of testing. They're completely removed from the WordPress community. The company is run by Jonathan Hassell who used to be in charge of accessibility at the BBC.

grahamarmfield commented Oct 10, 2018

@tofumatt If you want some testing by users with a variety of disabilities/impairments, I've done some work with a company called Hassell Inclusion, and they can organise that sort of testing. They're completely removed from the WordPress community. The company is run by Jonathan Hassell who used to be in charge of accessibility at the BBC.

@davidberman

This comment has been minimized.

Show comment
Hide comment
@davidberman

davidberman Oct 11, 2018

@tofumatt I am also very concerned about WordPress accessibility and eager to help you succeed. I can have our organization perform a full WCAG-EM compliant accessibility audit to whatever standard level you prefer, including both technical and functional testing, and donate whatever portion of the work you need to fit your budget.

I would think we should be looking at both ATAG and WCAG AA, and perhaps WCAG 2.1 rather than 2.0. We could also provide recommendations and coaching to help you choose the best and most sustainable way to close any gaps discovered.

We are conveniently arms length from your development community, yet know WordPress well for many years. I hold all IAAP certifications and have led dozens of such audits on the WordPress platform, over many years, for both private and public sector. Please check out our bona fides at davidberman.com/about to decide how we can best be of assistance.

davidberman commented Oct 11, 2018

@tofumatt I am also very concerned about WordPress accessibility and eager to help you succeed. I can have our organization perform a full WCAG-EM compliant accessibility audit to whatever standard level you prefer, including both technical and functional testing, and donate whatever portion of the work you need to fit your budget.

I would think we should be looking at both ATAG and WCAG AA, and perhaps WCAG 2.1 rather than 2.0. We could also provide recommendations and coaching to help you choose the best and most sustainable way to close any gaps discovered.

We are conveniently arms length from your development community, yet know WordPress well for many years. I hold all IAAP certifications and have led dozens of such audits on the WordPress platform, over many years, for both private and public sector. Please check out our bona fides at davidberman.com/about to decide how we can best be of assistance.

@afercia

This comment has been minimized.

Show comment
Hide comment
@afercia

afercia Oct 11, 2018

Contributor

Using Blocks

I'm not sure the team going to make the audit should be instructed or have knowledge of what a "block" is, at least not initially. They should be in the same initial condition of users who are going to use Gutenberg for the first time, with no special instructions. In a second phase, when going technical, they will probably already know what a block is 🙂

Core Publishing Flow

The accessibility team agrees the main accessibility issue in Gutenberg is the overall user experience. I'd suggest to expand this part: it shouldn't be limited to the publihsing flow, instead it should include more tasks, typically the ones that require navigating through the UI and jumping through the main sections (top bar, editing area, sidebar, publish panel). This is more related to the general Gutenberg design, rather than technical implementation details on the various components. In fact, accessibility is design.

I'd like to propose a few tasks:

  • build a post with a large number of blocks, say 20 as a minimum
  • edit one of the paragraphs in the middle of the post
  • change its font size and font color, then edit again its content
  • add new blocks using the inserters
  • insert a link
  • insert a mention
  • change block type
  • change something in the top bar settings and then go back to the block that was edited
  • jump to the sidebar, switch to Document / Block settings and then go back to the block that was edited
  • edit the post permalink
  • make a reusable block
  • add and edit an existing reusable block

Assistive Technologies to Consider

@tofumatt I'm a bit surprised you're mentioning only assistive technologies and, specifically, only screen readers. Accessibility evaluations can't be limited to only screen reader testing. Accessibility is a way broader topic and it's not limited to disabilities or impairments. It's about making the web accessible to everyone.

Even if we want to limit accessibility to disabilities and impairments, they're just too many to count. Just to name a few: speech recognition software users, motor disabilities, low vision, cognitive impairments, seizure disorders, alternative input devices, attention deficit hyperactivity disorder, dyslexia, short-term memory loss, etc., etc. What about things that can't be classified as real "disabilities", for example temporary impairments, environmental impairments, ageing?

The human condition is pretty unstable, and I've suggest you to consider We're all just temporarily abled.

too many to count – image courtesy of Denis Boudreau

Contributor

afercia commented Oct 11, 2018

Using Blocks

I'm not sure the team going to make the audit should be instructed or have knowledge of what a "block" is, at least not initially. They should be in the same initial condition of users who are going to use Gutenberg for the first time, with no special instructions. In a second phase, when going technical, they will probably already know what a block is 🙂

Core Publishing Flow

The accessibility team agrees the main accessibility issue in Gutenberg is the overall user experience. I'd suggest to expand this part: it shouldn't be limited to the publihsing flow, instead it should include more tasks, typically the ones that require navigating through the UI and jumping through the main sections (top bar, editing area, sidebar, publish panel). This is more related to the general Gutenberg design, rather than technical implementation details on the various components. In fact, accessibility is design.

I'd like to propose a few tasks:

  • build a post with a large number of blocks, say 20 as a minimum
  • edit one of the paragraphs in the middle of the post
  • change its font size and font color, then edit again its content
  • add new blocks using the inserters
  • insert a link
  • insert a mention
  • change block type
  • change something in the top bar settings and then go back to the block that was edited
  • jump to the sidebar, switch to Document / Block settings and then go back to the block that was edited
  • edit the post permalink
  • make a reusable block
  • add and edit an existing reusable block

Assistive Technologies to Consider

@tofumatt I'm a bit surprised you're mentioning only assistive technologies and, specifically, only screen readers. Accessibility evaluations can't be limited to only screen reader testing. Accessibility is a way broader topic and it's not limited to disabilities or impairments. It's about making the web accessible to everyone.

Even if we want to limit accessibility to disabilities and impairments, they're just too many to count. Just to name a few: speech recognition software users, motor disabilities, low vision, cognitive impairments, seizure disorders, alternative input devices, attention deficit hyperactivity disorder, dyslexia, short-term memory loss, etc., etc. What about things that can't be classified as real "disabilities", for example temporary impairments, environmental impairments, ageing?

The human condition is pretty unstable, and I've suggest you to consider We're all just temporarily abled.

too many to count – image courtesy of Denis Boudreau

@pento

This comment has been minimized.

Show comment
Hide comment
@pento

pento Oct 16, 2018

Member

We sure have been on a bit of an emotional rollercoaster lately, haven't we? 🎢

First of all, thank you everyone for the work you do. 💖 I know it sometimes isn't easy to be an accessibility advocate, it can feel like a constant up hill battle. Please know that you're having an impact: from the personal (I've learned a lot about accessibility from the people who make up the WordPress Accessibility team), to the foundational (human-centered design is an essential facet of modern design practices).

I really like the idea of a professional audit, though I don't recall us ever doing one of these in WordPress, certainly not a condition for a release. I'd love to see something like it happen at some point. WordPress has always tried to get most of the way there on accessibility by sticking to common patterns and semantics, with the difference covered by key efforts of volunteers everyone on the Accessibility team doing testing and filing actionable bug reports. Gutenberg's move to being an entirely JavaScript-based application has made it harder to apply those patterns, but we can work together to establish new patterns, a new baseline.

We know that Gutenberg still needs more polishing—which we're doing!—and like everything, it will have issues that we'll continue to iterate on in the months and years ahead. Based on a lot of conversations, it appears that the most critical issues are already filed, many of them are fixed, and we'll continue fixing them as they come up.

There are a bunch of issues that haven't been milestoned yet (thank you especially to everyone who's submitted issues in the last week or so!). It'd be good to see a bug scrub to go through these issues and prioritise them. We can ensure the issues actively preventing people from using Gutenberg are fixed, and then focus on the issues that are about making it easier for people to use.

I've mentioned this before, but it bears repeating: WordPress 5.0 isn't the finish line, it's the starter pistol. Many of us in this thread have been working together on WordPress for years before Gutenberg started, and I hope we'll be working together on WordPress for years to come. We've seen many new features come to WordPress over the years, and none of them have stayed the same as when they were first released. We've come up against challenges that most other projects would write off as being insurmountable, but we've made them work.

WordPress' mission continues to be to democratise publishing. Accessibility is about making things work for people who historically have been incredibly marginalised. These aren't just complimentary, WordPress' mission inherently requires it to be accessible.

What we release in WordPress 5.0 isn't set in stone, we'll continue to fix it, rework it, learn from it, and make it better. Gutenberg is the foundation upon which we can build the next generation of the web, and the potential for improving the accessibility of the entire web is there, too. Every component, every block, every interface should be entirely accessible, and every plugin and theme should ultimately be able to take advantage of that. The Gutenberg generation should provide an accessible experience by default. Not only that, the colour contrast checker and the heading hierarchy warnings are good examples of how Gutenberg can make creating accessible content the default behaviour, too.

I've unlocked and re-opened this issue, and milestoned it for the future, but we can easily move it to a closer milestone at a later point. I do have a special request that we all try to stay on topic, and focus on productive conversation. I'd love to see us take what's been started here, and figure out a framework for a quality accessibility audit, something that can expand to cover the scope of Gutenberg phase 2, and beyond. We might not be there yet, but we will be. ❤️

Member

pento commented Oct 16, 2018

We sure have been on a bit of an emotional rollercoaster lately, haven't we? 🎢

First of all, thank you everyone for the work you do. 💖 I know it sometimes isn't easy to be an accessibility advocate, it can feel like a constant up hill battle. Please know that you're having an impact: from the personal (I've learned a lot about accessibility from the people who make up the WordPress Accessibility team), to the foundational (human-centered design is an essential facet of modern design practices).

I really like the idea of a professional audit, though I don't recall us ever doing one of these in WordPress, certainly not a condition for a release. I'd love to see something like it happen at some point. WordPress has always tried to get most of the way there on accessibility by sticking to common patterns and semantics, with the difference covered by key efforts of volunteers everyone on the Accessibility team doing testing and filing actionable bug reports. Gutenberg's move to being an entirely JavaScript-based application has made it harder to apply those patterns, but we can work together to establish new patterns, a new baseline.

We know that Gutenberg still needs more polishing—which we're doing!—and like everything, it will have issues that we'll continue to iterate on in the months and years ahead. Based on a lot of conversations, it appears that the most critical issues are already filed, many of them are fixed, and we'll continue fixing them as they come up.

There are a bunch of issues that haven't been milestoned yet (thank you especially to everyone who's submitted issues in the last week or so!). It'd be good to see a bug scrub to go through these issues and prioritise them. We can ensure the issues actively preventing people from using Gutenberg are fixed, and then focus on the issues that are about making it easier for people to use.

I've mentioned this before, but it bears repeating: WordPress 5.0 isn't the finish line, it's the starter pistol. Many of us in this thread have been working together on WordPress for years before Gutenberg started, and I hope we'll be working together on WordPress for years to come. We've seen many new features come to WordPress over the years, and none of them have stayed the same as when they were first released. We've come up against challenges that most other projects would write off as being insurmountable, but we've made them work.

WordPress' mission continues to be to democratise publishing. Accessibility is about making things work for people who historically have been incredibly marginalised. These aren't just complimentary, WordPress' mission inherently requires it to be accessible.

What we release in WordPress 5.0 isn't set in stone, we'll continue to fix it, rework it, learn from it, and make it better. Gutenberg is the foundation upon which we can build the next generation of the web, and the potential for improving the accessibility of the entire web is there, too. Every component, every block, every interface should be entirely accessible, and every plugin and theme should ultimately be able to take advantage of that. The Gutenberg generation should provide an accessible experience by default. Not only that, the colour contrast checker and the heading hierarchy warnings are good examples of how Gutenberg can make creating accessible content the default behaviour, too.

I've unlocked and re-opened this issue, and milestoned it for the future, but we can easily move it to a closer milestone at a later point. I do have a special request that we all try to stay on topic, and focus on productive conversation. I'd love to see us take what's been started here, and figure out a framework for a quality accessibility audit, something that can expand to cover the scope of Gutenberg phase 2, and beyond. We might not be there yet, but we will be. ❤️

@postphotos

This comment has been minimized.

Show comment
Hide comment
@postphotos

postphotos Oct 16, 2018

Contributor

Related #10537.

Contributor

postphotos commented Oct 16, 2018

Related #10537.

@cgrymala

This comment was marked as off-topic.

Show comment
Hide comment
@cgrymala

cgrymala Oct 16, 2018

WordPress 5.0 isn't the finish line, it's the starter pistol.

Ultimately, though, that starter pistol appears to only work for those without differing abilities. A starter pistol works in practice, because the sound is loud enough to create a wake in the air that deaf people can feel, and because it makes a flash/smoke that they can see; it makes a sound for blind people to hear. A starter pistol is accessible to everyone; Gutenberg does not appear to be.

By refusing to even consider pausing Gutenberg's release until after an a11y audit has been performed, you are effectively telling anyone that physically can't use Gutenberg that they don't matter to the WordPress community. That flies in the face of the WordPress community standards, is irresponsible, and, quite frankly, it's embarrassing and dangerous for a product that is so ubiquitous to take that kind of attitude toward so many other communities.

cgrymala commented Oct 16, 2018

WordPress 5.0 isn't the finish line, it's the starter pistol.

Ultimately, though, that starter pistol appears to only work for those without differing abilities. A starter pistol works in practice, because the sound is loud enough to create a wake in the air that deaf people can feel, and because it makes a flash/smoke that they can see; it makes a sound for blind people to hear. A starter pistol is accessible to everyone; Gutenberg does not appear to be.

By refusing to even consider pausing Gutenberg's release until after an a11y audit has been performed, you are effectively telling anyone that physically can't use Gutenberg that they don't matter to the WordPress community. That flies in the face of the WordPress community standards, is irresponsible, and, quite frankly, it's embarrassing and dangerous for a product that is so ubiquitous to take that kind of attitude toward so many other communities.

@tofumatt tofumatt removed their assignment Oct 16, 2018

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 16, 2018

Member

I created the "Needs Accessibility Feedback" label for issues that require someone with accessibility knowledge to chime in, but since there's nothing actionable on this issue I'm removing the label 😄

Member

tofumatt commented Oct 16, 2018

I created the "Needs Accessibility Feedback" label for issues that require someone with accessibility knowledge to chime in, but since there's nothing actionable on this issue I'm removing the label 😄

@pento

This comment has been minimized.

Show comment
Hide comment
@pento

pento Oct 16, 2018

Member

Nobody is saying anything of the sort, @cgrymala. As I mentioned, a professional accessibility audit has never been a requirement for any feature to be merged in WordPress' history. I'd certainly like to see an audit happen, but writing that a lack of audit is "irresponsible... embarrassing and dangerous" is the kind of hyperbole that gets in the way of constructive dialog.

I'll re-iterate my request from my earlier comment, please stay on topic, and focus on productive conversation.

More to the point, a lack of audit doesn't prevent us from fixing issues. If accessibility is important to you, if you (or someone you know) uses assistive technology, now is an excellent time to be testing and reporting bugs.

Member

pento commented Oct 16, 2018

Nobody is saying anything of the sort, @cgrymala. As I mentioned, a professional accessibility audit has never been a requirement for any feature to be merged in WordPress' history. I'd certainly like to see an audit happen, but writing that a lack of audit is "irresponsible... embarrassing and dangerous" is the kind of hyperbole that gets in the way of constructive dialog.

I'll re-iterate my request from my earlier comment, please stay on topic, and focus on productive conversation.

More to the point, a lack of audit doesn't prevent us from fixing issues. If accessibility is important to you, if you (or someone you know) uses assistive technology, now is an excellent time to be testing and reporting bugs.

@davisshaver

This comment was marked as off-topic.

Show comment
Hide comment
@davisshaver

davisshaver Oct 16, 2018

Contributor

@pento, I didn't notice anyone argue that an audit was precedent? That seems like a strawman argument, and based on this thread, precedent simply wasn't the genesis of the audit idea.

The audit was suggested by @tofumatt because there is good reason to believe that Gutenberg is not as accessible as it should be according to WordPress' own standards. Without an independent audit, or even just A audit, why do you expect the community to suddenly believe that Gutenberg is accessible? The lack of an audit isn't irresponsible and dangerous, but this style of decision-making and dictatorial management is.

Given the hurdles of authentication I don’t think that bringing this into core provides benefits for WP beyond what the community gets from the plugin. I don’t believe in its current state the benefit outweighs the cost, and we should err on the side of simplicity.

That's what @m said during the WP API debate. It's yet another example of the hypocrisies evident this recycle cycle, starting with the fact that the "deadlines aren't arbitrary" company refused to set a release date for 5.0, until they found one they liked at which point there was no room for debate. I love Gutenberg and I use it every day. It provides great value as a plugin, and according to Matt's own heuristic we should err on the side of simplicity until it's truly ready.

Accessibility is important to many people on this thread, who have offered time, support, and money to help us navigate this impasse. It is disingenuous for you to suggest that if we really cared, we'd be fixing issues.

Regarding the reasons provided for cancelling the audit:

an audit will not be actionable given our release timeline, because…
the audit will not affect release timing, so...
it would be more prudent to explore an audit on a less rushed timeline in the future

Even if the idea of a 3rd party audit is dead, implicit in the idea of #10537 would be the auditing of Gutenberg for accessibility means. The fact that Automattic is willing to release Gutenberg to Core with no self-imposed standards for accessibility is extremely problematic. Ultimately I am disheartened to be reminded that at the end of the day, Automattic's interests appear to be promoted above the community's when it comes to the development and release of Gutenberg.

Contributor

davisshaver commented Oct 16, 2018

@pento, I didn't notice anyone argue that an audit was precedent? That seems like a strawman argument, and based on this thread, precedent simply wasn't the genesis of the audit idea.

The audit was suggested by @tofumatt because there is good reason to believe that Gutenberg is not as accessible as it should be according to WordPress' own standards. Without an independent audit, or even just A audit, why do you expect the community to suddenly believe that Gutenberg is accessible? The lack of an audit isn't irresponsible and dangerous, but this style of decision-making and dictatorial management is.

Given the hurdles of authentication I don’t think that bringing this into core provides benefits for WP beyond what the community gets from the plugin. I don’t believe in its current state the benefit outweighs the cost, and we should err on the side of simplicity.

That's what @m said during the WP API debate. It's yet another example of the hypocrisies evident this recycle cycle, starting with the fact that the "deadlines aren't arbitrary" company refused to set a release date for 5.0, until they found one they liked at which point there was no room for debate. I love Gutenberg and I use it every day. It provides great value as a plugin, and according to Matt's own heuristic we should err on the side of simplicity until it's truly ready.

Accessibility is important to many people on this thread, who have offered time, support, and money to help us navigate this impasse. It is disingenuous for you to suggest that if we really cared, we'd be fixing issues.

Regarding the reasons provided for cancelling the audit:

an audit will not be actionable given our release timeline, because…
the audit will not affect release timing, so...
it would be more prudent to explore an audit on a less rushed timeline in the future

Even if the idea of a 3rd party audit is dead, implicit in the idea of #10537 would be the auditing of Gutenberg for accessibility means. The fact that Automattic is willing to release Gutenberg to Core with no self-imposed standards for accessibility is extremely problematic. Ultimately I am disheartened to be reminded that at the end of the day, Automattic's interests appear to be promoted above the community's when it comes to the development and release of Gutenberg.

@tofumatt

This comment has been minimized.

Show comment
Hide comment
@tofumatt

tofumatt Oct 16, 2018

Member

This thread is about:

  1. conducting an audit
  2. filing issues based on the audit
  3. posting its results as a report to the Make/Accessibility blog

We recognise the usefulness of an audit and that folks would like one performed. Please keep comments related to the process of:

  1. conducting the audit
  2. scoping of the audit
  3. results of the audit

I'm hiding off-topic comments that are re-hashing what's been said and highlighting the importance of an audit. I would like to see one conducted. We're all in agreement there. Please keep comments on topic, productive, and make sure they are providing new information/adding to the issue at hand.

Member

tofumatt commented Oct 16, 2018

This thread is about:

  1. conducting an audit
  2. filing issues based on the audit
  3. posting its results as a report to the Make/Accessibility blog

We recognise the usefulness of an audit and that folks would like one performed. Please keep comments related to the process of:

  1. conducting the audit
  2. scoping of the audit
  3. results of the audit

I'm hiding off-topic comments that are re-hashing what's been said and highlighting the importance of an audit. I would like to see one conducted. We're all in agreement there. Please keep comments on topic, productive, and make sure they are providing new information/adding to the issue at hand.

@aaronjorbin

This comment has been minimized.

Show comment
Hide comment
@aaronjorbin

aaronjorbin Oct 16, 2018

Member

@davisshaver Please remember the WordPress etiquette, especially:

Contributions to the WordPress open source project are for the benefit of the WordPress community as a whole, not specific businesses or individuals. All actions taken as a contributor should be made with the best interests of the community in mind.

The WordPress open source project is a volunteer-run community. Even in cases where contributors are sponsored by companies, that time is donated for the benefit of the entire open source community.

While @pento, @tofumatt, @m and others are donated by Automattic, they are working for the betterment of the project. Automattic is not the ones that are deciding when Gutenberg will be released.

Auditing can and will continue as noted by @pento:

now is an excellent time to be testing and reporting bugs.

Anything that can be done to help contribute to that testing and reporting process is beneficial. There are a number of older issues that could use testing to see if they are still an issue that don't require any specialized assistive technology to test. Doing so would be a great assistance and is in part what an independent audit would have provided.

Member

aaronjorbin commented Oct 16, 2018

@davisshaver Please remember the WordPress etiquette, especially:

Contributions to the WordPress open source project are for the benefit of the WordPress community as a whole, not specific businesses or individuals. All actions taken as a contributor should be made with the best interests of the community in mind.

The WordPress open source project is a volunteer-run community. Even in cases where contributors are sponsored by companies, that time is donated for the benefit of the entire open source community.

While @pento, @tofumatt, @m and others are donated by Automattic, they are working for the betterment of the project. Automattic is not the ones that are deciding when Gutenberg will be released.

Auditing can and will continue as noted by @pento:

now is an excellent time to be testing and reporting bugs.

Anything that can be done to help contribute to that testing and reporting process is beneficial. There are a number of older issues that could use testing to see if they are still an issue that don't require any specialized assistive technology to test. Doing so would be a great assistance and is in part what an independent audit would have provided.

@davisshaver

This comment was marked as off-topic.

Show comment
Hide comment
@davisshaver

davisshaver Oct 16, 2018

Contributor

I wasn't aware that being release lead of WordPress.org involved Matt dropping his fiduciary responsibility to Automattic. While you could argue that the conflict is minimal, or that Matt actively works to mitigate the conflict, the WordPress etiquette you quoted is a recommendation and not a description of the world... Ultimately conflict does exist.

Contributor

davisshaver commented Oct 16, 2018

I wasn't aware that being release lead of WordPress.org involved Matt dropping his fiduciary responsibility to Automattic. While you could argue that the conflict is minimal, or that Matt actively works to mitigate the conflict, the WordPress etiquette you quoted is a recommendation and not a description of the world... Ultimately conflict does exist.

@LukePettway

This comment has been minimized.

Show comment
Hide comment
@LukePettway

LukePettway Oct 16, 2018

I'm sure a lot of people are lurking this topic at the moment and I will say if you are interested and want to help but you haven't used assistive technology before (screen readers) join the discussion over at https://make.wordpress.org/chat/ in the accessibility channel. Feel free to reach out with a DM to me even. You might be surprised at how easy it is to get setup with NVDA/Voice Over/JAWS so that you can start testing things out.

Beyond that there are a lot of tools which are available for testing WCAG requirements:
Chrome aXe:
https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd

Contrast Checker for WCAG: https://chrome.google.com/webstore/detail/wcag-contrast-checker/plnahcmalebffmaghcpcmpaciebdhgdf?hl=en

Chrome A11y Tools: https://chrome.google.com/webstore/detail/accessibility-developer-t/fpkknkljclfencbdbgkenhalefipecmb?hl=en

There are 113 open issues with the accessibility label that are currently open:
https://github.com/WordPress/gutenberg/labels/Accessibility

Whether we may agree with the current decisions being made or not, I think we can all agree that there is still a lot of work that we can all take part in lending a hand to help solve. Even in an absence of an official audit the community still has the power to help deliver a more accessible experience.

LukePettway commented Oct 16, 2018

I'm sure a lot of people are lurking this topic at the moment and I will say if you are interested and want to help but you haven't used assistive technology before (screen readers) join the discussion over at https://make.wordpress.org/chat/ in the accessibility channel. Feel free to reach out with a DM to me even. You might be surprised at how easy it is to get setup with NVDA/Voice Over/JAWS so that you can start testing things out.

Beyond that there are a lot of tools which are available for testing WCAG requirements:
Chrome aXe:
https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd

Contrast Checker for WCAG: https://chrome.google.com/webstore/detail/wcag-contrast-checker/plnahcmalebffmaghcpcmpaciebdhgdf?hl=en

Chrome A11y Tools: https://chrome.google.com/webstore/detail/accessibility-developer-t/fpkknkljclfencbdbgkenhalefipecmb?hl=en

There are 113 open issues with the accessibility label that are currently open:
https://github.com/WordPress/gutenberg/labels/Accessibility

Whether we may agree with the current decisions being made or not, I think we can all agree that there is still a lot of work that we can all take part in lending a hand to help solve. Even in an absence of an official audit the community still has the power to help deliver a more accessible experience.

@amandarush

This comment has been minimized.

Show comment
Hide comment
@amandarush

amandarush Oct 16, 2018

This issue was filed to publicise my work on the accessibility audit, to communicate that I'd like it to happen, and to track the sharing of that audit in a blog post. It's not meant as a discussion point around what should be considered a release blocker for WordPress 5.0. I've noted folks' opinions on the matter, and I'll relay them to @m, who is the release lead for WordPress 5.0. My understanding is:

1. I don't have have veto over 5.0 shipping, but even if I did:

2. I'm still not convinced there are sufficient Accessibility issues that prevent a release

If the second point changes, I'll relay that info. I plan to be an advocate, but I don't set the timelines and I also don't have solid data around accessibility. That's the point of the audit: so we can speak from a place of hard data. 🙂

Thanks for all the input, but as this discussion has become quite off-topic from its intended scope, I'm locking this issue's conversation for now. I'll leave it open and post updates surrounding the audit.

Literally every person with disabilities who has tested Gutenberg, both recently and at the outset, has flagged blocking issues as to why it's not accessible. And user testing is just as important to accessibility as is WCAG 2.0 level AA compliance. Probably should say WCAG 2.1 AA at this point but WCAG 2.0 still gets you a lot closer to where you need to be. Saying that you don't have data isinaccurate at best.

amandarush commented Oct 16, 2018

This issue was filed to publicise my work on the accessibility audit, to communicate that I'd like it to happen, and to track the sharing of that audit in a blog post. It's not meant as a discussion point around what should be considered a release blocker for WordPress 5.0. I've noted folks' opinions on the matter, and I'll relay them to @m, who is the release lead for WordPress 5.0. My understanding is:

1. I don't have have veto over 5.0 shipping, but even if I did:

2. I'm still not convinced there are sufficient Accessibility issues that prevent a release

If the second point changes, I'll relay that info. I plan to be an advocate, but I don't set the timelines and I also don't have solid data around accessibility. That's the point of the audit: so we can speak from a place of hard data. 🙂

Thanks for all the input, but as this discussion has become quite off-topic from its intended scope, I'm locking this issue's conversation for now. I'll leave it open and post updates surrounding the audit.

Literally every person with disabilities who has tested Gutenberg, both recently and at the outset, has flagged blocking issues as to why it's not accessible. And user testing is just as important to accessibility as is WCAG 2.0 level AA compliance. Probably should say WCAG 2.1 AA at this point but WCAG 2.0 still gets you a lot closer to where you need to be. Saying that you don't have data isinaccurate at best.

@amandarush

This comment has been minimized.

Show comment
Hide comment
@amandarush

amandarush Oct 16, 2018

A lack of an audit in this case is irresponsible because, for starters, unlike WordPress in the past, a framework has been chosen that does not come with accessible components out of the box, and unlike with past WordPress, you are heavily modifying the document object model with Gutenberg, as well as removing things from the accessibility tree, and in the case of browsers, the DOM and the accessibility tree are literally how assistive technology retrieves information to interpret. Even if you leave assistive tech out of this, as Gutenberg currently stands, the cognitive load is enormous. The classic editor plugin is a completely unacceptable substitute. For starters, it's site-wide. So if a site's using Gutenberg, and I need to go in and do something with content, I can't do that on a per-user basis, and I can't install the classic editor plugin, do my work, and then uninstall it and expect that everything will behave as it should once the reversion back to Gutenberg takes place. And yes, I've tested this. The fact that accessibility is being left as an afterthought, again, is literally how WordPress got into the mess it was in back in 2011/2012 when the accessibility team first got off the ground. And the project is quite literally making the exact same mistake again, and asking us to trust that it'll eventually get this right, again. I'm sorry, but how many times do people with disabilities have to be sent to the back of the bus for the greater good? How many times to we have to be satisfied with a whole lot of pretty words about accessibility, about how we promise we're going to do better next time, only to find when it comes down to brass tacks that accessibility really isn't a priority?

amandarush commented Oct 16, 2018

A lack of an audit in this case is irresponsible because, for starters, unlike WordPress in the past, a framework has been chosen that does not come with accessible components out of the box, and unlike with past WordPress, you are heavily modifying the document object model with Gutenberg, as well as removing things from the accessibility tree, and in the case of browsers, the DOM and the accessibility tree are literally how assistive technology retrieves information to interpret. Even if you leave assistive tech out of this, as Gutenberg currently stands, the cognitive load is enormous. The classic editor plugin is a completely unacceptable substitute. For starters, it's site-wide. So if a site's using Gutenberg, and I need to go in and do something with content, I can't do that on a per-user basis, and I can't install the classic editor plugin, do my work, and then uninstall it and expect that everything will behave as it should once the reversion back to Gutenberg takes place. And yes, I've tested this. The fact that accessibility is being left as an afterthought, again, is literally how WordPress got into the mess it was in back in 2011/2012 when the accessibility team first got off the ground. And the project is quite literally making the exact same mistake again, and asking us to trust that it'll eventually get this right, again. I'm sorry, but how many times do people with disabilities have to be sent to the back of the bus for the greater good? How many times to we have to be satisfied with a whole lot of pretty words about accessibility, about how we promise we're going to do better next time, only to find when it comes down to brass tacks that accessibility really isn't a priority?

@kevinwhoffman

This comment has been minimized.

Show comment
Hide comment
@kevinwhoffman

kevinwhoffman Oct 17, 2018

Contributor

For at least the time being, Automattic has decided to forgo conducting an Accessibility audit on Gutenberg. The main reasons being:

  1. an audit will not be actionable given our release timeline, because…
  2. the audit will not affect release timing, so...
  3. it would be more prudent to explore an audit on a less rushed timeline in the future

@tofumatt @pento Can you clarify whether the timeline mentioned here refers to a release date of November 19, 2018 or January 22, 2019 as mentioned in the Proposed WordPress 5.0 Scope and Schedule?

The January 22, 2019 date would allow more than three months between today and the release of 5.0 to complete an audit and take action. The reasons above suggest that we cannot get an audit completed and significantly improve accessibility in three months time. If true, that is all the more reason to start the process now and respond to the audit by fixing as many issues as we can before 5.0 releases.

The idea that the timeline will become less rushed after 5.0 (when it's in the hands of real-world users who need it most) makes no sense at all.

Contributor

kevinwhoffman commented Oct 17, 2018

For at least the time being, Automattic has decided to forgo conducting an Accessibility audit on Gutenberg. The main reasons being:

  1. an audit will not be actionable given our release timeline, because…
  2. the audit will not affect release timing, so...
  3. it would be more prudent to explore an audit on a less rushed timeline in the future

@tofumatt @pento Can you clarify whether the timeline mentioned here refers to a release date of November 19, 2018 or January 22, 2019 as mentioned in the Proposed WordPress 5.0 Scope and Schedule?

The January 22, 2019 date would allow more than three months between today and the release of 5.0 to complete an audit and take action. The reasons above suggest that we cannot get an audit completed and significantly improve accessibility in three months time. If true, that is all the more reason to start the process now and respond to the audit by fixing as many issues as we can before 5.0 releases.

The idea that the timeline will become less rushed after 5.0 (when it's in the hands of real-world users who need it most) makes no sense at all.

@rcemory

This comment was marked as off-topic.

Show comment
Hide comment
@rcemory

rcemory Oct 17, 2018

Sites that were WCAG 2.0 level AA compliance but new content edited with Gutenberg will not be compliant when 5.0 is released unleashes a legal nightmare.

rcemory commented Oct 17, 2018

Sites that were WCAG 2.0 level AA compliance but new content edited with Gutenberg will not be compliant when 5.0 is released unleashes a legal nightmare.

@pento

This comment has been minimized.

Show comment
Hide comment
@pento

pento Oct 17, 2018

Member

Thank you everyone who's commented here so far and kept the discussion on topic. I really appreciate it: with so much going on around WordPress 5.0, it really helps me be able to put aside the time that this issue deserves. 🙂

@amandarush: You mentioned several concerns, I'll try to answer them all, but please let me know if I miss anything.

Literally every person with disabilities who has tested Gutenberg, both recently and at the outset, has flagged blocking issues as to why it's not accessible.

I'm truly grateful to everyone who's flagged these issues. I had thought that we'd done a reasonable job of addressing these issues as they came up, but clearly there's more work to do. When @tofumatt has been referring to "not having data", I believe he's looking at it from the perspective that we've fixed a lot of accessibility issues, we don't have a full idea current state of things, so we want to get a clearer picture of the severity of the remaining issues.

A lack of an audit in this case is irresponsible because, for starters, unlike WordPress in the past, a framework has been chosen that does not come with accessible components out of the box

Are you referring to React here? My understanding is that applications built with React are similarly accessible to any other library that dynamically updates the DOM. Provided we make appropriate use of aria-* attributes (as well as ensuring predictable keyboard focussing), that it should be usable with assistive technology. Is this not correct?

The classic editor plugin is a completely unacceptable substitute. For starters, it's site-wide. So if a site's using Gutenberg, and I need to go in and do something with content, I can't do that on a per-user basis, and I can't install the classic editor plugin, do my work, and then uninstall it and expect that everything will behave as it should once the reversion back to Gutenberg takes place.

Thank you for bringing up this use case, it's a good example of where the Classic Editor plugin currently fails. We can certainly add a per-user option.

And the project is quite literally making the exact same mistake again, and asking us to trust that it'll eventually get this right, again. I'm sorry, but how many times do people with disabilities have to be sent to the back of the bus for the greater good? How many times to we have to be satisfied with a whole lot of pretty words about accessibility, about how we promise we're going to do better next time, only to find when it comes down to brass tacks that accessibility really isn't a priority?

I'm sorry. Despite the best intentions of everyone on the Gutenberg team, we haven't done enough. I can honestly say that accessibility has always been a priority, but it hasn't been a high enough priority, and we've done a poor job of communicating where accessibility has been improved. I mentioned some of those improvements in my earlier comment, but those improvements are of no benefit if we haven't hit the baseline accessible experience.

Of course, an apology is no use without actions to make it right, so here's what I'm proposing. Right now, the intention is to prioritise and fix as many accessibility issues as possible. We've fixed hundreds already, and there are fixes already in progress for some of the open issues. We'll be testing, of course, but we do need the experience of people who use assistive technology to uncover bugs. Any testing and bug reporting you're able to make time for is greatly appreciated.

While I believe we can get the block editor to a state where it has an acceptable UX for assistive technology users by the time 5.0 is released, I do recognise there's a possibility we can't. Apart from the per-user setting you suggested, how else can we ensure the Classic Editor plugin is a reasonable option for folks to use until they're happy with the state of the block editor?

I would still like to see an independent accessibility audit happen, but without the rushed timeline of the 5.0 release. That's where this issue can be used to create the framework for a comprehensive audit. As problems are uncovered, they can be fixed in 5.0.x releases, there's certainly no need to wait for 5.1.

Member

pento commented Oct 17, 2018

Thank you everyone who's commented here so far and kept the discussion on topic. I really appreciate it: with so much going on around WordPress 5.0, it really helps me be able to put aside the time that this issue deserves. 🙂

@amandarush: You mentioned several concerns, I'll try to answer them all, but please let me know if I miss anything.

Literally every person with disabilities who has tested Gutenberg, both recently and at the outset, has flagged blocking issues as to why it's not accessible.

I'm truly grateful to everyone who's flagged these issues. I had thought that we'd done a reasonable job of addressing these issues as they came up, but clearly there's more work to do. When @tofumatt has been referring to "not having data", I believe he's looking at it from the perspective that we've fixed a lot of accessibility issues, we don't have a full idea current state of things, so we want to get a clearer picture of the severity of the remaining issues.

A lack of an audit in this case is irresponsible because, for starters, unlike WordPress in the past, a framework has been chosen that does not come with accessible components out of the box

Are you referring to React here? My understanding is that applications built with React are similarly accessible to any other library that dynamically updates the DOM. Provided we make appropriate use of aria-* attributes (as well as ensuring predictable keyboard focussing), that it should be usable with assistive technology. Is this not correct?

The classic editor plugin is a completely unacceptable substitute. For starters, it's site-wide. So if a site's using Gutenberg, and I need to go in and do something with content, I can't do that on a per-user basis, and I can't install the classic editor plugin, do my work, and then uninstall it and expect that everything will behave as it should once the reversion back to Gutenberg takes place.

Thank you for bringing up this use case, it's a good example of where the Classic Editor plugin currently fails. We can certainly add a per-user option.

And the project is quite literally making the exact same mistake again, and asking us to trust that it'll eventually get this right, again. I'm sorry, but how many times do people with disabilities have to be sent to the back of the bus for the greater good? How many times to we have to be satisfied with a whole lot of pretty words about accessibility, about how we promise we're going to do better next time, only to find when it comes down to brass tacks that accessibility really isn't a priority?

I'm sorry. Despite the best intentions of everyone on the Gutenberg team, we haven't done enough. I can honestly say that accessibility has always been a priority, but it hasn't been a high enough priority, and we've done a poor job of communicating where accessibility has been improved. I mentioned some of those improvements in my earlier comment, but those improvements are of no benefit if we haven't hit the baseline accessible experience.

Of course, an apology is no use without actions to make it right, so here's what I'm proposing. Right now, the intention is to prioritise and fix as many accessibility issues as possible. We've fixed hundreds already, and there are fixes already in progress for some of the open issues. We'll be testing, of course, but we do need the experience of people who use assistive technology to uncover bugs. Any testing and bug reporting you're able to make time for is greatly appreciated.

While I believe we can get the block editor to a state where it has an acceptable UX for assistive technology users by the time 5.0 is released, I do recognise there's a possibility we can't. Apart from the per-user setting you suggested, how else can we ensure the Classic Editor plugin is a reasonable option for folks to use until they're happy with the state of the block editor?

I would still like to see an independent accessibility audit happen, but without the rushed timeline of the 5.0 release. That's where this issue can be used to create the framework for a comprehensive audit. As problems are uncovered, they can be fixed in 5.0.x releases, there's certainly no need to wait for 5.1.

@pento

This comment has been minimized.

Show comment
Hide comment
@pento

pento Oct 17, 2018

Member

@kevinwhoffman: The timeline refers to the Nov 19 (or Nov 27 at the latest) release date. Pushing the release date to January 22 wouldn't give us much extra time: between WCUS (and the preparation required leading up to it), needing to organise a WordPress 4.9.9 release, then folks being on vacation over Christmas and New Year, we might gain an extra two full work weeks by pushing the release date by two months. An extra three months sounds like a generous amount, but it would give us very little actual results, and we'd still need a speedy audit, so we could be working on addressing it in November.

@rcemory: While your comment is kind of off topic, it's worth noting that this discussion is related to the block editor interface itself, not the content that it produces to display on the front end of your site. As with WordPress now, content produced in the block editor is accessible. In many cases, it's more accessible than before, as the block editor does a better job of adding appropriate aria-* labels, it warns when you're using colour combinations that aren't WCAG compliant, or when you're putting heading elements in the wrong order.

Member

pento commented Oct 17, 2018

@kevinwhoffman: The timeline refers to the Nov 19 (or Nov 27 at the latest) release date. Pushing the release date to January 22 wouldn't give us much extra time: between WCUS (and the preparation required leading up to it), needing to organise a WordPress 4.9.9 release, then folks being on vacation over Christmas and New Year, we might gain an extra two full work weeks by pushing the release date by two months. An extra three months sounds like a generous amount, but it would give us very little actual results, and we'd still need a speedy audit, so we could be working on addressing it in November.

@rcemory: While your comment is kind of off topic, it's worth noting that this discussion is related to the block editor interface itself, not the content that it produces to display on the front end of your site. As with WordPress now, content produced in the block editor is accessible. In many cases, it's more accessible than before, as the block editor does a better job of adding appropriate aria-* labels, it warns when you're using colour combinations that aren't WCAG compliant, or when you're putting heading elements in the wrong order.

@markkap

This comment was marked as off-topic.

Show comment
Hide comment
@markkap

markkap Oct 17, 2018

The classic editor plugin is a completely unacceptable substitute. For starters, it's site-wide. So if a site's using Gutenberg, and I need to go in and do something with content, I can't do that on a per-user basis, and I can't install the classic editor plugin, do my work, and then uninstall it and expect that everything will behave as it should once the reversion back to Gutenberg takes place.

Thank you for bringing up this use case, it's a good example of where the Classic Editor plugin currently fails. We can certainly add a per-user option.

This would be great but this is a concern that was raised more than a year ago, and wasn't addressed so far. Since in the current state of affairs once a post goes GB it can not be reverted to "plain" and then back to GB again while retaining all the relevant information.
Therefor the plan to have a user based option is just not realistic as it means that editors which opt out of GB will have a hard time editing content created by authors using GB.

markkap commented Oct 17, 2018

The classic editor plugin is a completely unacceptable substitute. For starters, it's site-wide. So if a site's using Gutenberg, and I need to go in and do something with content, I can't do that on a per-user basis, and I can't install the classic editor plugin, do my work, and then uninstall it and expect that everything will behave as it should once the reversion back to Gutenberg takes place.

Thank you for bringing up this use case, it's a good example of where the Classic Editor plugin currently fails. We can certainly add a per-user option.

This would be great but this is a concern that was raised more than a year ago, and wasn't addressed so far. Since in the current state of affairs once a post goes GB it can not be reverted to "plain" and then back to GB again while retaining all the relevant information.
Therefor the plan to have a user based option is just not realistic as it means that editors which opt out of GB will have a hard time editing content created by authors using GB.

@grahamarmfield

This comment has been minimized.

Show comment
Hide comment
@grahamarmfield

grahamarmfield Oct 17, 2018

The comments from @amandarush are powerful and from the heart. And, I know, based on the past experience of trying to advance accessibility within WordPress.

Since I have been involved in the Accessibility Team there have been 4 (I think) major functionality upgrades in the admin area: Custom menu builder, Media manager, Customiser, and now Gutenberg.

As originally conceived and built all of these components were pretty much (or totally) inaccessible to screen reader users and sighted keyboard users. In my view this is due to accessibility not being thought about at all at the graphic design and UX stages. After the Accessibility Team have cried foul, accessibility improvements have been made to all 4, but the underlying design and UX remain largely the same.

So I share her view that WordPress keeps making the same mistake over and over again. And sadly I share her scepticism when we hear "Accessibility is really important to us". It's very frustrating, and one of the reasons I seldom contribute to WP anymore.

It seems unlikely now that accessibility concerns will halt the push to release WP5.0 in November, which of course is a huge let down after the list of accessibility 'blockers' was first drafted in the spring of this year. And the oft-quoted desire for WP to democratise the web, and the promise not to release any functionality that is not WCAG2.0 compliant.

But I think that an independent accessibility review - involving not just testing for WCAG2.1, but also proper user testing - would be a sensible thing to do now. At least that way we all would know where we stand, and appropriate focus can be given to prioritising and fixing the accessibility defects/issues in future releases.

In response to this from @pento

Are you referring to React here? My understanding is that applications built with React are similarly accessible to any other library that dynamically updates the DOM. Provided we make appropriate use of aria-* attributes (as well as ensuring predictable keyboard focussing), that it should be usable with assistive technology. Is this not correct?

Yes, that's true, but the devs need to understand how to make their components accessible in design, semantics, operation, focus management, and with the correct use of ARIA. Sadly, this seldom happens in my experience.

grahamarmfield commented Oct 17, 2018

The comments from @amandarush are powerful and from the heart. And, I know, based on the past experience of trying to advance accessibility within WordPress.

Since I have been involved in the Accessibility Team there have been 4 (I think) major functionality upgrades in the admin area: Custom menu builder, Media manager, Customiser, and now Gutenberg.

As originally conceived and built all of these components were pretty much (or totally) inaccessible to screen reader users and sighted keyboard users. In my view this is due to accessibility not being thought about at all at the graphic design and UX stages. After the Accessibility Team have cried foul, accessibility improvements have been made to all 4, but the underlying design and UX remain largely the same.

So I share her view that WordPress keeps making the same mistake over and over again. And sadly I share her scepticism when we hear "Accessibility is really important to us". It's very frustrating, and one of the reasons I seldom contribute to WP anymore.

It seems unlikely now that accessibility concerns will halt the push to release WP5.0 in November, which of course is a huge let down after the list of accessibility 'blockers' was first drafted in the spring of this year. And the oft-quoted desire for WP to democratise the web, and the promise not to release any functionality that is not WCAG2.0 compliant.

But I think that an independent accessibility review - involving not just testing for WCAG2.1, but also proper user testing - would be a sensible thing to do now. At least that way we all would know where we stand, and appropriate focus can be given to prioritising and fixing the accessibility defects/issues in future releases.

In response to this from @pento

Are you referring to React here? My understanding is that applications built with React are similarly accessible to any other library that dynamically updates the DOM. Provided we make appropriate use of aria-* attributes (as well as ensuring predictable keyboard focussing), that it should be usable with assistive technology. Is this not correct?

Yes, that's true, but the devs need to understand how to make their components accessible in design, semantics, operation, focus management, and with the correct use of ARIA. Sadly, this seldom happens in my experience.

@aardrian

This comment has been minimized.

Show comment
Hide comment
@aardrian

aardrian Oct 18, 2018

If/as Automattic goes down the path of an audit, it may be worth not just identifying critical flows / user journeys, but also common widgets to review. If these widgets can be identified and audited alone, they can also form the basis of an accessible pattern library. This can help mitigate accessibility risks as more features are built using those patterns.

aardrian commented Oct 18, 2018

If/as Automattic goes down the path of an audit, it may be worth not just identifying critical flows / user journeys, but also common widgets to review. If these widgets can be identified and audited alone, they can also form the basis of an accessible pattern library. This can help mitigate accessibility risks as more features are built using those patterns.

@postphotos

This comment has been minimized.

Show comment
Hide comment
@postphotos

postphotos Oct 18, 2018

Contributor

it may be worth not just identifying critical flows / user journeys, but also common widgets to review. If these widgets can be identified and audited alone, they can also form the basis of an accessible pattern library. This can help mitigate accessibility risks as more features are built using those patterns.

@aardrian That's an interesting starting point that may already exist that will be required for the audit that we may be able to use now to action on: I haven't seen user journeys for Gutenberg, and that is probably the most obvious starting point to starting to scope said audit.

@karmatosed @jwold - Do you know if said flows/journeys for using Gutenberg already exist? Perhaps a list of "widgets" or "patterns" as described here?

I think while this general thread has a lot of emotion (and I agree Gutenberg must be usable by all and ship when ready) - I'd love to find common ground and next steps. I'd love to shift the conversation places to identifying where we're ready to action.

Said user flow would also give us a task list of places to audit for developers, testers and bug hunters to start dividing up and looking for things to report.

Contributor

postphotos commented Oct 18, 2018

it may be worth not just identifying critical flows / user journeys, but also common widgets to review. If these widgets can be identified and audited alone, they can also form the basis of an accessible pattern library. This can help mitigate accessibility risks as more features are built using those patterns.

@aardrian That's an interesting starting point that may already exist that will be required for the audit that we may be able to use now to action on: I haven't seen user journeys for Gutenberg, and that is probably the most obvious starting point to starting to scope said audit.

@karmatosed @jwold - Do you know if said flows/journeys for using Gutenberg already exist? Perhaps a list of "widgets" or "patterns" as described here?

I think while this general thread has a lot of emotion (and I agree Gutenberg must be usable by all and ship when ready) - I'd love to find common ground and next steps. I'd love to shift the conversation places to identifying where we're ready to action.

Said user flow would also give us a task list of places to audit for developers, testers and bug hunters to start dividing up and looking for things to report.

@aardrian

This comment has been minimized.

Show comment
Hide comment
@aardrian

aardrian Oct 18, 2018

@postphotos , I think @afercia outlined a good set of flows above in #10318 (comment).

aardrian commented Oct 18, 2018

@postphotos , I think @afercia outlined a good set of flows above in #10318 (comment).

@postphotos

This comment has been minimized.

Show comment
Hide comment
@postphotos

postphotos Oct 18, 2018

Contributor

@aardrian Agreed, that's a great place to start! 👍 Thanks for recalling that.

My ask still stands - I'd love to know, beyond that initial critical flow if there's additional work that's already been put into user journeys as it relates to features as it'll help validate the work we're doing here. There may be other items we (this group) might want to add @afercia's initial list, or sequence past an initial audit. 😄

For example, we've not called out the Content Structure feature, undo/redo, etc. or viewing a post, and those tasks we might agree are important and helpful for users to be able to use - the things that we agree make Gutenberg incredibly useful and should be accessible. It'd be great to find consensus actively if we can.

Contributor

postphotos commented Oct 18, 2018

@aardrian Agreed, that's a great place to start! 👍 Thanks for recalling that.

My ask still stands - I'd love to know, beyond that initial critical flow if there's additional work that's already been put into user journeys as it relates to features as it'll help validate the work we're doing here. There may be other items we (this group) might want to add @afercia's initial list, or sequence past an initial audit. 😄

For example, we've not called out the Content Structure feature, undo/redo, etc. or viewing a post, and those tasks we might agree are important and helpful for users to be able to use - the things that we agree make Gutenberg incredibly useful and should be accessible. It'd be great to find consensus actively if we can.

@mor10

This comment has been minimized.

Show comment
Hide comment
@mor10

mor10 Oct 18, 2018

An additional benefit to doing an audit is contribution back to the larger React project. React has its own accessibility issues, and an audit of Gutenberg may well identify both issues and solutions that relate to and can help React core.

mor10 commented Oct 18, 2018

An additional benefit to doing an audit is contribution back to the larger React project. React has its own accessibility issues, and an audit of Gutenberg may well identify both issues and solutions that relate to and can help React core.

@leepeterson

This comment has been minimized.

Show comment
Hide comment
@leepeterson

leepeterson Oct 18, 2018

Well, this is exciting news: https://make.wordpress.org/core/2018/10/18/regarding-accessibility-in-gutenberg/

Keyboard shortcuts for region and block navigation, slash commands, focusable toolbar and sidebar improvements from improved ARIA landmarks and roles, automated end-to-end tests, and more.

leepeterson commented Oct 18, 2018

Well, this is exciting news: https://make.wordpress.org/core/2018/10/18/regarding-accessibility-in-gutenberg/

Keyboard shortcuts for region and block navigation, slash commands, focusable toolbar and sidebar improvements from improved ARIA landmarks and roles, automated end-to-end tests, and more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment