report: reorganize a11y audit groups #7129

robdodson · 2019-01-31T21:06:47Z

Summary

Improve the categorization of accessibility audits.

Note: Also enables the accesskeys audit which was accidentally turned off in a previous PR.

Related Issues/PRs
fixes #7019

patrickhulce

nice LGTM outside the potential scoring policy

patrickhulce · 2019-01-31T22:36:00Z

lighthouse-core/config/default-config.js

        {id: 'valid-lang', weight: 1, group: 'a11y-language'},
-        {id: 'video-caption', weight: 4, group: 'a11y-describe-contents'},
-        {id: 'video-description', weight: 3, group: 'a11y-describe-contents'},
+        {id: 'audio-caption', weight: 4, group: 'a11y-audio-video'},


is the new sorting scheme alphabetical unless it's audio-caption? :)

I think I did that one first while I was still figuring out if the audits were sorted by id or group. Fixed

patrickhulce · 2019-01-31T22:37:24Z

lighthouse-core/config/default-config.js

@@ -369,42 +369,42 @@ const defaultConfig = {
      description: str_(UIStrings.a11yCategoryDescription),
      manualDescription: str_(UIStrings.a11yCategoryManualDescription),
      auditRefs: [
+        {id: 'accesskeys', weight: 3, group: 'a11y-navigation'},


@paulirish did we officially decide what our criteria for breaking changes was for v4 going forward? do we need to introduce this as weight: 0 to start or are we doing that starting v5?

do we need to introduce this as weight: 0 to start or are we doing that starting v5?

I think since this is not in the performance section it's less of a big deal (people might appreciate it, actually).

It was also a misunderstanding that disabled this audit, so we can treat the temporary weight: 0 as a bug that we're now fixing? :D

patrickhulce · 2019-01-31T22:38:08Z

lighthouse-core/config/default-config.js

-        {id: 'button-name', weight: 10, group: 'a11y-element-names'},
-        {id: 'bypass', weight: 10, group: 'a11y-describe-contents'},
+        {id: 'button-name', weight: 10, group: 'a11y-names-labels'},
+        {id: 'bypass', weight: 10, group: 'a11y-navigation'},
        {id: 'color-contrast', weight: 6, group: 'a11y-color-contrast'},


heh, is this really the only audit in the group?

It is! Sorry, it's an odd duck. But oh so important.

exterkamp

LGTM from a proto round trip perspective.

But if we are changing a11y scoring weights would that be a major change? And changing the JSON group/audit keys might be considered breaking, its not crazy, but its definitely a change to the JSON structure.

paulirish · 2019-02-05T22:33:27Z

Before / After:

(via lighthouse http://localhost:10200/a11y/a11y_tester.html)

paulirish · 2019-02-06T01:51:17Z

@meggynw as fyi

robdodson · 2019-02-06T18:33:25Z

I think all of the weights should be the same with the exception of accesskeys which was formerly listed as a manual audit. I could move it back to 0 weight and consider giving it a weight at a later, more breakage friendly time?

brendankenny · 2019-02-07T20:33:49Z

lighthouse-core/config/default-config.js

@@ -31,44 +31,44 @@ const UIStrings = {
  diagnosticsGroupTitle: 'Diagnostics',
  /** Description of the diagnostics section of the Performance category. Within this section are audits with non-imperative titles that provide more detail on the page's page load performance characteristics. Whereas the 'Opportunities' suggest an action along with expected time savings, diagnostics do not. Within this section, the user may read the details and deduce additional actions they could take. */
  diagnosticsGroupDescription: 'More information about the performance of your application.',
-  /** Title of the Accessibility category of audits. This section contains audits focused on making web content accessible to users with disabilities. Also used as a label of a score gauge; try to limit to 20 characters. */
+  /** Title of the Accessibility category of audits. This section contains audits focused on making web content accessible to all users. Also used as a label of a score gauge; try to limit to 20 characters. */


brendankenny · 2019-02-07T20:37:09Z

lighthouse-core/config/default-config.js

@@ -369,42 +369,42 @@ const defaultConfig = {
      description: str_(UIStrings.a11yCategoryDescription),
      manualDescription: str_(UIStrings.a11yCategoryManualDescription),
      auditRefs: [
+        {id: 'accesskeys', weight: 3, group: 'a11y-navigation'},


do we need to introduce this as weight: 0 to start or are we doing that starting v5?

I think since this is not in the performance section it's less of a big deal (people might appreciate it, actually).

It was also a misunderstanding that disabled this audit, so we can treat the temporary weight: 0 as a bug that we're now fixing? :D

robdodson · 2019-02-07T22:14:37Z

Do I need to run that special command to regenerate the sample json?

brendankenny · 2019-02-07T22:19:05Z

yeah, rerunning is probably a lot easier than dealing with the merge conflict. merge or rebase, delete or whatever any conflict in sample_v2.json, then rerun yarn update:sample-json

robdodson · 2019-02-08T00:02:07Z

Updated PTAL

brendankenny · 2019-02-08T01:27:15Z

there's a test snapshot that needs to be updated (it's in the travis logs, but you can run yarn run unit-cli:ci -u to automatically fix) and the smoke test expectations will need to be switched back to not a manual result (

lighthouse/lighthouse-cli/test/smokehouse/a11y/expectations.js

Lines 232 to 235 in 5c0483c

    
           'accesskeys': { 
        
             score: null, 
        
             scoreDisplayMode: 'manual', 
        
           },

)

robdodson · 2019-02-09T01:44:53Z

Updated PTAL

brendankenny

ok, a few more things, but otherwise LGTM.

Does anyone else want to have sign off on this?

brendankenny · 2019-02-12T21:25:38Z

lighthouse-cli/test/smokehouse/a11y/expectations.js

@@ -230,8 +230,12 @@ module.exports = [
        },
      },
      'accesskeys': {
-        score: null,
-        scoreDisplayMode: 'manual',
+        score: 0,


to fix the CI error, it looks like this might be

score: null, scoreDisplayMode: 'notApplicable',

?

But did something change with the audit since removal in #7020?

brendankenny · 2019-02-12T21:29:54Z

lighthouse-core/config/default-config.js

+  /** Title of the best practices section of the Accessibility category. Within this section are audits with descriptive titles that highlight common accessibility best practices. */
+  a11yBestPracticesGroupTitle: 'Best practices',
+  /* Description of the best practices section within the Accessibility category. Within this section are audits with descriptive titles that highlight common accessibility best practices. */
+  a11yBestPracticesGroupDescription: 'These are opportunities to improve the interpretation of your content by users in different locales.',


not clear what different locales have to do with duplicate-id/meta-refresh/meta-viewport?

also seems like for almost all of these we could remove the beginning "These are", but maybe we want to wait for the efforts looking at all descriptions in the report

different locales

ah, looks like this may be a c/p holdover from the "Internationalization and localization" group

paulirish

ditto on the bestpractices group description. but other than that... lgtm

brendankenny · 2019-02-20T02:44:04Z

lighthouse-cli/test/fixtures/a11y/a11y_tester.html

+    <section>
+      <button id="accesskeys1" accesskey="s">Foo</button>
+      <button id="accesskeys2" accesskey="s">Bar</button>
+    </section>


👍 👍 thanks for adding this!

robdodson requested review from brendankenny, patrickhulce, paulirish and exterkamp as code owners January 31, 2019 21:06

robdodson changed the title ~~Rename a11y audit categories. Fixes #7019~~ report: Rename a11y audit categories. Fixes #7019 Jan 31, 2019

patrickhulce reviewed Jan 31, 2019

View reviewed changes

brendankenny changed the title ~~report: Rename a11y audit categories. Fixes #7019~~ report: reorganize a11y audit groups Jan 31, 2019

exterkamp reviewed Feb 5, 2019

View reviewed changes

brendankenny reviewed Feb 7, 2019

View reviewed changes

robdodson force-pushed the a11y-reorg branch from c133138 to 004186d Compare February 8, 2019 00:01

robdodson force-pushed the a11y-reorg branch from 004186d to 9b7bbea Compare February 9, 2019 01:44

brendankenny requested changes Feb 12, 2019

View reviewed changes

paulirish approved these changes Feb 14, 2019

View reviewed changes

robdodson added 7 commits February 15, 2019 15:41

Rename a11y audit categories.

e3f7f16

Update samples

f0cc4fa

Update category test.

7cc15e9

Fix audit order.

f33ae38

Update sample json

4473ea7

Update tests.

e08dd7f

Fix busted description and test.

35759ac

robdodson force-pushed the a11y-reorg branch from 9b7bbea to 35759ac Compare February 15, 2019 23:53

update sample json

eeb2291

Update snapshot

c9443f8

brendankenny approved these changes Feb 20, 2019

View reviewed changes

brendankenny merged commit 0b94507 into master Feb 20, 2019

brendankenny deleted the a11y-reorg branch February 20, 2019 02:45

anil-alt mentioned this pull request Mar 27, 2019

feat: optional accessibility check with loadPage testiumjs/testium-driver-wd#43

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

report: reorganize a11y audit groups #7129

report: reorganize a11y audit groups #7129

robdodson commented Jan 31, 2019 •

edited by brendankenny

patrickhulce left a comment

patrickhulce Jan 31, 2019

robdodson Jan 31, 2019

patrickhulce Jan 31, 2019

brendankenny Feb 7, 2019

patrickhulce Jan 31, 2019

robdodson Jan 31, 2019

exterkamp left a comment

paulirish commented Feb 5, 2019

paulirish commented Feb 6, 2019

robdodson commented Feb 6, 2019

brendankenny Feb 7, 2019

brendankenny Feb 7, 2019

robdodson commented Feb 7, 2019

brendankenny commented Feb 7, 2019

robdodson commented Feb 8, 2019

brendankenny commented Feb 8, 2019

robdodson commented Feb 9, 2019

brendankenny left a comment

brendankenny Feb 12, 2019

brendankenny Feb 12, 2019

brendankenny Feb 12, 2019

brendankenny Feb 12, 2019

paulirish left a comment

brendankenny Feb 20, 2019

report: reorganize a11y audit groups #7129

report: reorganize a11y audit groups #7129

Conversation

robdodson commented Jan 31, 2019 • edited by brendankenny

patrickhulce left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

exterkamp left a comment

Choose a reason for hiding this comment

paulirish commented Feb 5, 2019

paulirish commented Feb 6, 2019

robdodson commented Feb 6, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robdodson commented Feb 7, 2019

brendankenny commented Feb 7, 2019

robdodson commented Feb 8, 2019

brendankenny commented Feb 8, 2019

robdodson commented Feb 9, 2019

brendankenny left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paulirish left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

robdodson commented Jan 31, 2019 •

edited by brendankenny