New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit accessibility check to WCAG failures, not best practices #1087

Closed
aardrian opened this Issue May 21, 2018 · 4 comments

Comments

Projects
None yet
3 participants
@aardrian

aardrian commented May 21, 2018

When running an online scan at sonarwhal.com, the results from aXe are presented as accessibility failures, but it is not clear what is a true WCAG violation and what is a best practice as defined by the aXe team.

By default, limit the output of Sonarwhal to only present WCAG failures, or flag items that are best practices as such. I understand that the results from aXe expose whether a failure is a best practice or not. The aXe browser plug-in makes that distinction, so aligning with the aXe browser plug-in would also be a bonus.

Alternatively, an option to categorize failures (or disable/enable) by WCAG level and/or best practice might be a nice feature for those reading these reports.

Example

If I run Delta.com through Sonarwhal (specifically the system unavailable page), the report returns the following messages (among others):

ERROR
Page must contain one main landmark
ERROR
Content should be contained in a landmark region

If I run aXe in my browser, the report excludes the first error and says (note use of should and best-practice):

Content should be contained in a landmark region
Issue tags: category: keyboard best-practice

Neither of the issues listed in the online scanner is a WCAG failure, but the must language can confuse users who do not know WCAG and are using the tool to prioritize remediation efforts.

Environment

Sonarwhal.com online scanner

URLs in this report

@patrickhlauke

This comment has been minimized.

patrickhlauke commented May 21, 2018

Strongly agree here. While it's perfectly fine (and in keeping with other non-accessibility related test) that not following best practice is flagged, when it comes to accessibility there needs to be a clear distinction between faliures (hard failures against normative WCAG 2 success criteria) and not following opinionated best practice suggestions

@molant

This comment has been minimized.

Member

molant commented May 21, 2018

Looks like there is a problem in aXe's documentation. I've opened this issue in their repo.

Our intention was to only run WCAG 2.0 Level A and AA so I'll update the rule to be explicit on that aspect.

Thanks for reporting this @aardrian!

@molant

This comment has been minimized.

Member

molant commented May 21, 2018

There's a PR now

@aardrian

This comment has been minimized.

aardrian commented May 22, 2018

I just ran Sonarwhal against the same Delta page, and it now comes back without any errors from aXe. Since I last ran it, the Delta page has added a lang attribute and a <main> element, so maybe I chose a poor example. I mean, who thought I would catch them the day before they deployed fixes?

@alrra alrra closed this in e1ae0f6 May 22, 2018

alrra added a commit that referenced this issue May 22, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment