Web map tools accessibility evaluation (WCAG 2.1)

[Malvoz]

I have manually evaluated the accessibility support of the reviewed web map tools. While I don't consider the evaluation report complete, my intention is to open bug reports to get the attention of the web map tools' authors. I'm not an accessibility expert, though I have studied WAI-ARIA for some time, and have read the WCAG Evaluation Methodology. It hasn't been an easy task to interpret all the WCAG documentation, so I'd appreciate any feedback.

Here it is: web-maps-wcag-evaluation.

/cc @Maps4HTML/contributors

[prushforth]

My impression is that this type of analysis could form a good argument that maps should be standardized, because of variation, often negative, in the accessibility of the maps. Of course, some of this variation is likely due to how we have applied the library in question; not always. However, that still doesn't negate the argument that HTML integration should and would iron out these variations.

Still looking at this, but it might be a good analysis to present at the workshop, if you're willing.

[nchan0154]

Hi @Malvoz, excellent work here! I'm working on a longer response to this but it's taking me a while, so I'm just commenting to let you know that your hard work has not gone unnoticed. 😂

[AmeliaBR]

This is great, @Malvoz! (I mean, it's not great for the web that the results are so disappointing, but it's great to have it written down. As Peter says, it's a good argument for “the status quo isn't good enough” when it comes to web map standardization.)

Have you contacted any of the teams responsible for these projects to point them to it?

About the Maps for the Web workshop:
I'd asked @nchan0154 to do a presentation on the “state of accessibility of web map widgets”, based on the reviews she did for the Use Cases report. But she rightfully pointed out that this new review of yours is more comprehensive. She said she'd be happy to defer to you if you'd be interested in taking over that presentation slot, but another option is for you two to collaborate to prepare the info for a single presentation.

Get in touch with me at amelia.bellamy.royds@gmail.com to let me know what you'd be interested in & we can connect you all up.

[Malvoz]

@AmeliaBR Sorry for the delayed response, unfortunately I wont be able to make presentations - @nchan0154 please feel free to refer to/use the evaluation report as you see fit. You'll do great!

Have you contacted any of the teams responsible for these projects to point them to it?

I've opened a few Leaflet issues already, they're linked to (marked "[Leaflet issue]") in the report, but haven't yet pointed them to the report itself. I thought I'd ask for feedback before doing so, and before contacting the other teams.

@prushforth #10 (comment)

this type of analysis could form a good argument that maps should be standardized, because of variation, often negative, in the accessibility of the maps.

Indeed.

Of course, some of this variation is likely due to how we have applied the library in question

I don't think that's the case in our particular implementation, if anything we've enhanced the accessibility, e.g.: Maps4HTML/HTML-Map-Element-UseCases-Requirements#200, perhaps something that many authors wouldn't have done themselves.

[nchan0154]

Hi Malvoz, thank you for taking the initiative to work on this! I am sorry to hear that you won't be able to make presentations, thank you for allowing me to refer to your notes thus far, I hope I will do us all proud 😂 I have some corrections and additional thoughts below.

1.1.1 Non-text content

Google Maps Embed: The lack of alt attributes on the zoom control's is overriden by the aria-label on button control. (but the Google graphic is still unlabelled!)

Google Maps API: Same as above for the zoom and full screen controls, they are both labelled.

1.1.3 Info & Relationships

MapKit JS (Apple Maps) API: I am seeing that the zoom in and out buttons have the aria-disabled="true" attribute applied correctly.

MapBox GL JS API: Has an aria-label of "Map" which does identify the map container's purpose.

TomTom Maps SDK for Web: Does have an aria-label of map on the map container, but it is not the container that receives focus, so a NVDA user might hear all the text in the container, and then a "Map" label a little later.

1.4.3 Contrast (Minimum)

I believe it is valuable to evaluate the contrast of the default map controls/content as often times they cannot be changed by the developer.

Web map tool	Result	Notes
Google Maps embed	Fail	View larger map button has a contrast ratio of 3.79. Some map text (ie. ocean labels) do not mean the minimum contrast ratio.
Google Maps Platform API	Fail	Same as above
Bing Maps embed	Fail	Some map labels (ie. bodies of water, district labels) do not meet the minimum contrast ratio.
Bing Maps Control API	Fail	Same as above
MapKit JS (Apple Maps) API	Fail	Some map labels (ie. bodies of water, minor street labels) do not meet the minimum contrast ratio.
Open Street Map Tiles (used by Leaflet JS API/OpenStreetMap embed/Open layers)	Fail	Some map labels (ie. regional/districts) do not meet the minimum contrast ratio.
MapBox Studio embed	Pass	Pass
MapBox GL JS API	Pass	Same as above
TomTom Maps SDK for Web	Fail	Some map labels (ie. buildings) do not meet the minimum contrast ratio.

1.4.11 Non-text Contrast

All the default maps fail this one. I recognize the difficulty of making a map where each discrete element (country boundaries, bodies of water, roads etc) remain in sharp contrast to eachother at all levels of zoom while remaining balanced and having a clear hierarchy amongst important shapes versus less important shapes. I think an important mitigating factor of this is having higher levels of zoom show distict borders between shapes, and offering some support for high contrast mode.

I'm not sure how this fits into your work, but just thought I'd throw it out there.

2.4.3 Focus Order

Fantastic work with these charts, these are great!

3.2.5 Change on Request

This success criterion covers link behavior. It is often not recommended to open links in a new window, but in this case, since user context on the map may be lost, it is understandable to make the link open in a new window, but ideally it should come with a warning

Web map tool	Result	Notes
Google Maps embed	Fail	Both the 'View Larger Map' link and the 'Terms of Use' link open in new windows without warning. The 'View Larger Map' button is especially unexpected as stylistically, it resembles a button, not a link.
Google Maps Platform API	Fail	'Terms of Use' link opens in new window without warning. The Google logo link contains the screen reader label "Open this area in Google Maps", which does give the user sufficient warning, as it may open in the native Maps app on mobile.
Bing Maps embed	Fail	Both the Bing Maps logo and the terms link open in new windows without warning.
Bing Maps Control API	Fail	Same as above
MapKit JS (Apple Maps) API	Fail	Legal link is incorrectly marked up as a button, opens in new tab without warning.
Leaflet JS API	Pass	No external links.
OpenStreetMap embed		'Report a problem' and 'OpenStreetMap' link open in a new tab without warning.
OpenLayers API		'OpenStreetMap' link opens in a new tab without warning.
MapBox Studio embed	Pass	Mapbox logo link opens in a new tab without warning
MapBox GL JS API	Pass	Same as above
TomTom Maps SDK for Web	Pass	No links

I was able to reproduce and confirm all the additional issues that you reported. I wasn't sure where to place this but I noticed in the Mapbox Studio embed, the autocomplete suggestions for the search bar are not announced to screen reader users, it just reads the content of the search input over and over as you navigate the suggestions.

[Malvoz]

Thanks for the thorough review Nic!

1.1.1 Non-text content

Google Maps Embed: The lack of alt attributes on the zoom control's is overriden by the aria-label on button control. (but the Google graphic is still unlabelled!)

Google Maps API: Same as above for the zoom and full screen controls, they are both labelled.

I understand that if the controls did not have labels/accessible names then it'd be imperative that the image elements have appropriate alt attributes (e.g. alt="zoom in" role="none|presentation"), however the controls do have accessible names and I'm not failing those, I am however failing the images as I worry that in some reader modes a screen reader could redundantly announce "graphic" for example, unless they're explicitly hidden (empty alt or aria-hidden="true") thus I chose to fail those images.

So strictly speaking the images should be explicitly hidden from screen readers I think, although most screen readers may be smart enough not to announce them in this scenario. Are you suggesting it's safe to not fail those images?

1.1.3 Info & Relationships

MapKit JS (Apple Maps) API: I am seeing that the zoom in and out buttons have the aria-disabled="true" attribute applied correctly.

You're right, I will correct this. Thanks!

MapBox GL JS API: Has an aria-label of "Map" which does identify the map container's purpose.

That's true, if it didn't I would have failed the element for missing name in 4.1.2 Name, Role, Value. However for SC 1.1.3 I don't think an aria-label (of "Map") is sufficient to convey the "web map's semantic structure as a distinct piece of content", I would, in addition to the aria-label, expect 1) an approparite role, e.g. region, and 2) the element to enclose all its components.

TomTom Maps SDK for Web: Does have an aria-label of map on the map container, but it is not the container that receives focus, so a NVDA user might hear all the text in the container, and then a "Map" label a little later.

Yes, the failure is failing to convey the "web map's semantic structure as a distinct piece of content".

1.4.3 Contrast (Minimum)

Nice job! I'll incorporate this, thanks!

1.4.11 Non-text Contrast

All the default maps fail this one. I recognize the difficulty of making a map where each discrete element (country boundaries, bodies of water, roads etc) remain in sharp contrast to eachother at all levels of zoom while remaining balanced and having a clear hierarchy amongst important shapes versus less important shapes. I think an important mitigating factor of this is having higher levels of zoom show distict borders between shapes, and offering some support for high contrast mode.
I'm not sure how this fits into your work, but just thought I'd throw it out there.

Conforming to this SC is not as straight forward and not as easily actionable as other SCs, perhaps we can add this as an item (to support high contrast mode) in the "Substantial accessibility issues not yet mapped/added to Success Criteria" section for now?

2.4.3 Focus Order

Fantastic work with these charts, these are great!

I used Microsoft's Accessibility Insights for Web to highlight the tab stops, pretty cool indeed!

3.2.5 Change on Request

Thanks I'll see about incorporating this too!

OpenStreetMap embed		'Report a problem' and 'OpenStreetMap' link open in a new tab without warning.
OpenLayers API		'OpenStreetMap' link opens in a new tab without warning.
MapBox Studio embed	Pass	Mapbox logo link opens in a new tab without warning
MapBox GL JS API	Pass	Same as above

Did you intend to "Fail" all of the above?

I was able to reproduce and confirm all the additional issues that you reported.

Again, thank you very much @nchan0154 for reviewing!

[nchan0154]

So strictly speaking the images should be explicitly hidden from screen readers I think, although most screen readers may be smart enough not to announce them in this scenario. Are you suggesting it's safe to not fail those images?

I see! I think it is safe to not fail these because the use of aria-label should override native naming, and is a technique that is often recommended by several experts (quotes pulled out)! While if it were me building it I'd probably slap an empty alt -just in case-, I wouldn't categorize it as a failure due to similar implementations across browsers, and if we did notice the buttons images being announced on any screen reader, I'd report it as a bug to the browser for not matching the spec.

When aria-label is used on a button, the contents of the attribute will override the contents inside the button as the accessible name. This means that, if you have an icon or even other text content inside your , that content will no longer be announced as part of the button’s name. VoiceOver will announce Menu, Button, and the devtools will now indicate that the accessible name of the button was provided by the aria-label attribute:

Authors should be aware that use of aria-label will override any native naming such as alt on images or label associated with a form field using the for attribute.

That's true, if it didn't I would have failed the element for missing name in 4.1.2 Name, Role, Value. However for SC 1.1.3 I don't think an aria-label (of "Map") is sufficient to convey the "web map's semantic structure as a distinct piece of content", I would, in addition to the aria-label, expect 1) an approparite role, e.g. region, and 2) the element to enclose all its components.

Fair enough! I noticed when doing NVDA Chrome, Mapbox Studio Embed and Mapbox GL announced the exact same thing, so it did not make sense for me to fail one and not the other. However, when I tested in Firefox (the more popular browser with NVDA), the GL version is significantly worse (just calls the container "clickable"). I agree with the universal fail result here, but also think with the crappiness of iframes labelling, the details of this failure criterion may vary significantly from browser to browser 🤔 This is a solid argument for why this needs some standardization 😂

Did you intend to "Fail" all of the above?

Yes, I forgot to update that column, ahaha. Please go based on the notes and not that column value!

[Malvoz]

I think it is safe to not fail these

Okay, I have removed those failures in Malvoz/web-maps-wcag-evaluation@c316932, thanks!

Mapbox Studio Embed and Mapbox GL announced the exact same thing, so it did not make sense for me to fail one and not the other.

I believe I failed MapBox GL in SC 1.1.3 Info & Relationships because it is lacking a "semantic" relationship between the "map component" and its controls (I realize iframe doesn't have an implicit role, but it is announced and together with title help identify the purpose of the thing):

<div id="mapbox-api-map">
  <canvas aria-label="Map">
  </canvas>

  <div class="mapboxgl-control-container">
    <!-- controls -->
  </div>
</div>

whereas MapBox Studio Embed has its "map component" and controls enclosed in a labelled iframe:

<iframe title="MapBox Studio">
  #document
  <!-- all the things -->
</iframe>

in this particular scenario, whether the lack of relationship between the map and the map components of MapBox GL has any significant implications on users I'm not sure, I suppose it could depend on the tool(s) the user is using to navigate the web page. It does have an appropriate aria-label, and is the first thing announced, so that's good, but that may not be the case once authors start customizing the map and its components. Not sure what to make of it, if you feel MapBox GL shouldn't fail in this instance then I'll remove the failure, and will have to review the other tools with the same failure, but not sure how to reason about it at this point. 😅

3.2.5 Change on Request

Did you intend to "Fail" all of the above?

Yes

Added SC 3.2.5 in Malvoz/web-maps-wcag-evaluation@75cc0dd, thanks!

[nchan0154]

Not sure what to make of it, if you feel MapBox GL shouldn't fail in this instance then I'll remove the failure, and will have to review the other tools with the same failure, but not sure how to reason about it at this point. 😅

Sorry, my comment was more of a runaway train of thought. Once I tested further, I have to agree with the assessment that all of them should fail as the output in NVDA/Firefox lacks a ton of context. I don't think it requires any further change :)

I would like to cite this evaluation in the presentation and create a shortlink to it, if that's okay with you? If so, how would you like to be referenced? @Malvoz

[Malvoz]

@nchan0154 Yes that's okay!

how would you like to be referenced?

As Robert Linder, if affiliation is desired it'd be Maps for HTML CG.

[Malvoz]

I'm happy to announce that MapBox have already started implementing accessibility improvements in mapbox/mapbox-gl-js#9991. 🎉

[Malvoz]

I commented on the Accessibility for Google Maps API issue. A recent response stated that "the engineering team has been making strides to improve the accessibility", with multiple accessibility issues fixed in release 3.43.1a and 3.43.2. 🎉

Additionally, it is now possible to pan the map display using a keyboard in Google Maps Embed.

Great news for users of Google Maps to end the year with. 😄

[erictheise]

I'm new to this project but wonder if this issue should be closed @Malvoz . New mapping libraries do come along – e.g., HERE Maps API for JavaScript is not listed in the .json file – and it seems evaluation would be periodic and never-ending.

[Malvoz]

@erictheise Welcome! And thank you for your comment.

The tools included in the evaluation are the reference tools currently listed in the Use Cases and Requirements report (per the defined evaluation scope). That list of reference tools may be expanded to include other tools in the future and as such other tools become subject to evaluation.

Periodic evaluation would be great, and especially important to keep any notes on accessibility issues up to date. I will consider conducting a new evaluation in the future, which is likely going to follow WCAG 2.2 or WCAG 3.0 guidelines (whichever guideline will have reached Candidate Recommendation status by then). But that'd be a separate document (probably in the same repo), and as this issue is about the WCAG 2.1 evaluation I think closing this issue was the right thing to do.

[erictheise]

Thanks for your thorough explanation @Malvoz. This effort has a lot of tendrils and I appreciate your taking the time to trace them out for me.