Sign language section: needs evidence, context, and explicit examples

[ashleemboyer]

This issue is related to the following section and its content at the time of writing:

3.10 Sign language

The utility of sign language in relation to WCAG presently stands as a Level AAA success criterion that is not as widely implemented within policy-related legislative frameworks, with compliancy up to Level AA appearing to be the most adopted. Thus, support for sign language is limited within time-based media such as online videos.

However, thanks to generative AI, it is now feasible to provide language translation services from text or symbol-based languages to sign language. Some websites and movie apps already offer a limited version of this capability. On the other hand, while this observation shows promise for the future use of sign language, the effectiveness of generative AI in translating sign language, coupled with the diverse localised variations in sign language across countries, poses challenges for providing a fully effective automated solution at this time.

My initial questions/comments (I do have more):

1. What is the evidence that generative AI can be used to translate text/symbols to sign language?
2. What are the "websites and movie apps" this section refers to?
3. There is no discussion of vital elements in many signed languages: facial expressions, body movement, simultaneous combinations of both.

[wittjeff]

I'm sold on the idea that generative AI could make ASL interpretation as well as Audio Description actually broadly readily achievable (where I don't believe either have been actually readily achievable up until now). I'm also convinced that AI could soon provide captions that are sufficiently accurate, and also possibly annotate other audio events as needed. But if we're going to assert that these new technologies are available and sufficient, they all need benchmarks with reasonable qualifying thresholds for success. I'd be happy to be involved in the creation of these benchmarking tools and usability studies, but they will need sponsorship -- financial and stakeholder buy-in.

[wittjeff]

Related but not sufficient human-signer corpus-gathering work from NVIDA

[wittjeff]

Related SAFE-AI (Stakeholders Advocating for Fair and Ethical AI in Interpreting) Task Force and Deaf-safe AI: A Legal Foundation for Ubiquitous Automatic Interpreting report

[ashleemboyer]

Related but not sufficient human-signer corpus-gathering work from NVIDA

Here’s the key part from this page (not sure if this is what you were referring to when you said “not sufficient”):

“While Signs currently focuses on hand movements and finger positions for each sign, ASL also incorporates facial expressions and head movements to convey meaning. The team behind Signs is exploring how these non-manual signals can be tracked and integrated in future versions of the platform.”

It’s also worth noting that the platform is aimed at people learning sign language, not fluent signers. Totally different use cases.

[ashleemboyer]

Related SAFE-AI (Stakeholders Advocating for Fair and Ethical AI in Interpreting) Task Force and Deaf-safe AI: A Legal Foundation for Ubiquitous Automatic Interpreting report

Will read these soon 🙂