-
Notifications
You must be signed in to change notification settings - Fork 124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider addition of a property for input controls to provide untranslated Unicode Braille Patterns (U+2800..U+28FF) #771
Comments
nvaccess/nvda#8340 proposes aria-braille-input. I wonder whether it might be better to have |
As a concrete use case for this feature, the Desmos four function and scientific calculators are able to accept mathematical input and display mathematical output in either Nemeth or UEB.
Currently, for this feature to work, we have to ask users to switch their input into "uncontracted 8-dot computer braille" (the terminology varies platform-to-platform). This is necessary because contractive Braille tables (which are very commonly used) will expand certain Braille cells as contractions and interfere with our ability to interpret the user input as Nemeth. We have found that asking users to switch their input table can be a significant hurdle; especially for students and teachers who may not be expert screen reader users. If we could hint to the screen reader that a particular input knows how to handle raw Braille characters, then users could enter Nemeth or UEB math into the calculator without having to switch their system-wide input table. |
This is very important for us at Benetech and the DIAGRAM Center as we are working currently on a web based digital math editor (called MathShare) and being able to have raw braille input into our editor will be crucial for those students using braille as their input modality into our editor. We have just received funding to continue to work on MathShare from Cisco Foundation and General Motors |
Just to write down what I said on today's call before it started formally. I'd be happy to spec this out for 1.3 (alongside the other braille properties) but I really don't know enough about input. So if the group can figure out how this can work, I'm happy to write it up and iterate. |
@jnurthen quick ping to put this on the agenda some time. |
To gather two specific comments from an early prototype at nvaccess/nvda#8340 @michaelDCurran wrote at nvaccess/nvda#8340 (comment)
@jcsteh wrote at nvaccess/nvda#8340 (comment)
|
And in response to @jcsteh's request
IIRC desmos and Pearson's AEE both use hacks to get raw braille input (of Nemeth Braille, I believe). I can imagine similar use cases for chemical braille and braille music. |
I think I agree with @jcsteh when he says in #771 (comment):
Also, if it is merely a hint to the screen reader that it should provide unicode braille as input, how are we supposed to ensure that no other characters are ending up in the text control? I assume the web author is supposed to deal with that, but it is easy to overlook such a thing. There should probably be a best practice that gives directions to do this. |
Sina's example: https://sinabahram.github.io/aria-playground/BrailleInput.html |
There are probably several ways this could be done, but after some brainstorming today, here's one possibility. <!--
Notes:
- ⠉⠕⠎ is "cos" short for cosine in Nemeth Math braille. Trying to keep the example short.
- The field label "formula" is intended to explicitly show that that the label attrs
should not be confused with or overloaded with values.
-->
<input aria-label="formula" aria-brailleinput="nemeth" value="⠉⠕⠎">
<!-- optionally with another aria-speechvalue attr if the app
wants to handle the explicit back translation -->
<input aria-label="formula" aria-speechvalue="cosine" aria-brailleinput="nemeth" value="⠉⠕⠎">
Privacy:
|
Currently this issue is planned for broader (but still informal) discussion in the ARIA Deep Dive call on December 10, 2020 at 9 AM Pacific. This time uses the same IRC and Zoom channel as the ARIA WG meeting, but is one hour earlier. |
Thanks for this @cookiecrook . A meta issue first, if I may. I fear this is a lot of potential scope creep, and I'm concerned all of the above phrases this issue in such a way that AT vendors will not want to implement it due to it being a lot of work. I suppose what I was trying to convey on the call was simply an element with no implications around speech or indicating translation tables or any of that. Simply a mechanism for the screen reader to pass through the raw braille dots entered into an edit field. Everything else needs to be their own respective issues it seems, and could be handled via application logic (though of course I would love for screen readers to help in that loop). I'm happy to respond to the above in detail and have all those conversations around specifics, but I first wish to ask if you feel it appropriate to simplify the scope of this request because currently it has callouts to speech and such, whereas I was advocating for just raw braille dots to be passed in if this attribute is set to true. If not, and you feel all of these issues need to be discussed together, then I'm delighted to dive in and begin responding. Thank you again for the writeup, and I can't wait to begin figuring this out. |
That's what the combination of two attributes would allow. Requiring The implementation would only become trickier if |
The December 10th discussion time is no longer going to work for the Apple engineers that wanted to attend. What about the following week, December 17th, or in January, if too many people will vacation early ahead of the December holiday? |
Either works for me in terms of dates, though I am trying to wrap most work up that week with the 17th. This is super-important, though, so I'm happy to make time for it. I also owe you answers to the above comment, and I very much have not forgotten that/am planning on still doing so. |
I changed the name of the issue because, in discussion, no one has asked for raw braille key events. AT vendors would likely object to providing raw braille display key events anyway. Instead, the need is to mark an input field as accepting untranslated unicode braille symbols, referred to in the Unicode spec as Braille Patterns (U+2800..U+28FF) such as dots124 (⠋), dots1234567 (⡿), dots12468 (⢫). |
Either date works for me as well.
Thanks for the clarifications.
Sam
…On Thu, Dec 3, 2020, 11:33 AM James Craig ***@***.***> wrote:
The December 10th discussion time is no longer going to work for the Apple
engineers that wanted to attend. What about the following week, December
17th, or in January, if too many people will vacation early ahead of the
December holiday?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#771 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABEIHUAQSH2RACUZHUFJZ63SS7DWFANCNFSM4FB2OXHA>
.
|
I can attend on December 17 to add in the perspective of what we've done so far at Desmos to support Braille input. I am glad some Apple engineers will be present to discuss this issue in more detail as we have had the most challenges making our Braille editor work consistently on iOS and the Mac. |
Thanks @sclower. If you're saying that you've gotten untranslated braille input working on other systems, are you able to share more details how you achieved that in the absence of a standard? If you're referring to other, unrelated braille issues, ping me offline or let's discuss those in other relevant issue threads. |
Thanks James.
Speaking for the Pearson equation editor, it requires the braille terminal
user select ASCII braille output, which is sent to a hidden input control
in the web page where the javascript intercepts the key events. The
javascript editor provides a mode switch to allow the user to select
between standard keyboard input, braille terminal input, and braille input
using the home row keys on the standard keyboard. The input handler then
knows how it should interpret the key events that are coming into the input
control, and respond to the corresponding braille patterns as Nemeth
input. However, it puts the burden on the user to make sure the input
device being used matches the input mode expected by the software.
The idea is that if there were a way to have the braille terminal send
U+28xx braille patterns, that could potentially simplify both the input
logic and the user experience.
Sam
…On Tue, Dec 8, 2020 at 1:31 AM James Craig ***@***.***> wrote:
Thanks @sclower <https://github.com/sclower>.
If you're saying that you've gotten untranslated braille input working on
other systems, are you able to share more details how you achieved that in
the absence of a standard?
If you're referring to other, unrelated braille issues, ping me offline or
let's discuss those in other relevant issue threads.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#771 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABEIHUATAKJK475IO7MVPSLSTXI4NANCNFSM4FB2OXHA>
.
|
Ah okay, so the user types ASCII braille (e.g. ⠉⠕⠎) which is sent to the webpage as standard keyboard input (e.g. "cos"). Then the web app translates to Nemeth ⠉⠕⠎ and interprets as "cosine." Is that right? |
Hi James, On the Desmos side, we have added Nemeth and UEB I/O modes to our four-function, scientific, and graphing calculators. When either mode is active, we render standard text input fields which accept Braille ASCII. The ASCII is internally converted to the UNICODE dot patterns, and we have designed our Simbraille font to render the dot patterns for both ASCII and UNICODE versions of each character. A six-key input mode is also available, and when active, we translate chorded input on the homewor keys into the expected Braille patterns. Some compromises we have had to make in our design to best support Apple devices include:
Like Pearson, we require the user to disable any active literary translation on the user's device beforehand because there is no way to do so automatically. We have considered creating JAWS scripts or NVDA add-ons to handle this for Windows users, but again, lack of VoiceOver scripting makes this impossible on Apple devices. Though we have written documentation for disabling Braille translation for all of the platforms we support, it is still an obstacle for some, and may not be easily accomplished by those who are uncomfortable with (or may be barred from) changing screen reader settings (such as in the context of a locked machine during end-of-year exams). Then, too, the user must remember to switch the translator back on when he/she has finished working with the editor. A means by which an input could communicate the need for Braille to a screen reader, and then for the AT product to automatically disable both input and output translation to the scope of that control would go a long way in relieving this burden from our users. Mapping custom speech to Braille characters in such circumstances (which didn't require use of live regions) would also be helpful. |
James,
I’d be happy to take the Apple-specific discussion offline if you or your team would be interested in more specifics and demonstrations about what I wrote above.
Best regards,
Steve
|
Please do. DM same username on Twitter or A11y slackers. Or email to first initial + last name at company. |
@jnurthen If we can have the Dec 17th deep dive time slot, I'll have two other Apple engineers joining the discussion for untranslated unicode braille input. |
@cookiecrook sounds good. Look for confirmation in tomorrow's meeting minutes. |
Attempting to capture the outcome of the discussion:
The expectation from @samdooley (Pearson) is that these fields would allow both braille and other unicode characters, leaving disambiguation of the braille entirely to the web app, and allowing the web app to switch from translated to untranslated braille input by modifying the attribute in real-time. (e.g. while aria-brailleinput is true, braille comes in as raw unicode braille dot patterns. If the web app removed the attribute while the user typed, the braille input would switch back to the user's specified braille table, such as UEB, and non-braille characters would flow to the field.) Prior to the call, I had not seen this desire to mix braille and non-braille in the same field... If necessary, it may limit the value of specifying the braille table on the element, because braille characters inside the field could be a mix and match of multiple tables, in the same way that a user can type multiple spoken languages in substrings of a single field. "Hello! Hola!" Prior to hearing @samdooley's desire to mix braille and non-braille in the same field, @sclower (Desmos) and I discussed the possibility of utilizing the HTML
A few people (@samdooley, @sclower, and @sinabahram IIRC) expressed a desire to use this similar to a live region announcement that would not be displayed in braille. For example, if the field contained "⠭⠐⠶⠽" (meaning "x=y" in UEB), the web app would need to track selection, and update the attribute accordingly. If the user selected or traversed across the "⠭" character, the web app should change the speech attribute value to "x"... Screen reader could then send "x" to the speech engine, but leave "⠭⠐⠶⠽" undisturbed on the display, denoted the cursor on the first cell. I noted some concerns, namely:
|
@pkra please move to a later release if this is too big for 1.3 |
@jnurthen done. |
This issue is being posted there at Joanie's request as a place to start the discussion around how to provide untranslated Unicode braille input codes from a screen reader to a web application. We're not saying that this should necessarily be an ARIA feature, but it seemed that this would be a good place to start the discussion.
Example: A math editor has a visual user interface that is designed to accept and interpret Unicode braille symbols produced by a braille device, and would like to distinguish input events for braille symbols from input events for ascii characters from a computer keyboard. The math editor is prepared to accept key events that include a property that would indicate a braille key press in the Unicode braille range U+2800 to U+28FF to indicate the braille key combination.
The text was updated successfully, but these errors were encountered: