Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't add a post title using speech recognition software #5468

Open
afercia opened this issue Mar 7, 2018 · 37 comments

Comments

@afercia
Copy link
Contributor

commented Mar 7, 2018

Highly recommended video (only 3 minutes) to understand how building non-native interfaces can easily break the most obvious features.
/cc @karmatosed @jasmussen @mtias @aduth @youknowriad

Adding a title to a Wordpress post using Gutenberg 2.3 and Dragon 14
https://www.youtube.com/watch?v=w7uG17NvHLQ

Thanks to Eric Wright for the excellent video and to Rian Rietveld for involving assistive technologies testers.

Gutenberg Version 2.3
Dragon 14
Chrome latest

Two issues:

1 (minor)
Saying "click Add Title" won't set my focus there
This is because the title textarea doesn't have a properly associated label. This is one of the reasons why form fields should always have visible associated labels. Annoying issue, but at least users can tab to the title.

2 (major)
when tabbing away from the title, the dictated text disappears completely. I have no idea why. Of course, this doesn't happen on a simple textarea. I can only guess there's some conflict between the events Gutenberg uses on this textarea and events triggered by Dragon.

@rianrietveld any chance to involve the Dragon team here for some feedback?

@aduth

This comment has been minimized.

Copy link
Member

commented Mar 7, 2018

Very informative video. Thanks for sharing.

Saying "click Add Title" won't set my focus there

In saying "click Add Title" or otherwise using voice dictation to navigate to a field, what are the attributes that these tools are using to determine where the field is on that page? Is this something we could "fake" with an attribute of "Add Title" which is not the placeholder? A hidden label?

Aside: Should software like this be taking placeholder into consideration?

when tabbing away from the title, the dictated text disappears completely.

This looked particularly bad. Digging into the implementation of PostTitle, it's responding to the change event from a textarea field. I'm curious if this would be handled any better on an input event (which is probably the more appropriate event to be using anyways).

@aduth

This comment has been minimized.

Copy link
Member

commented Mar 7, 2018

From: http://supportcontent.nuance.com/kbfiles/2626_DNS_HTML.pdf

NAVIGATION (VOICE COMMANDS)

[...]

Text Areas (<TEXTAREA>)
You can assign an ALT or TITLE attribute to a text area. As with edit controls, you need to find a
way to make the associated text apparent to the user, and the text should conform to the general
recommendations.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 7, 2018

what are the attributes that these tools are using to determine where the field is on that page? Is this something we could "fake" with an attribute of "Add Title" which is not the placeholder? A hidden label?

Normally, that would be a properly associated, visible label. Software can easily check when a <label> has a for attribute, grab the value, search for an element with that ID. Or, check if a label wraps an element. This is also used to give a field a so called "accessible name".

Of course, the label needs to be visible to the user, otherwise the user wouldn't know what to say.

Even using an alt or title attribute as suggested in the note you posted above, there's the need to "find a way to make the associated text apparent to the user" before the field is focused.

That said, this is a minor issue because it is still possible to tab to the field, although that's slower for speech recognition software users.

As a last resource, Dragon can split the screen in a "grid" with 9 cells see http://nuance.custhelp.com/app/answers/detail/a_id/3430/~/information-on-the-mousegrid-command but that's a very slow process and not relevant here.

Aside: Should software like this be taking placeholder into consideration?

Apparently they don't and I doubt they will. As the accessibility team is trying to communicate, placeholders shouldn't be use as replacement for labels. For many reasons, including that in the near future placeholder attributes won't contribute any longer to the "accessible name" calculation, see https://core.trac.wordpress.org/ticket/40331#comment:3

I'm curious if this would be handled any better on an input event

I really have no idea why the text disappears, and really don't know if an input field or the planned title block would behave better.

@StommePoes

This comment has been minimized.

Copy link

commented Mar 7, 2018

Yup, visible labels that actually are the label are a major thing for voice users.

I could see AT taking in placeholders as part of the accessible-name algorithm-- I've seen some screen readers do it, though what they announce when sometimes depends on how the user got to the input (by tabbing in a focus mode or by arrowing in a browsing mode for example). But I wouldn't want to encourage placeholder use even if they did start taking it into consideration.

You can assign an ALT or TITLE attribute to a text area.

Alt doesn't make any sense to me. Is that a WP thing? In HTML alt's an image-only thing.

I am in general wary of onchange listeners on inputs (browsers do differ on when something counts as onchange-- for example on select elements, IE considers merely arrowing around the options in order to see them as a change, while other browsers don't), and it does sound like something to try (changing the listener from onchange to oninput, which I also like for inputs. The last thing I remember not working with oninput was IE9 wouldn't count hitting the delete key on that one). If listeners are conflicting with each other, that can be an easy way to stop that.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 7, 2018

In HTML alt's an image-only thing.

Actually, that's not completely exact 😉 It can be used on other elements too, though I wouldn't consider it in this specific case.
https://www.w3.org/TR/html52/sec-forms.html#the-input-element

Worth reminding the React onchange is something a bit different from the native change event. Not sure how this could relate to text disappearing issue though.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 7, 2018

Just came into my mind: as far as I can tell, Dragon should be able to get aria-label attributes, at least in its latest versions /cc @grahamarmfield

Since there's already a visible "Add title", the aria-label should give the same accessible name:
aria-label="Add title" and hopefully help.

This won't help when the title is already populated with some text, unless users remember the field name is "Add title". Note: it's a workaround, not the best example of accessible labeling 🙂

The second issue, the disappearing text, is still a mystery to me.

@aduth

This comment has been minimized.

Copy link
Member

commented Mar 7, 2018

I'd be happy to put together a test branch changing the PostTitle to react to a textarea's input event instead of change, but I don't have a means to test myself whether it has any impact (no access to Dragon).

@rianrietveld

This comment has been minimized.

Copy link
Member

commented Mar 8, 2018

@aduth
give me a ping when you have a PR ready, then I'll organize the Dragon troops for a test.

@aduth

This comment has been minimized.

Copy link
Member

commented Mar 9, 2018

@rianrietveld I've not made a pull request for it (yet), but you can use the branch try/post-title-input (aa2ba67) to test.

Let me know if you need any help setting this up.

@akorik

This comment has been minimized.

Copy link

commented Mar 9, 2018

Hi Everyone,

I took a look at the video Eric Wright put together and I have a couple of comments and a workaround.
In the video Eric is tabbing down to the Title field...I know you guys are discussing using the mouse grid and that is a great Dragon feature but may I suggest instead you try using the (show links) command. Since you are using Dragon with Chrome you should have our Dragon Web Extension installed.

Example: Navigate to the post-new.php page.
Turn on your mic and say "show links" - you will see a bunch of green squares with numbers in them
With the mic still on say "choose 23" - 23 is the number on my page that correlates to the Title field.

As far as the dictated title text disappearing - That is an interesting bug. You guys already know that if you key in the title it remains (great clue). My workaround isn't elegant but it will do the job.

Prerequisite: Copy a word to your clipboard
Example: Navigate to the post-new.php page.
Turn on your mic and say "show links" - you will see a bunch of green squares with numbers in them.
With the mic still on say "choose 23" - 23 is the number on my page that correlates to the Title field.
Say "paste that" - The word from your clipboard will be pasted into the Title field.
Now say "delete that" - The word will be deleted.
Now dictate your title and say "tab key" - Your title will remain and your cursor will to the next block.

Hope this helps,
Alex

@rianrietveld

This comment has been minimized.

Copy link
Member

commented Mar 13, 2018

@aduth I made an install on http://wpaccess.org/gutenberg-git/ with the try/post-title-input branch and asked the Dragon testers to have a look at adding a title to test. Will report back when they tested.

@ewaccess

This comment has been minimized.

Copy link

commented Mar 13, 2018

I was able to test the latest install. Dragon still did not recognize "click Add title," but it might be an issue with Dragon. I wrote some simple HTML5 pages with nothing but a <textarea> and an aria-label, and Dragon focus those either.

@akorik, thank you for the pointer on "Show links!" I've been using "click [element]" for decades, but I had no idea "Show links" existed. I was able to set focus to the title block using that. Once I had it in my head that the basic HTML navigation commands were available in Gutenberg, I was also able to set focus to the title block using "click Edit" and "click Text."

Initially I was unsure how to set focus because Gutenberg struck me as an app, so I thought I only had the desktop commands available to me, in other words "click [accessible name]"

Unfortunately, text is still not persistent when dictating directly into the Title block. It is if I use Alex's workaround above (or the Dictation Box). Dictated text is persistent when dictating into the text block below the Title block.

Thank you for your work on this!

@StommePoes

This comment has been minimized.

Copy link

commented Mar 13, 2018

I've not seen anywhere in Dragon command lists "show links" nor would I think of an input as a link. Is this more like "Show Numbers" in Windows Speech Rec (shows all "clickables" regardless of type)?

So while there's now a type of solution, it's not really since then Gutenberg would need to include such instructions to speech users.

I wrote some simple HTML5 pages with nothing but a <textarea> and an aria-label, and Dragon focus those either.

Eric, is that it did or did not focus?

@ewaccess

This comment has been minimized.

Copy link

commented Mar 13, 2018

@StommePoes Yikes, sorry for the critical omission there. That's did not focus. I know Dragon recognizes aria-label on some inputs, but apparently not on textarea.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 13, 2018

@ewaccess thanks so much for all your testing and feedback! And the excellent videos :) Unfortunately I can't test with Dragon :( When you have a chance, would you mind trying to add a role="textbox" to the textarea in your HTML test to see if the aria-label is recognized?

@ewaccess

This comment has been minimized.

Copy link

commented Mar 13, 2018

Good idea, but unfortunately Dragon doesn't recognize it. Here's a simple pen: https://codepen.io/ewaccess/pen/zWrPeq

To confirm, I do have "show links" and "click Edit" at my disposal to set focus the title block. Neither occurred to me at the time because I was thrown by the app-like nature of Gutenberg and thought I could only say a name.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2018

Thanks @ewaccess. Hm... new attempt: with a visually hidden <label> element, see this pen forked from your one: https://codepen.io/afercia/pen/LdGXQR

@ewaccess

This comment has been minimized.

Copy link

commented Mar 14, 2018

@afercia success! The visually hidden label works. Confirmed with Dragon 14 and 15 (latest version).

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 14, 2018

Yay! 🎉 So, aria-label doesn't work at all on textarea elements with Dragon. I'd be curious to know if there's some documentation about Dragon support for ARIA. Maybe @grahamarmfield ?

@StommePoes

This comment has been minimized.

Copy link

commented Mar 15, 2018

if there's not, perhaps Eric and I could make such documentation? Eric's shown several other issues that developers don't tend to know about (and can't do so much about either) such as the problem with using input type=number.

Edit I don't mean to sign Eric up for volunteer work :P But I'm very much a Dragon newb, I mostly only use it in IE to play with web pages.

@ewaccess

This comment has been minimized.

Copy link

commented Mar 17, 2018

@StommePoes that sounds like a great idea. Happy to document Dragon accessibility support with you. The more the merrier!

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 17, 2018

Re: the "click Add title" issue, there's now a PR to use a visually hidden <label> for the post title: #5669

Re: the disappearing dictated text, I've prepared two pens to quickly test what happens using onInput instead of onChange. While in React these are basically equivalent, the event name the browser exposes to assistive technologies could play a role. It's' just a guess, but worth testing.

Textarea:
https://codepen.io/afercia/full/EEgGWY/

Search field (to cover the case of the insert link issue):
https://codepen.io/afercia/full/QmKZeE/

Some testing with Dragon would be greatly appreciated :)

@ewaccess

This comment has been minimized.

Copy link

commented Mar 17, 2018

All inputs successful! Each field has Full Text Control. I was able to dictate text into and manipulate text within all four inputs. Text was persistent when tabbing and shift tabbing into the fields.

@grahamarmfield

This comment has been minimized.

Copy link

commented Mar 18, 2018

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 18, 2018

@ewaccess @grahamarmfield thanks so much for testing. Sorry for the onChange / onInput labels, I've used them just for testing purposes.

neither dictating the text, nor using voice commands to tab around seemed to fire either the onchange or input event handlers

Hm do you mean the text below the fields didn't update? That's weird. Makes me wonder which event gets fired when Dragon populates the field with the dictated text.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 18, 2018

While testing, I've noticed some inconsistencies in the way the Undo/Redo commands work. I'm going to open a separate issue for this. One possible reason for the disappearing dictated text here is that, for some reason, using Dragon triggers the "Undo" in Gutenberg so the text gets removed.

One way to test this would be trying to dictate text in a post title that already contains some text:

  • create a post, add a title and a short text in a paragraph without using Dragon
  • publish
  • navigate away
  • reopen the post
  • using Dragon, add some text to the existing post title
  • tab away with Dragon and see what happens
@ewaccess

This comment has been minimized.

Copy link

commented Mar 18, 2018

Hi @afercia, I just followed the steps above (Dragon 15/Chrome 65) and recorded a 2-minute video. https://www.youtube.com/watch?v=VToDhQOQdSQ

Result: The dictated text disappears when I tab away. The typed text remains. Anecdotally, this supports what I saw earlier with the Ctrl+Z shortcut not working as expected.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented Mar 22, 2018

Thanks again @ewaccess for your testing :) That's very interesting and it seems to me we can now exclude some kind of bug that "empties" the title content.
Instead, seems to me what's happening here is that, for some reason, tabbing away while using Dragon triggers the Gutenberg "Undo" command so only the last added text gets removed. Now we have just to understand why :)

On a side note, can I publish somewhere the full 48 minutes video of your testing session? I'd really like to make it available to all the people in the team.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented May 15, 2018

For reference:

What we learned from getting our autocomplete tested for accessibility
https://accessibility.blog.gov.uk/2018/05/15/what-we-learned-from-getting-our-autocomplete-tested-for-accessibility/

When she dictated ‘France’, the text would appear as normal. When she tried to continue, the field would go blank.
Later we found out this was because Dragon uses a special way to input the text, and autocomplete wasn’t expecting this.

Maybe someone at gov.uk knows the answer.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented May 24, 2018

I've tweeted about the "disappearing" text and got a helpful hint from the UK gov digital service team, see https://twitter.com/tvararu/status/999758793276829700 They faced a very similar issue for their autocompleter implementation and ended up with a solution where they "poll" the input field. See https://github.com/alphagov/accessible-autocomplete/pull/15/files

@aduth when you have a chance: does it shed some light on what's going on with the disappearing text?

@afercia afercia referenced this issue May 25, 2018
3 of 3 tasks complete
@aduth

This comment has been minimized.

Copy link
Member

commented May 29, 2018

To be honest, polling is an imperfect solution, and isn't so much a compatibility solution as it is abandoning there being any possibility of interpreting the change intent from Dragon directly†.

The costs of polling are:

  • Performance impact of code running on an interval. This should be limited as much as possible.
  • Changes are not reflected immediately, which could manifest itself in various negative experiences:
    • Flickering UI as changes take effect
    • Race conditions between the user's intended input and the polled interpretation of the value

Based on the previous-mentioned Tweet describing Dragon's behavior of modifying the value property, some options we could potentially explore include:

† For all I know, there is no way to interpret this change directly, in which case polling may be the only option. And it is certainly a short-term resolution, but one which we should acknowledge comes at the cost of future maintainability. Which is to say: I don't mean to slander with the phrase "imperfect"; sometimes an imperfect solution is the correct solution.

@tvararu

This comment has been minimized.

Copy link

commented May 29, 2018

@aduth that's a solid writeup! I don't recall us considering Object.defineProperty, that definitely seems worth exploring as it's compatible back to IE9. 👍

@aduth

This comment has been minimized.

Copy link
Member

commented May 29, 2018

Another idea I considered which unfortunately won't work: If value was a reflective property, meaning it sets the equivalent value attribute when changed, then a MutationObserver object observing with an attributes: true options value could detect the change in value attribute. However, from what I can tell, value property changes do not reflect back to the attribute.

@aduth

This comment has been minimized.

Copy link
Member

commented May 29, 2018

Object.defineProperty looks promising:

@aduth

This comment has been minimized.

Copy link
Member

commented May 29, 2018

I guess it should be stated: Dragon is partly at fault here with setting the property explicitly, since it doesn't trigger any corresponding events, requiring us to resort to such workarounds. Ideally they'd set the value in a fashion more in line with default browser behaviors (where events would be triggered).

@aduth

This comment has been minimized.

Copy link
Member

commented May 29, 2018

I don't recall us considering Object.defineProperty, that definitely seems worth exploring as it's compatible back to IE9. 👍

To the point of compatibility, it should only need to go back so far as you intend to support a browser with Dragon dictation, if it's assumed you can rely on input / change events for all other usage.

@afercia

This comment has been minimized.

Copy link
Contributor Author

commented May 30, 2018

I guess it should be stated: Dragon is partly at fault here

Welcome to my world, where fighting with assistive technologies is like fighting with the browsers we used to have 15 years ago 😬That's one of the reasons I always try to recommend to keep things as simple and "standard" as possible. Dragon has clear responsibilities in what's going on here. On the other hand, maybe the post title textarea is going a bit beyond what can be considered "standard".

Aside: why the isSelected state is set to false at any keypress event?

Thanks again to @tvararu @aduggin and all at the gov.uk digital service. (Love open source ❤️)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
8 participants
You can’t perform that action at this time.