Unable to position the cursor in the editor using braille routing keys in NVDA

[bramd]

• VSCode Version: Code 1.12.2 (19222cd, 2017-05-10T13:20:36.315Z)
• OS Version: Windows_NT ia32 10.0.15063

---

Steps to Reproduce:

1. Open a file in the editor
2. Ensure NVDA is in focus mode (for example by pressing space/enter on the editor)
3. Use a cursor routing button on a connected braille display and click on a character in the text

Expected:

The cursor moves to the chosen position.

Actual:

Cursor doesn't move in the editor, but the cursor is displayed on the right position in braille in the textarea used for editor accessibility until it refreshes (e.g. by moving cursor using the keyboard).

[alexdima]

@bramd Do you know if there is any way to simulate what NVDA does in this case without having a real braille display?

[bramd]

@alexandrudima I had a look in the NVDA source and it's not clear to me what NVDA does exactly. I guess it sends messages through the winapi or another API to instruct the textarea to move the cursor. I didn't catch any useful events with accevent and the default settings. Probably @jcsteh knows?

If you know another way to catch the events/messages being sent, I'm glad to provide a log.

[jcsteh]

NVDA calls IAccessible2::setCaretOffset. I believe this is supposed to be reflected as a selectionChange DOM event. However, I think this may have only been implemented correctly in a fairly recent Chromium version, though I can't find a reference for this. Furthermore, MDN seems to suggest this is only supported for contentEditable in Chrome, not for input/textarea, though this could be outdated information. (I can't remember whether VS Code uses contentEditable or textarea, but vaguely recall textarea.)

[zersiax]

After doing some testing, this most likely is a Chromium issue. I had a play with Postman running in a Chrome Stable instance and ran into the same issue, haven't tried any experimental builds though. Looking at the behavior of the braille display it seems the accessibility API registers the event and acts on it, but doesn't actually properly move the caret.

[jcsteh]

@zersiax commented on 13 Jun 2017, 04:26 GMT+10:

After doing some testing, this most likely is a Chromium issue. I had a play with Postman running in a Chrome Stable instance and ran into the same issue

Don't know anything about Postman, but assuming it does custom management of the caret, even if Chrome does fire the selectionChange DOM event, Postman would need to support this event and move its internal idea of the caret accordingly.

I confirmed that VS Code uses textarea. I also confirmed that Chrome (59) does fire selectionChange for textarea:

data:text/html,<textarea>foo</textarea><script>document.addEventListener("selectionchange", function() {alert('selectionChange');});</script>

If I call IAccessibleText::setCaretOffset (as will happen when you press a cursor routing key on a braille display), I get the alert.

[zersiax]

For braille users this is a bit of a productivity hog, do we know how we want to proceed on this one yet?

[ushiushix]

I use braille a lot especially for programming. I tried to use Code as my primary editor for Windows, but this issue blocked me.
Capability of jumping the cursor to the touching position immediately is the biggest advantage of braille displays.

[alexdima]

I have briefly investigated listening to the browser event selectionchange. I received it all the time from the textarea, even when clicking around in VS Code or when alt+tabbing, so I cannot add some code that always resets the editor's cursor to the textarea caret when this event is received.

So I would need to write some filtering condition to figure out if this event is coming in due to the use of a Braille display and if we should react to it or not.

The problem I ran into is that I don't know at this time how to emulate/simulate a Braille display caret change i.e. as @jcsteh mentions, the IAccessible2::setCaretOffset call NVDA does, to try to discriminate the false selectionchange events from the ones we should react to.

I can poke around at building NVDA from source and try to get it to call IAccessible2::setCaretOffset or I could create a special version of VS Code where I log all the incoming selectionchange events and then ask you for a trace.

[jcsteh]

Routing the caret to the review cursor would also cause setCaretOffset to be called. To do this: 1. Move the review cursor away from the caret. For example, you might press numpad9 (laptop layout: NVDA+downArrow) to move to the next line. 2. Press NVDA+shift+numpadMinus twice quickly (laptop layout: NVDA+shift+backspace twice quickly) to route the caret to the review cursor. In this example, NVDA will attempt to route the caret to the start of the line after the caret. In IA2 terms, this means calling IAccessibleText::setCaretOffset.

[alexdima]

Thank you @jcsteh for the alternative steps. I pushed a change where we now listen to "selectionchange" and, with some heuristics, ignore the plethora of events generated by ourselves programmatically.

After these changes, I was quite successful in using Numpad9 or Numpad3 to navigate the review cursor and in using NDVA+Shift+NumpadMinus to move the editor cursor to the review cursor.

The change will show up in our Insiders channel once 1.15 makes it to the Stable channel (i.e. the first Insiders with version 1.16). I will ping you once that happens, to kindly ask for you to validate with a real Braille display.

[bramd]

Cool, glad to test it when this lands in Insiders.

[ushiushix]

Now I installed 1.16.0 insider build. Cursor routing keys moves the cursor in the document to the desired position.
I've tested it with JAWS.

Great! And thanks.

[zersiax]

Confirmed working in NVDA, provided braille is tethered to focus (nvda+ctrl+t to toggle). Excellent work :)

[alexdima]

That is great news! Thank you @ushiushix and @zersiax for verifying it works ❤️ !

[derekriemer]

great work guys!

[bramd]

Thanks for this, it works fine in most cases. I've had it fail a few times and it seems that in the failure cases, focus is placed outside the editor, I can't exactly determine where. Also, I can't reproduce the failure reliably, so no idea yet what causes this. I'll comment here if I have a clue. That being said, I'm very happy with this functionality and it makes me more productive in VSCode.