New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Single letter navigation from a braille terminal #7306

Closed
MichelSuch opened this Issue Jun 20, 2017 · 10 comments

Comments

Projects
None yet
5 participants
@MichelSuch

MichelSuch commented Jun 20, 2017

When in browse mode, user should be able to type a single letter command from his braille terminal.
For example, typing "h" on the terminal should move to the next title.
I have tested it on 2 braille devices (Eurobraille Esys and Baum Vario Ultra) and it does not work on both.
Maybe does it work on other devices?

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Jun 20, 2017

Collaborator
Collaborator

leonardder commented Jun 20, 2017

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Jun 20, 2017

Collaborator
Collaborator

leonardder commented Jun 20, 2017

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Jun 20, 2017

Collaborator

@MichelSuch: Would you like to test this using next? I"m using next-14122,9b0534fc and with the pc keyboard braille input add-on, I'm able to use single letter quick navigation perfectly fine. Backwards doesn't work though, but that, as I mentioned, can be implemented in #6213

Also note that on (Eurobraille Esys, NVDA's braille input layer is not being used, so we can't do anything about that.

Collaborator

leonardder commented Jun 20, 2017

@MichelSuch: Would you like to test this using next? I"m using next-14122,9b0534fc and with the pc keyboard braille input add-on, I'm able to use single letter quick navigation perfectly fine. Backwards doesn't work though, but that, as I mentioned, can be implemented in #6213

Also note that on (Eurobraille Esys, NVDA's braille input layer is not being used, so we can't do anything about that.

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jun 21, 2017

Contributor

@leonardder commented on 20 Jun 2017, 23:44 GMT+10:

@jcsteh: I assume that, from a technical point of view, pressing dot
1235 on a braille keyboard (letter h) now sends the input to the browser
itself?

I assume you mean 125. :) It sends the Unicode character (not the key) to the system.

@leonardder commented on 21 Jun 2017, 01:14 GMT+10:

I"m using next-14122,9b0534fc and with the pc keyboard braille input add-on, I'm able to use single letter quick navigation perfectly fine.

This is interesting; I didn't expect this, so I did a bit of investigation. The reason this works is that if you have NVDA's "Handle keys from other applications" option enabled, NVDA intercepts the keys generated by the system when we send Unicode characters. This didn't work properly before, but it got fixed by 243d39f (which was actually intended to address an entirely different symptom). That's nice, but it won't play so well with modifiers, since it isn't an emulated gesture, and it also doesn't work if "Handle keys from other applications" is disabled (which doesn't make much sense).

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

Backwards doesn't work though, but that, as I mentioned, can be implemented in #6213

That'll help for braille displays which have dedicated modifier keys, but many don't. I think we still need a way to enter modifiers using the braille keyboard (space+dots combinations). Interestingly, that's probably significantly cleaner to implement than #6213.

Contributor

jcsteh commented Jun 21, 2017

@leonardder commented on 20 Jun 2017, 23:44 GMT+10:

@jcsteh: I assume that, from a technical point of view, pressing dot
1235 on a braille keyboard (letter h) now sends the input to the browser
itself?

I assume you mean 125. :) It sends the Unicode character (not the key) to the system.

@leonardder commented on 21 Jun 2017, 01:14 GMT+10:

I"m using next-14122,9b0534fc and with the pc keyboard braille input add-on, I'm able to use single letter quick navigation perfectly fine.

This is interesting; I didn't expect this, so I did a bit of investigation. The reason this works is that if you have NVDA's "Handle keys from other applications" option enabled, NVDA intercepts the keys generated by the system when we send Unicode characters. This didn't work properly before, but it got fixed by 243d39f (which was actually intended to address an entirely different symptom). That's nice, but it won't play so well with modifiers, since it isn't an emulated gesture, and it also doesn't work if "Handle keys from other applications" is disabled (which doesn't make much sense).

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

Backwards doesn't work though, but that, as I mentioned, can be implemented in #6213

That'll help for braille displays which have dedicated modifier keys, but many don't. I think we still need a way to enter modifiers using the braille keyboard (space+dots combinations). Interestingly, that's probably significantly cleaner to implement than #6213.

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Jun 21, 2017

Collaborator

Replying to @jcsteh:

I assume you mean 125. :)

Oops, yes

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

I'm not sure what you mean with non-editable cases here

That'll help for braille displays which have dedicated modifier keys, but many don't.

My idea was to allow people to set whatever key they like to a modifier, e.g. setting a scroll key to shift, or even control + shift wil automatically consider the key to be a modifier as it emulates a modifier or a group of modifiers. But that's something that needs to be discussed in #6213.

I think we still need a way to enter modifiers using the braille keyboard (space+dots combinations). Interestingly, that's probably significantly cleaner to implement than #6213.

Why the space+dots combinations specifically? That would only allow the space to be a modifier if I understand you correctly?

Collaborator

leonardder commented Jun 21, 2017

Replying to @jcsteh:

I assume you mean 125. :)

Oops, yes

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

I'm not sure what you mean with non-editable cases here

That'll help for braille displays which have dedicated modifier keys, but many don't.

My idea was to allow people to set whatever key they like to a modifier, e.g. setting a scroll key to shift, or even control + shift wil automatically consider the key to be a modifier as it emulates a modifier or a group of modifiers. But that's something that needs to be discussed in #6213.

I think we still need a way to enter modifiers using the braille keyboard (space+dots combinations). Interestingly, that's probably significantly cleaner to implement than #6213.

Why the space+dots combinations specifically? That would only allow the space to be a modifier if I understand you correctly?

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jul 19, 2017

Contributor

@leonardder commented on 21 Jun 2017, 20:50 GMT+12:

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

I'm not sure what you mean with non-editable cases here

I mean a control that doesn't accept characters as text input. You can use the new private NVDAObject._hasNavigableText property for part of this, but you must also treat browse mode as non-editable.

I think we still need a way to enter modifiers using the braille keyboard (space+dots combinations).

Why the space+dots combinations specifically? That would only allow the space to be a modifier if I understand you correctly?

I mean space+dots, then character. For example, to do shift+h with Narrator in Windows 10 Insiders, you might do space+dot8+dot4, release everything, then braille h.

Contributor

jcsteh commented Jul 19, 2017

@leonardder commented on 21 Jun 2017, 20:50 GMT+12:

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

I'm not sure what you mean with non-editable cases here

I mean a control that doesn't accept characters as text input. You can use the new private NVDAObject._hasNavigableText property for part of this, but you must also treat browse mode as non-editable.

I think we still need a way to enter modifiers using the braille keyboard (space+dots combinations).

Why the space+dots combinations specifically? That would only allow the space to be a modifier if I understand you correctly?

I mean space+dots, then character. For example, to do shift+h with Narrator in Windows 10 Insiders, you might do space+dot8+dot4, release everything, then braille h.

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Jul 31, 2017

Collaborator

@jcsteh commented on 20 jul. 2017 01:55 CEST:

I mean space+dots, then character. For example, to do shift+h with Narrator in Windows 10 Insiders, you might do space+dot8+dot4, release everything, then braille h.

This is interesting. However, what would nvda+Control+Shift+H look like, for example? Initially, the idea of #6213 popped into my mind due to the amount of function keys on Hims displays which can be used as such. In that case, ctrl+H would be just one key stroke, f3+dot1+dot2+dot5. Your proposal is slightly different. I assume I can build upon the new brailleInput system which allows to delay translation to occur after multiple key presses?

Collaborator

leonardder commented Jul 31, 2017

@jcsteh commented on 20 jul. 2017 01:55 CEST:

I mean space+dots, then character. For example, to do shift+h with Narrator in Windows 10 Insiders, you might do space+dot8+dot4, release everything, then braille h.

This is interesting. However, what would nvda+Control+Shift+H look like, for example? Initially, the idea of #6213 popped into my mind due to the amount of function keys on Hims displays which can be used as such. In that case, ctrl+H would be just one key stroke, f3+dot1+dot2+dot5. Your proposal is slightly different. I assume I can build upon the new brailleInput system which allows to delay translation to occur after multiple key presses?

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Aug 1, 2017

Contributor

@leonardder commented on 1 Aug 2017, 05:06 GMT+10:

This is interesting. However, what would nvda+Control+Shift+H look like, for example?

With Narrator, you'd do the following (changing NVDA to caps lock because that's what Narrator uses): space+dot8+dot6 (hold caps lock), space+dot8+dot3 (hold control), space+dot8+dot4 (hold shift), then finally braille h. See this article about the Insider build which introduced this for further details. Note that with JAWS braille, you can (and I think must) combine modifiers; in this case, space+dot8+dot5+dot3+dot7, then braille h. The JAWS way of doing it requires less keystrokes.

Initially, the idea of #6213 popped into my mind due to the amount of function keys on Hims displays which can be used as such.

For sure. It certainly makes sense to support those where available, but not all displays have such keys.

Your proposal is slightly different.

My proposal is intended to allow modifiers to be entered purely using the braille keyboard and I'd argue it's more broadly applicable. However, the two proposals aren't at all mututally exclusive, though they'll require somewhat separate implementations.

I assume I can build upon the new brailleInput system which allows to delay translation to occur after multiple key presses?

I don't think this modifier stuff can really benefit from existing code, since that code is all about maintaining text state in a buffer. However, you could track held modifiers in brailleInput for sure and then make sure they get pressed before the braille characters are sent and released afterwards (in BrailleInputHandler.sendChars).

One other thing to consider is that ideally, if the user is using a contracted table, we'd want to switch to an uncontracted table when there are held modifiers. We don't support the concept of a configured uncontracted table yet, though.

Contributor

jcsteh commented Aug 1, 2017

@leonardder commented on 1 Aug 2017, 05:06 GMT+10:

This is interesting. However, what would nvda+Control+Shift+H look like, for example?

With Narrator, you'd do the following (changing NVDA to caps lock because that's what Narrator uses): space+dot8+dot6 (hold caps lock), space+dot8+dot3 (hold control), space+dot8+dot4 (hold shift), then finally braille h. See this article about the Insider build which introduced this for further details. Note that with JAWS braille, you can (and I think must) combine modifiers; in this case, space+dot8+dot5+dot3+dot7, then braille h. The JAWS way of doing it requires less keystrokes.

Initially, the idea of #6213 popped into my mind due to the amount of function keys on Hims displays which can be used as such.

For sure. It certainly makes sense to support those where available, but not all displays have such keys.

Your proposal is slightly different.

My proposal is intended to allow modifiers to be entered purely using the braille keyboard and I'd argue it's more broadly applicable. However, the two proposals aren't at all mututally exclusive, though they'll require somewhat separate implementations.

I assume I can build upon the new brailleInput system which allows to delay translation to occur after multiple key presses?

I don't think this modifier stuff can really benefit from existing code, since that code is all about maintaining text state in a buffer. However, you could track held modifiers in brailleInput for sure and then make sure they get pressed before the braille characters are sent and released afterwards (in BrailleInputHandler.sendChars).

One other thing to consider is that ideally, if the user is using a contracted table, we'd want to switch to an uncontracted table when there are held modifiers. We don't support the concept of a configured uncontracted table yet, though.

leonardder added a commit to BabbageCom/nvda that referenced this issue Aug 17, 2017

Implemented support for braille display key modifiers:
	* For example, if L{self.id} is 'key1+key2', key1 is bound to 'kb:control' and key2 to 'kb:tab', this gesture should execute 'kb:control+tab'
	* Combining modifiers with braille input (nvaccess#7306) is not yet supported

@ehollig ehollig added the Braille label Aug 21, 2017

michaelDCurran added a commit that referenced this issue Oct 3, 2017

Basic support for braille display key modifiers (#7503)
* Implemented support for braille display key modifiers:

	* For example, if L{self.id} is 'key1+key2', key1 is bound to 'kb:control' and key2 to 'kb:tab', this gesture should execute 'kb:control+tab'
	* Combining modifiers with braille input (#7306) is not yet supported

* Review action

* Removed line of code which has become obsolete after the review action
@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Nov 18, 2017

Collaborator

Requests for this are increasing, so I will adopt this, unless someone else wants to do it. Some rough ideas. @jcsteh, as you know the code much better than I do, your ideas are welcome.

@jcsteh commented on 21 jun. 2017 07:51 CEST:

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

I belief this is actually not that difficult. I propose:

  1. In BrailleInputHandler._translate, instead of throwing everything into sendChars:

    • Check for editable text cases as well as the length of newText
    • If not editable text and newText is exactly one character, emulate a gesture instead of sending unicode cahracters
  2. The case I'd also like to cover is contracted braille table input. I belief that quick nav should be possible with contracted tables as well. This will mostly require a user to type one letter combination and force translation with dots7-8. I belief this is actually as simple as checking the length of newText and, if more than one character, warn the user that the given braille input consists of more than one character and is therefore not supported in the currently focused control, and return false.

Collaborator

leonardder commented Nov 18, 2017

Requests for this are increasing, so I will adopt this, unless someone else wants to do it. Some rough ideas. @jcsteh, as you know the code much better than I do, your ideas are welcome.

@jcsteh commented on 21 jun. 2017 07:51 CEST:

I think the correct solution here is to detect non-editable cases in brailleInput and emulate keystrokes instead of Unicode characters.

I belief this is actually not that difficult. I propose:

  1. In BrailleInputHandler._translate, instead of throwing everything into sendChars:

    • Check for editable text cases as well as the length of newText
    • If not editable text and newText is exactly one character, emulate a gesture instead of sending unicode cahracters
  2. The case I'd also like to cover is contracted braille table input. I belief that quick nav should be possible with contracted tables as well. This will mostly require a user to type one letter combination and force translation with dots7-8. I belief this is actually as simple as checking the length of newText and, if more than one character, warn the user that the given braille input consists of more than one character and is therefore not supported in the currently focused control, and return false.

@leonardder leonardder self-assigned this Nov 18, 2017

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Nov 22, 2017

Collaborator

There is a prototype for this in the @BabbageCom i7306 branch, which makes use of liblouis partial trans for contracted input. This means that, even if we use an contracted table, we can treat its input as uncontracted outside text controls.

Collaborator

leonardder commented Nov 22, 2017

There is a prototype for this in the @BabbageCom i7306 branch, which makes use of liblouis partial trans for contracted input. This means that, even if we use an contracted table, we can treat its input as uncontracted outside text controls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment