Join GitHub today
Use braille display keys as modifier keys #6213
The feature I propose is best described with the following example.
There are several braille displays with function keys assigned to specific keyboard modifier keys, one of which is the Hims Braille Edge. This display has 8 function keys, from left to right meant to be assigned to Esc, Tab, Ctrl, Alt, Shift, Insert, Windows, and the application key. This display also has two circular groups of four buttons, meant to be arrow keys.
The mentioned braille display function keys are especially handy if you combine them with other function keys or arrow keys. So when you press advance4 (ctrl) + advance1 (tab), you should be able to simulate the ctrl+tab gesture on the keyboard.
IN the current situation, every single combination has to be manually defined in NVDA. Thus, if you assign br(hims):advance3 to kb:control and br(hims):leftSideLeftArrow to kb:leftArrow, the combination advance3+leftSideLeftArrow isn't automatically assigned to kb:control+leftArrow, unless you specifically do this, either manually or in the driver itself. Especially the in-driver assignments consume a lot of code and are error-prone, since both the keyboard key and braille key assignments reside in strings and there is currently no meganism to test them other than with the braille display in front of you.
This leads to the idea of adding a way to use braille display keys assigned to keyboard modifier keys as modifier keys themselves. In the case of Hims, if advance3 is assigned to ctrl and advance4 to alt, br(hims):advance3+advance4+leftSideLeftArrow should sum up to kb:ctrl+alt+leftArrow without the need of manually assigning this combination. Using these modifier keys in other composite key strokes (containing arrow key emulators or even braille input combinations) should be possible without manual assignments. This would open great possibilities, pressing key strokes like ctrl+c, ctrl+v, ctrl+space without keyboard intervention and manual assignments.
I understand that this proposal can conflict with current key assignments in certain display drivers, which is a point to discuss. I see the following options:
In other words, point 1 stands for freedom, point 2 stands for concistency. Since I see reasons why both options could have preference, I would vote for making this an option in the braille settings dialog
Some research revealed the following:
added a commit
Jul 28, 2016
referenced this issue
Jun 20, 2017
I'd be more inclined to make braille display drivers choose, rather than confusing users. I think in most cases, it'd make sense for drivers to rebind commands which conflict. Any ideas what other screen readers do here?
In most (maybe all?) drivers, these are already combined; BrailleDisplayGesture has special support for BrailleInputGesture when they're both inherited. (If either
I'm not a massive fan of this, since it means processing the entire gesture map with very specific code for keyboard modifiers. inputCore is really meant to be as input agnostic as it can. I'm also not sure it makes sense to show all these dynamically created keys. They're either modifiers or they're not. If they're modifiers, selectively rebinding them seems inconsistent/confusing.
Again, any idea what other screen readers do here?
I belief that this functionality would be unique to NVDA. As far as I know, JAWS and SuperNova have no such thing as a modifier key on a braille display, so they require display vendors to bind all combinations manually. I'm not 100% sure, though.
That's a good idea, I'll need to keep that one in mind whenever I'd start working on this.
Does this mean you prefer the scriptHandler._getObjScript approach, or do you have a third alternative in mind?
I started working on this, and it seems there is a nicer approach than _getObjScript, namely overriding _get_script on BrailleDisplayGesture. This way, most of the work can be done in the braille module itself.
@jcsteh: There are some braille display drivers which set the keyNames property on their input gesture class. However, some use sets, some use lists, and it seems that, especially for the last drivers you wrote or improved, you used a list. Do you have a particular reason why you went for a list and not for a set here? I'd say using a list can result in ids like routing+routing+routing for the baum driver, and that makes no sense as there is only one routing index.
The reason why I'm asking, I'd like to have the braille display specific gestures to have a keyNames property, which defaults to set(self.id.split("+")) in my current code base but can also be set by the gesture code itself, like id. However, that requires me to change some of the current driver's gestures to use a set for keyNames. I'd like to know whether you have problems with that approach.
We previously used sets because we were using set order to normalize identifiers. However, since that order isn't reliable, we no longer do this and instead sort. So, building a set is pointless because we're just going to go and sort it anyway. One other advantage of not using a set is that the id is also used as the display name If you use a set instead, the order reported to the user is arbitrary. In contrast, if you use a list, the display has control over this. This might just be based on the order of the key identifiers, but this normally still makes more sense than set order. Is there any particular reason keyNames needs to be a set? Note that the cost of building a small set can actually outweigh the cost of linear searching a list in some cases. Also, if you really do need a set, you could consider having a cached property which converts to a set for use in specific cases.
Sets are great for checking whether a group of keys is part of another group of keys. I assume with lists, this consumes more resources as well as more code, right? I think I will go for a list for keyNames, as I understand and support your arguments for this.