Change routingIndex to cellIndexes on BrailleDisplayGesture and use multiRouting

[LeonarddeR]

Related issues, PRs or discussions

none

What is the current state of the codebase?

braille.BrailleDisplayGesture defines the routingIndex attribute used for routing to specific cells with a braille display when pressing routing keys.

Why are changes required?

There are several issues with this approach/wording:

1. The scope of routing is too small. For example, Handy Tech displays have the so called Active Tactile Control feature that let's you register what/where you're reading. In that case, it makes sense to generate a braille display gesture with the current reading index or indexes. Using cellIndexes makes clear that the gesture has to deal with the index of a specific cell, whether it is routing or other behavior related to that particular cell, such as touching it.
2. Using a singular index has several disadvantages, most notably that pressing multiple routing keys causes unspecified behavior.

What technical changes are required?

1. Create a new list property called cellIndexesonBrailleDisplayGesture`
2. Deprecate the older routingIndex property using a getter/setter. routingIndex is always an instance property, so deprecating it on a getter/setter behavior should ensure backwards compatibility. setter with value should set cellIndexes = [value], getter should return the first value of cellIndexes
3. Ensure that all builtin drivers set cellIndexes instead of routingIndex, let them appropriately deal with multiple routing buttons pressed. When multiple routing buttons are pressed, use the multiRouting id instead of the routing id.
4. Bonus, create a script that lets you select text using multi routing, e.g. pressing routing buttons for cell 1 and cell 4 selects the word from cell 1 up to and including 4.

Are the proposed technical changes API breaking?

No

Are there potential risks or issues with the proposed implementation?

Backwards compatibility should be considered carefully, as well as testing should be performed with the braille display drivers. Multi routing should only be implemented for displays that can be tested.

[copilot-swe-agent]

The agent encountered an error and was unable to start working on this issue: The service is temporarily unavailable. Please try again later. (Request id: E427:19B8F8:E32985A:AA1C886:69E8F70B)

[LeonarddeR]

Thoughts @bramd, @Emil-18, @CyrilleB79?

[CyrilleB79]

Disclaimer: I'm not at all an expert of braille code, nor a frequent braille user.

That being said:

2. Using a singular index has several disadvantages, most notably that pressing multiple routing keys causes unspecified behavior.

Can you be more specific?
Currently with routing gestures:

• is the NVDA gesture triggered upon key down or key up? Or not clearly defined, e.g. depending on the driver / device?
• What happens today, i.e. which gesture(s) is triggered, in the two following cases?
  1. routing cell1 down, routing cell 2 down, routing cell2 up, routing cell1 up
  2. routing cell1 down, routing cell 2 down, routing cell1 up, routing cell2 up

getter should return the first value of cellIndexes

Depending on the answer to my previous question, this may or may not be backward compatible in case of multi-routing presses.

4. Bonus, create a script that lets you select text using multi routing, e.g. pressing routing buttons for cell 1 and cell 4 selects the word from cell 1 up to and including 4.

Nice idea!

So there would be a gesture with 2 routing cursors, let's call them left and right.

But I wonder if excluding the character of right routing cursor would not be better, because:

1. Else there is no way to select only one character
2. This would be closer to what happens visually, especially when selecting from right to left (anchor at the end of selection), so we would probably avoid side effects that could occur if we include both ends.

[bramd]

Basically looks good to me, but I wonder how consistent the current behavior is when pressing multiple routing keys on various devices. Since this behavior is undetermined I'm not sure if we should strive for full backwards compatibility.

I see your use case for ATC, but is a simple list of cells enough for that functionality? If I remember correctly, ATC gives raw pressure data as well. If you do the abstraction to reading positions and you end up with a list of cells that the user is touching that might work. Besides the ATC case, having multi routing gestures is an interesting improvement by itself.

[LeonarddeR]

@CyrilleB79 wrote:

2. Using a singular index has several disadvantages, most notably that pressing multiple routing keys causes unspecified behavior.

Can you be more specific? Currently with routing gestures:

* is the NVDA gesture triggered upon key down or key up? Or not clearly defined, e.g. depending on the driver / device?

The latter. Most modern drivers execute gestures on the first key release, e.g.:

1. press key 1
2. Press key 2 (hold key 1)
3. Release key 2: executing key1+key2
4. Press key 2 again
5. Release key 1: again executing key1+key2

This applies to displays that send both key press and key release. I think most of them do otherwise combining keys is a pain. However, several drivers directly execute a gesture when pressing a routing key, and in that case, multi routing wouldn't be possible.

* What happens today, i.e. which gesture(s) is triggered, in the two following cases?
  
  1. routing cell1 down, routing cell 2 down, routing cell2 up, routing cell1 up
  2. routing cell1 down, routing cell 2 down, routing cell1 up, routing cell2 up

I believe Handy Tech and Brailliant execute routing+routing. The stored keys are stored in a set, so the routing index attached to the gesture is dependent on python set order. For Baum, the highest cell index takes precedence, the routing id is duplicated for every routing key, so also routing+routing.

getter should return the first value of cellIndexes

Depending on the answer to my previous question, this may or may not be backward compatible in case of multi-routing presses.

Drivers that will support multi routing will always have a multiRouting identifier in their gesture, so no longer the routing+routing+routing stuff. I wouldn't put this in the category of compatibility per se. It would be a new feature.

But I wonder if excluding the character of right routing cursor would not be better, because:

That makes much sense to me.

@bramd wrote:

Since this behavior is undetermined I'm not sure if we should strive for full backwards compatibility.

I think we can't. For example, multi routing just can't be added to displays that route on press, that would require a driver change that is out of scope for this feature. It can only be added for displays that route on release.

I see your use case for ATC, but is a simple list of cells enough for that functionality?

Not really, but I definitely need the list of cells for this. I already looked into the multi indexes case during my time at @BabbageCom, but never came to propose it.

ATC is a bit out of scope here, but in short, I probably need to subclass BrailleDisplayGesture, also store the pressure per touched cell, and then come up with something that calculates position and reaching end of line correctly.