New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Braille doesn't follow when extending a selection to the left or right #5770
Comments
I think the underlying issue is the same as #4682, just a slightly different manifestation. In short, NVDA has no idea which end of the selection is the anchor and which end is the active end, since most APIs don't give us that info. We did have some thoughts about how to fix this, but never got around to making them more solid/documenting them. |
It should be possible to have browse mode expose this info. As for Ctrl+Shift+arrows, this info could be obtained by looking at which keys the user is pressing, but that is ugly. |
Sure, browse mode could certainly provide it, but because no other APIs One way we could do it without changing the TextInfo API at all is to A simpler, less expensive way of doing this is to have an One open question, though: while this nicely fixes #4682, I'm worried |
In Notepad++ when you Shift+DownArrow, braille shows the new line below the selection. I think this is how SuperNova works too. So I’m wondering if when extending the selection up/down, NVDA should show the last selected line or the first unselected line, if that makes sense. I also just noticed that if you select downward in Windows Notepad the horizontal line of dots 7-8 is one cell longer than the text content. My first guess would be that this is because of the newline character at the end of the line. Not really an issue, but it may be a bit confusing. Long story short, this definitely needs some more thinking. :) |
Could we catch this by detecting when both the start and the end of the selection changed? In this case it isn't obvious which end is more important, so we could show the start initially. Even when searching backward you would probably want to see the start of the newly found text first. One more detail: does it make sense to default to showing the start of the selection for RTL languages? |
What I would like to propose is a tristate value:
But there doesn't seem to be a good single point of definition for this. My understanding is that at least EditableText and CursorManager need to be able to expose this value. But the nearest common ancestor is ScriptableObject, which is a bit too high up in the hierarchy. Alternatively the property could be implemented as int, with negative value for start, positive for end, and zero for unknown. Or even a boolean that may also be None. Key point is that the unknown state needs to be conveyed somehow. That said, calculating the anchor for the selection in EditableText turns out to be quite reliable in standard controls. This strategy looks like a good choice. |
Hum, just read up on the mix-in classes like CursorManager. I think I get that now, but am not entirely sure about the relation between NVDAObject and TreeInterceptor. I see that a braille.Region can contain either one as its The developer guide mentions that events not processed by a TreeInterceptor are passed to the NVDAObject, but clearly this does not mean I can ignore TreeInterceptors. :) |
In the past, we discussed having some subclass of ScriptableObject for
things which have a TextInfo; i.e. NVDAObject and
DocumentTreeInterceptor. There didn't seem to be much point back then,
but if we're adding more common stuff, maybe it makes sense now. The
question is where to put it and what to call it. It could go in
baseObject, but I feel like I want it to go in textInfos. And maybe it
could be called TextProvider or something.
Regarding the tri-state, I certainly understand that you sometimes can't
work out where the anchor is, but you still need to do something sane
and consistent in those cases. I imagined we'd just pick either start or
end for those cases and be done with it, which would make speech and
braille consistent. However, I guess it does make sense to try to
preserve the unknown state so that consumers have a choice if they might
do something different to start or end in that case. How do you intend
to handle the unknown case for braille?
If we do have a tri-state, I'd go for the constants, but probably
shorten them to SELANCHOR instead of SELECTION_ANCHOR just to make them
shorter. :)
|
I think it makes sense if we use a tri-state variable with constants, but if it is going to be a boolean that alone doesn't justify this subclass IMO.
Currently I use a boolean that can be True/False/None. If True the selection is anchored at the start, else it is anchored at the end or unknown. In both of the latter cases I have braille scroll the start into view. My reasoning is that it makes the most sense for LTR languages, and if it's anchored at the end you obviously want to show the start since that can potentially move. Here None can mean:
It is possible to add a value indicating the selection hasn't changed too, but I can't think of a use case. In fact, if instead we set a sane default directly in EditableText the None value is avoided. This seems good enough for now. |
Ugh, I'm running into an issue with scrolling to the right end of the selection. In WordPad you can apparently navigate onto the carriage return at the end of the line, and even select it. In NVDA 2016.1 this leads to the selection indicator (dots 7 and 8) extending one cell beyond the text. But if I deliberately scroll the selection's end into view I get a LookupError from regionPosToBufferPos. My first guess is that this is because TextInfoRegion trims line endings. In Word 2016 I can also select line endings, but there braille doesn't show that such is happening. Should something be tweaked to avoid this oddness? What do sighted people see? Note that you have to select forward for the carriage return to be included. |
Regarding Wordpad, when you say you're scrolling the end into view, are
you accounting for the fact that the end of a range is exclusive? That
is, the last character of the selection is actually end - 1, not end.
You don't have to worry about that for the caret because the caret is a
collapsed range.
Regarding Word 2016, I'm guessing the problem is that Word doesn't
include the line ending character when you expand to line. We'd need
some feedback from @michaelDCurran on this. For now, just ignore this
particular quirk. :)
|
No. I'm scrolling to region.brailleSelectionEnd. The idea was that you would always see the next character to be included in the selection when extending. I was relying on the fact that there is always an extra space at the end of the region so the caret/insertion point can move beyond the last character. Clearly in WordPad this goes wrong. This behavior was observed in SuperNova, so I took it as my starting point. Do you think it makes more sense to ensure the moving end of the selection is always visible instead? Regarding Word 2016, not including line endings when expanding to line actually makes for a less confusing experience. I'd say WordPad is the weird one here. :) But if we use the inclusive selction as described above it's not a problem. |
On second thought, there is a serious problem with this. If you are selecting with shift+downArrow it is far more intuitive to see the next line to be selected rather than the end of the selection. |
One more problem that I hope is a bug. Reproducible in Notepad.
The invocation of super also calls detectPossibleSelectionChange(). Could this duplicate call be avoided? It causes my anchor detection to fail because on the second call there is obviously no change. Stack traces attached. |
Regarding the Notepad issue, yeah, remove the
detectPossibleSelectionChange in Edit.event_caret. I have this feeling
it used to be necessary, perhaps because IAccessible.event_caret wasn't
calling/couldn't call super, but even if that were true, that code has
been significantly refactored since then. Obviously, just check it
doesn't break anything for speech, but looking at your stacks, I can't
see why it would.
|
The thing is that in this case, there is no possible character beyond the last that the insertion point can move to. If we go exclusive end, I think we just need to handle that case and try for end - 1; I don't think there's a bug here as such. It's just an edge case.
Firefox, which includes line endings for |
Slight clarification here: the line ending is confusing because in braille it comes out as a space in some braille tables, and also the braille cursor doesn't move if the insertion point is located right after the last character and you press rightArrow to move it to the line ending. This because line endings are trimmed in braille. My feeling is that using the exclusive selection end makes for a better experience, because it is better suited for selecting entire lines with shift+downArrow. It would be good to know what other screen readers do (besides SuperNova). Not that we should copy them just because, but this seems one of those areas that people have already given some thought to. With speech you do hear what was last selected, but in braille this gets confusing if a line is longer than the display length and you end up seeing e.g. only the last 5 characters of the line. Hence my preference for the exclusive end, which would show you the next thing to be selected. |
I think we strip line endings and always convert them to one space anyway, so this bit isn't applicable.
I don't follow this. If the insertion point is located right after the last character, it's already on the line ending, no?
I still don't quite follow this. If you're selecting an entire line, the next thing to be selected is the next line, but we definitely don't want to (and can't) show that. |
|
Can this not be done by finding the last line of the selection and calling An additional point of consideration is that some editors (Notepad++, Eclipse) notify braille when you press shift+downArrow, so the braille display follows while you're selecting lines. This uses the behavior of showing the next line to be selected, which I understand isn't what we want for NVDA. So the question is if we should prefer NVDA's selection detection over the caret events the editor sends during a selection. In short: which has higher priority, caret or selection? Hopefully I've now found all the tricky corner cases! |
…TreeInterceptor. The property defaults to False. Also use this to have braille scroll the selection's end into view if the anchor is at the start. re nvaccess#5770
Pushed some initial code I wrote last week. In particular, there is an attribute CursorManager._lastSelectionMovedStart which is similar but not the same as _isSelectionAnchoredAtStart. Maybe it's best to discuss this approach before extending it to multi-line selections. So far I have tested in Notepad, WordPad, Word and browse mode. |
According to @leonardder JAWS also follows the end of the selection when you select multiple lines. So that should probably be the way to go, instead of showing the next line to be selected as SuperNova does. Especially since that is technically difficult or impossible. |
Picking this patch up again, should finish it this week. There is one remaining task: get braille to show the moving end of a multi-line selection. So here's a heads up on my ideas for that. The good stuff happens in TextInfoRegion.update(). This first grabs the cursor, expands it to a line or paragraph and then grabs the selection and restricts it to that reading unit. This goes wrong if the cursor is reported at one end of the selection while the other end is actually moving. I plan to work around this by replacing the cursor with the line at the moving end of the selection. This should result in the behavior as described above for JAWS. It sounds feasible, but real-world tests will have to tell... |
A few changes to braille.TextInfoRegion.update() seem to have done the trick, except for a few corner cases. However, my solution shows you the line that is about to be selected rather than the end of the last line that is selected. Presumably because of the way TextInfo.expand() is implemented. That is exactly the behavior I was looking for initially. I spoke to @leonardder and he agrees. But @jcsteh said this is technically impossible and also not what should go into NVDA. How can we proceed? I can push my code so we can verify if it is technically sound, but how do we reach consensus on the desired behavior? Should we address the issue of single-line selection first and then the multi-line issue separately? |
The reason why I agree is that, as soon as you select something on one line with shift+down arrow and your cursor was on the first character of that line initially, the cursor is now at the first character of the second line which is about to be selected when you press shift+down arrow again. Thus, I assume the cursor is on the second line visually as well, so that's the line the braille display should show. |
That doesn't currently work because panning past a line causes the TextInfo to be asked for the previous/next line, which typically moves the cursor. The first paragraph Leonard wrote probably explains why this behavior is so intuitive to me. Similar logic can be applied to Shift+rightArrow: braille should pan if all but the last character visible on the display are selected, since the cursor is ahead of the selection. But I remember hearing that this is not how it works visually. Still, we should pick the most intuitive solution IMO. One problem with this approach is that some TextInfos cause errors when they encompass the last character, you collapse it to the end and then expand to line. At first glance this collapses onto |
…TreeInterceptor. The property defaults to False. Also use this to have braille scroll the selection's end into view if the anchor is at the start. re nvaccess#5770
@jcsteh Could we pick this up again in 2017? I plan to read up on this issue in the week after Christmas. As far as I recall there are two issues:
The rebased code lives in i5770. This is as far as I got and it doesn't work properly, but it showcases the idea we had about detecting the "selection anchor". I got quite good results with this approach, but in some TextInfos it caused exceptions. Anyway, the festive season comes first! :) |
P2 because this prevents braille users from accessing selection properly. |
One of these days I'm going to report an issue not related to braille. Until then...
STR:
The same in reverse applies when you extend the selection to the left. Extending up or down works as expected in most cases (but not in browse mode: #4682). I'm opening a separate issue for this because it concerns selections on a single line.
This is kind of the continuation of #5410, but I only noticed it just recently because now I use NVDA much more than before. My guess is that this is non-trivial to fix, in which case I'd say it's a low-priority issue.
The text was updated successfully, but these errors were encountered: