Automatic virtual buffer pass through

[nvaccessAuto]

Reported by valiant8086 on 2008-08-25 16:55
This feature will be available in the next public release of jaws. I have listened to their latest podcast on the technology, and I believe it's a really good idea. FS is going to rock the world with this. I believe such a feature could come in very handy when it comes to browsing the web and filling out forms with NVDA.

Many people have spoken about this on the NVDA mailing lists and there is a lot of disagreement on whether this is or isn't a good idea. I believe it is. Many say that if such a feature existed, we'd wind up typing into edit fields when we meant to use navigation keys, or we'd wind up using navigation keys when we wanted to type into an edit area. Here's my scoop on how we can get around that.

When I use navigation keys, let's say f for moving to the next form, virtual buffer passthrough will not be enabled. This will allow me to press f again or any other navigation key to continue moving to the next desired element.

When I use my arrow keys to navigate through a page or press tab or shift+tab to do the same, virtual buffer passthrough will be enabled. If I continue pressing the arrow keys or tab or shift+tab, virtual buffer passthrough will be disabled, unless I navigated directly onto another element which virtual buffer passthrough is needed for, then it will be enabled again.

there is room for discussion here, much of it, in fact. The idea is that if you press down arrow through a page, you can sure type your email address into the right field, continue down arrowing, and type into another field. This allows one to better take advantage of the layout of a page. It eliminates having to turn on and off virtual buffer passthrough. It eliminates the tendency of users to keep virtual buffer passthrough on and just tab to the next form control, whereby they verry well may bypass important information that was located between the previous form control and the newly focused one. This feature should be togglable on and off. So we may use it the way we always used to if we so desire.

[nvaccessAuto]

Comment 1 by valiant8086 (in reply to comment description) on 2008-08-25 17:14
Replying to valiant8086:

This feature will be available in the next public release of jaws. I have listened to their latest podcast on the technology, and I believe it's a really good idea. FS is going to rock the world with this. I believe such a feature could come in very handy when it comes to browsing the web and filling out forms with NVDA.

Many people have spoken about this on the NVDA mailing lists and there is a lot of disagreement on whether this is or isn't a good idea. I believe it is. Many say that if such a feature existed, we'd wind up typing into edit fields when we meant to use navigation keys, or we'd wind up using navigation keys when we wanted to type into an edit area. Here's my scoop on how we can get around that.

When I use navigation keys, let's say f for moving to the next form, virtual buffer passthrough will not be enabled. This will allow me to press f again or any other navigation key to continue moving to the next desired element.

When I use my arrow keys to navigate through a page or press tab or shift+tab to do the same, virtual buffer passthrough will be enabled. If I continue pressing the arrow keys or tab or shift+tab, virtual buffer passthrough will be disabled, unless I navigated directly onto another element which virtual buffer passthrough is needed for, then it will be enabled again.

there is room for discussion here, much of it, in fact. The idea is that if you press down arrow through a page, you can sure type your email address into the right field, continue down arrowing, and type into another field. This allows one to better take advantage of the layout of a page. It eliminates having to turn on and off virtual buffer passthrough. It eliminates the tendency of users to keep virtual buffer passthrough on and just tab to the next form control, whereby they verry well may bypass important information that was located between the previous form control and the newly focused one.

[nvaccessAuto]

Comment 2 by Iris on 2008-08-25 21:58
This is much the same method that System access uses.
However, how would one get around flash movies?
Would it be like Jaws where the buttons will be visible without the need to get trapped in the frame of buttons and graphics?

[nvaccessAuto]

Comment 3 by jteh on 2008-08-28 06:57
I've started working on this in a bzr branch:
http://bzr.nvaccess.org/nvda/vbufPassThroughImprovements/
This is still very much work in progress.

[nvaccessAuto]

Comment 4 by valiant8086 (in reply to comment 3) on 2008-08-28 21:05
Replying to jteh:

I've started working on this in a bzr branch:

http://bzr.nvaccess.org/nvda/vbufPassThroughImprovements/

This is still very much work in progress.

Hope I'm not making things too confusing. Still haven't figured out how to comment and stuff verry well.

I have a couple more things I want to input on the subject.

I downloaded the test code you've got so far last knight. It's a good start. I'm aware it's not complete yet. What I'd like to do is take a minute to get a discussion on how to take care of some issues this introduces.

You've likely already noticed that when you land on an edit field which has a history of things previously typed into it, and you continue pressing your down or up arrow key, you wind up arrowing through the history inside this edit field, rather than passing by the edit field and continuing on your Marry way. A good place to test this is google.com. I haven't played with the feature enough so far to know how the current progress works when you land on combo boxes, list boxes and the like. So I don't yet know whether the same problem currently exists for such. My idea, though, assuming noone comes up with a better solution, to use alt+up or down arrow keys overridden to open and close things like this, combo boxes, list boxes, and edit boxes. It would, in the case of edit boxes, control whether or not you had access to the history of things previously typed into the current edit field. If you pressed alt+down arrow, access to the history would be enabled and when you then pressed up or down arrow keys, you would be cycling through the history. If you'd pressed alt+up arrow, on the other hand, you'd turn off this access, and so when you pressed an arrow key, you would find yourself arrowing through the rest of the page. The access to the history items should be disabled by default. Alt+up and down arrow keys are and have been a windows keystroke that opened and enabled access to list boxes in internet explorer and, I do believe, in a few cases, elsewhere in windows. Alt+up and down arrows are used in jaws to allow one to elect to navigate through a combo box or list box by pressing alt+down arrow in order to make a selection. Alt+up arrow is used to make the selection and close the list box or combo box, and the user can then press their arrow keys to continue reviewing the page. So I ppropose forceably adding this alt+up and down arrow functionality to edit fields, giving the user an option of whether they want to navigate the edit field's history, or pass on by and go elsewhere on the page.

May have noticed jaws is having this problem too, and it's being a pain in the computer. I don't know if my solution is workable or satisfactory, do to the fact that Jamy and MDCurran dislike overriding default keystrokes. Having said that, anyone got ideas on how this should work.

[nvaccessAuto]

Comment 5 by jteh (in reply to comment 4) on 2008-08-28 22:03
Replying to valiant8086:

You've likely already noticed that when you land on an edit field which has a history of things previously typed into it, and you continue pressing your down or up arrow key, you wind up arrowing through the history inside this edit field, rather than passing by the edit field and continuing on your Marry way.

This code doesn't yet handle escaping pass through mode by arrowing out of a field. This is quite tricky to manage in NVDA, as our scripts are context sensitive. I have a few ideas, but haven't gotten to trying them yet.

My idea, though, assuming noone comes up with a better solution, to use alt+up or down arrow keys overridden to open and close things like this, combo boxes, list boxes, and edit boxes.

As far as I know, alt+up and down arrows are not standard Windows keystrokes for lists/list boxes. They are only standard for combo boxes and edit combos, the latter also being applicable to autocomplete edit fields. In general terms, they are only functional in controls that can be expanded or collapsed. Lists are always expanded, so they do not apply. Therefore, I do not believe we should use alt+up and down arrows for lists, as this is contrary to standard behaviour. Rather, the user should press enter as normal.

[nvaccessAuto]

Comment 6 by valiant8086 (in reply to comment 5) on 2008-09-14 00:57
Replying to jteh:
We seam to be enabling interaction mode at the proper time most of the time, but we're having a lot of trouble getting back into browse mode. I keep landing in edit fields and find myself unable to arrow past them, I must press insert+space. if the search field has a history, I wind up arrowing through the history as I've already mentioned. I believe when I press an arrow key on any control, I should be put back in browse mode, that is, if interaction mode was enabled at all. We've discussed this at length and can't decide what to do, should we use alt+up or down arrow key? should we press enter and escape? We're hesitant to use things because one way or the other we're overriding a keystroke, and using practices that don't really exist without the screen reader, which both are things we want to avoid if possible.

[nvaccessAuto]

Comment 7 by jteh (in reply to comment 6) on 2008-09-14 07:19
Replying to valiant8086:

I keep landing in edit fields and find myself unable to arrow past them, I must press insert+space. if the search field has a history, I wind up arrowing through the history as I've already mentioned. I believe when I press an arrow key on any control, I should be put back in browse mode, that is, if interaction mode was enabled at all.

I'm working on code to allow up and down arrow keys to go through to the virtual buffer for single line edit fields, which will solve this problem. Conveniently, pressing alt+downArrow does allow one to enter the autocomplete list in those fields. I suspect it is unintentional because alt+upArrow doesn't close the list, but you can do this with escape or enter.

Btw, you note that you believe you should be returned to browse mode when you press an arrow key on "any control". This is not ideal; combo boxes and list boxes should not exhibit this behaviour. I assume this was just an oversight in your comment?

[nvaccessAuto]

Comment 8 by valiant8086 (in reply to comment 7) on 2008-09-14 20:09
Replying to jteh:

I'm working on code to allow up and down arrow keys to go through to the virtual buffer for single line edit fields, which will solve this problem. Conveniently, pressing alt+downArrow does allow one to enter the autocomplete list in those fields. I suspect it is unintentional because alt+upArrow doesn't close the list, but you can do this with escape or enter.

I don't think you can, when I press escape, even if I press it twice, and then press an arrow key, I again arrow through the search history. I haven't tried enter yet.

Btw, you note that you believe you should be returned to browse mode when you press an arrow key on "any control". This is not ideal; combo boxes and list boxes should not exhibit this behaviour. I assume this was just an oversight in your comment?

Ah, yeah good point.

Hmm, did we decide how we would get out of combo boxes if we want to continue on in browse mode? currently to get out after you've used alt+down arrow to open a combo box, you have to hit escape like twice, and then insert+space. If you don't hit insert+space, you wind up opening the combo box again I think, either that or changing it's selection without opening it. there's not as many list boxes around so I haven't gotten to play with one of those yet. Also, for multy-line edit, will we be able to arrow out of the edit field if we wish? That is, if we reach the bottom or top, and keep going.

[nvaccessAuto]

Comment 9 by jteh (in reply to comment 8) on 2008-09-14 22:28
Replying to valiant8086:

Replying to jteh:

pressing alt+downArrow does allow one to enter the autocomplete list in those fields. I suspect it is unintentional because alt+upArrow doesn't close the list, but you can do this with escape or enter.

I don't think you can, when I press escape, even if I press it twice, and then press an arrow key, I again arrow through the search history.

This is because the code that forces the arrow keys through to the virtual buffer when focused on a single line edit field is not yet implemented. Even once you get out of the autocomplete list, you'll just end up back in the list again next time you press an arrow key.

Hmm, did we decide how we would get out of combo boxes if we want to continue on in browse mode? currently to get out after you've used alt+down arrow to open a combo box, you have to hit escape like twice, and then insert+space.

Either escape or alt+upArrow should now work. I don't understand why you need a double press. Have you updated from the bzr branch lately?

Also, for multy-line edit, will we be able to arrow out of the edit field if we wish? That is, if we reach the bottom or top, and keep going.

Yes.

[nvaccessAuto]

Comment 11 by jteh on 2008-11-04 10:54
Merged in r2468. There have been some noteworthy changes since my last update to this ticket:

• Split the auto pass through option into two options: "Automatic focus mode for focus changes" and "Automatic focus mode for caret movement".
• Added these two settings to the Virtual Buffers dialog.
• The former (focus changes) is enabled by default; the latter (caret movement) is not.
• For caret movement, the autocomplete list will still appear in single line edit fields. This is why it is disabled by default. Autocomplete needs to be disabled in Firefox to make this useful.
  • Overriding autocomplete when the user has it enabled is controversial. This should be debated in another ticket if desired.
    Changes:
    Changed title from "It could be a nice enhancement if NVDA were able to automatically passthrough the virtual buffer if certain cases are true" to "Automatic virtual buffer pass through"
    State: closed

[nvaccessAuto]

Comment 12 by davidb on 2008-11-18 16:23
In 0.6p2, I'm not getting automatic pass through for role=tab objects (via ARIA). Is there a bug for adding addition roles for pass through? Please re-close if there is, or if I simply need a newer version of NVDA.
Changes:
State: reopened

[nvaccessAuto]

Comment 13 by davidb on 2008-11-18 16:24
E.g. http://test.cita.uiuc.edu/aria/tabpanel/tabpanel2.php

[nvaccessAuto]

Comment 14 by pvagner on 2008-11-18 21:26
Auto focus mode has been implemented after version 0.6 P2 was released. Anyway we do still have unresolved bug with aria tabs not reported correctly in #216 so we may make these 2 dependant on each other if desired I think.

[nvaccessAuto]

Comment 15 by jteh on 2008-11-18 21:31
If you do want to test auto pass through, you can try a binary snapshot.

Incorrect handling of ARIA widgets for pass through is covered by #205. If you don't believe this covers the issue, please open a new ticket.
Changes:
State: closed