Ability to read specific columns in list views on demand

[nvaccessAuto]

Reported by RamboUdin on 2010-08-16 09:23
Reading list view is very important. To read listview, i suggest NVDA should have the listview commands, such as ctrl+nvda+1 to read the first column heather along with it's text, CTRL+NVDA+2 reads the second heather with text and so on. This features will be great to users.

[nvaccessAuto]

Comment 1 by jteh on 2010-08-16 10:23
Changes:
Changed title from "Keystrokes wanted" to "Ability to read specific columns in list views on demand"

[nvaccessAuto]

Attachment sysListView32.patch added by pvagner on 2011-01-05 09:41
Description:

[nvaccessAuto]

Comment 2 by pvagner on 2011-01-05 09:55
Recently after reading some posts in the development list I've found out NVDA does not honor proper order of listview columns when reading list item values.
I've reimplemented listItem value property and I am gathering all the header and subitem texts using win32 api functions. There is also an ability to report a specified columns on demand by pressing NVDA+control+1 through NVDA+Control+9.
Some points I'd like to get feetback on before committing this into main:

• Now all column headers are presented including the header name for the initial subitem. This might be too verbose. Since original MSAA implementation reported most of these column headers, I've implemented it so we can discuss it further if needed.
• Perhaps there is a room for code optimization in terms of efficiency but I believe there is a lot of inter process communication going on while retrieving the texts. Comparing it with previous implementation it may be a little more resource intensive.

The main question is do we want it in NVDA?
What can be improved and what do you dislike?
This can be tested e.g. in the my computer window when the list of files is set to details.

Again what this patch brings in very simple terms:

• adds reporting for all column headers
• respects order of the list view columns
• Adds ability to read and copy individual columns on demand.

[nvaccessAuto]

Comment 3 by jteh on 2011-01-05 10:37
I haven't looked at your code yet. However, one thing worth noting is that I believe the column reading needs to be made more generic, as we can use this elsewhere in NVDA. I haven't given this a great deal of thought yet, but I envisage some sort of mix-in class (probably a behaviour) which implements scripts that call a method implemented by subclasses. This way, we can utilise this functionality in Mozilla tree tables, etc. Eventually, scripts could be added to allow one to move down only reading the current column, just like we do for tables in virtual buffers.

It sounds like you've already done the bulk of the work for list views. We just need to abstract it so it can be used elsewhere.

Regarding cross-process calls, we might need to move this in-process if it proves to be too slow.

[nvaccessAuto]

Comment 4 by pvagner on 2011-01-05 11:36
I's thinking about this a little as well.
I think individual listview columns are not really focusable so we would need to emulate something which feels like navigational feature. Something like cursor manager for edit controls.
Perhaps we might just agree on a common implementation for retrieving columns and then we can make both list view controls as well as mozilla tables let inheriting from this.
For reviewing NVDA+Control+numbers is already very familiar idea to most users who are coming from other AT's.

[nvaccessAuto]

Comment 5 by pvagner on 2011-02-05 13:25
I have reimplemented these changes as a NVDA global plugin so you can test it with any recent binary snapshot. The file is attached.

[nvaccessAuto]

Attachment enhancedListViewSupport.py added by pvagner on 2011-02-06 08:00
Description:
Global plugin implementing the added functionality

[nvaccessAuto]

Comment 6 by k_kolev1985 on 2011-02-06 19:49
Hello,

Let me express my thoughts on the subject, if I may:

1. In the future, there should be an option whether to report column headers or not, because sometimes it is useful to know them, specially when the column in question is after the 9th in the order of columns in the ListView and we can't check its name by pressing CTRL+INSERT+1-9.
2. If possible, there should be the possibility to navigate through the columns from left to right by some keystroke (maybe INSERT+SHIFT+left and right arrows (SHIFT, because CTRL is now occupied by the "Settings ring")). And if when NVDA performs this kind of navigation it speaks the columns headers, that solves the problem from my previous suggestion (p.1) of not knowing the header of a certain column from the list.
   Thanks for reading!

[nvaccessAuto]

Attachment enhancedListViewSupport.2.py added by norrumar on 2011-02-13 21:56
Description:
Test plugin for columns list navigation

[nvaccessAuto]

Comment 7 by k_kolev1985 on 2011-04-04 15:22
Hello,

The last version of this plugin with implemented column navigation is great! I press CTRL+SHIFT+INSERT+Left/Right arrow and NVDA reads the previous/next column of the currently selected item. CTRL+SHIFT+INSERT+Down Arrow reads the currently selected column and if pressed twice, it copies the column content together withe column header to the clipboard, witch may be very useful in some cases.

But there is an issue, witch I would like to report: The usage of this plugin breaks the workarround implemented in uTorrent for reading the torrent name column in the files list when adding a torrent for downloading. So if I use this plugin, NVDA doesn't read that name column.

This is for now. If I incounter any other issues in this plugin - I'll let you know.

[nvaccessAuto]

Comment 8 by k_kolev1985 on 2011-06-04 09:06
I've found another issue. While this plugin is loaded and used by NVDA, there are problems in NOD32's interface. In short, if I point the mouse cursor (with mouse tracking enabled) over a list item in NOD32's interface (e.g. in the list of scan results), the egui.exe (the process witch creates the graphical interface of NOD32) crashes. Can someone please look into this and eventually fix it!? Because I like this plugin, but was forced to stop using it due to the crashing of NOD32 that it causes.

[nvaccessAuto]

Comment 9 by norrumar on 2011-08-15 16:28
Hi, I have look at this ticket today, and I have seen issues when using Torrent and Nod32. I don't use theese programs and so I can not test this plugin with those applications.
I have changed the plugin and now it is possible turn off and on it, pressing NVDA+a. When it is activated or turned off, we must change the focus or moving PC cursor after that. In other words, the plugin should be turned off befor focusing a list or application if you know issues using it.

[nvaccessAuto]

Attachment enhancedListViewSupport.3.py added by norrumar on 2011-08-15 16:32
Description:
enhancedListViewSupport 3

[nvaccessAuto]

Comment 10 by k_kolev1985 on 2011-08-15 18:35
OK, Thanks. The option to turn it off while working in those problematic applications is a solution. Not a perfect one, but it is better than nothing. I will just have to remember to keep it turned off when I don't need it, or to turn it off when I start using the lists in NOD32, because otherwise the plugin causes NOD32's interface process to crash.

[nvaccessAuto]

Comment 11 by norrumar (in reply to comment 10) on 2011-08-15 21:31
Replying to k_kolev1985:

OK, Thanks. The option to turn it off while working in those problematic applications is a solution. Not a perfect one, but it is better than nothing. I will just have to remember to keep it turned off when I don't need it, or to turn it off when I start using the lists in NOD32, because otherwise the plugin causes NOD32's interface process to crash.

The plugin will not really be turned off, because the script to toggle the possible problematic function is writen in this file. I forgot it in my previous comment. I think that, turning it off, you can work without problems, but I can not test Nod32. For turning plugin off by default, you can change the value of the global var turnOff, in enhancedListViewSupport.py. So, if you need the plugin, you wil remember that you have tu turn it on.

[nvaccessAuto]

Comment 12 by norrumar on 2011-10-22 16:17
In the last version of NVDA, in development, this plugin fails on Desktop window, and when a list has just one column. It produces problems and so I have modified the file. I have added localization code for the plugin, and a possible spanish translation.

[nvaccessAuto]

Attachment enhancedListViewSupport.4.py added by norrumar on 2011-10-22 16:20
Description:
Plugin for NVDA's development version, ready to be translated

[nvaccessAuto]

Attachment enhancedListViewSupport.exe added by norrumar on 2011-10-22 16:24
Description:
Extract into the folder for user settings: %appdata%/nvda in installed versions. Plugin with spanish translation

[nvaccessAuto]

Attachment enhancedListViewSupport.5.py added by pvagner on 2011-12-03 20:56
Description:
Updated the plugin to also handle Mozilla tables

[nvaccessAuto]

Comment 13 by pvagner on 2011-12-03 20:57
Hello,
I have attempted to update this plugin.
Since last version posted by norrumar it brings the following changes
* turn on / off feature is removed since I don't like it and exceptions should be handled seperatelly.
* Nod 32 self defense feature most likelly breaks when attempting to block our in process calls. This plugin will no longer enhance listview controls in that application.
* In the utorrent torrent content list this plugin will no longer enhance the listview because there was a conflict.
* Changed column navigation keybindings to ctrl+alt+left / ctrl+alt+right to be consistant with table navigation commands and to free the keys for the new unified laptop layout proposed in #804.
* removed reset column navigation and report current column bindings.
* Attempted to abstract the implementation a little so it can be used elsewhere.
* Implemented individual column reporting and column navigation for the mozilla tables such as the message list in Thunderbird.
* I don't know how to reliably retrieve column headers so in the mozilla tables it currently only reports columns without their respective headers.

This is verry simple column navigation implementation. While Mozilla tables implement each column as an individual IAccessible object in case of listview controls I would need to emulate that at the NVDA's side. I havent tried doing that yet. It may help to make this more universal but definatelly is more complex and I am not sure is worth the troble right now.
I'm looking for your comments.

[nvaccessAuto]

Comment 14 by k_kolev1985 on 2011-12-03 21:53
Alright, here are my comments on the changes:

1. Yes, I also think that exceptions should be handled separately
2. Thanks for adding the exceptions for uTorrent and NOD32. And your explanation of why does NOD32's egui.exe process crash makes sense, at least to me. Because it seams that NVDA is trying to access NOD32 in a way, witch the antivirus self-protection mechanism doesn't allow, but it doesn't handle properly as well.
3. The changes to the keystrokes sound reasonable and logical to me and I approve them.
4. The removal of the commands to report the current cell and to return to the first cell in the row however, I personally do not approve. I find them usefull, specially the double press of down arrow to copy the content of the cell to the clipboard, witch I think is not available anymore.
5. Column navigation in Thunderbird's list of messages is a very good and useful feature. I hope that in the future, the reporting of headers can be made possible in there as well.
   Thanks for the good work and improvements!

[nvaccessAuto]

Comment 15 by mk360 on 2011-12-03 22:53
Hi,
It has some problems in explorer and emule: In explorer I can't read the colums with NVDA ctrl num, and in emule NVDA read the list columns in a diferent order than the usual.

[nvaccessAuto]

Comment 16 by pvagner on 2011-12-04 01:23
yes the plugin also fixes the order of listview columns.
MSAA implementation provides static order of columns which may not be the same as the sighted people see on the screen. After adding this plugin you are supposed to get the verry same order of columns sighted people can see on the screen. Thus I see this as a feature rather than as a bug. Perhaps list columns in emule can be dragged into the order you are used to.
What are you getting instead while using ctrl+nvda+numbers in the explorer? Does alt+ctrl+left / ctrl+lt+right work in there?

[nvaccessAuto]

Comment 17 by pvagner (in reply to comment 14) on 2011-12-04 01:28
Replying to k_kolev1985:

1. The removal of the commands to report the current cell and to return to the first cell in the row however, I personally do not approve. I find them usefull, specially the double press of down arrow to copy the content of the cell to the clipboard, witch I think is not available anymore.

This is still possible with NVDA+ctrl+number so unless you are in a huge list you can still copy.
I think it would be better to report / copy the selection rather than just the current column. This does not do the same but I think is much better.
Also finally you can copy using the review cursor if you really miss it. In the tables on the web / in word we don't have report current cell command either.

[nvaccessAuto]

Attachment nvda.log added by norrumar on 2011-12-04 11:40
Description:

[nvaccessAuto]

Comment 18 by norrumar on 2011-12-04 11:57
Hi. I don't like the feature for turning on this plugin, and I also think that exception should be handled separately if possible. I like the new keystrokes too. In the other hand:

1. I think that reporting the current column is useful: so you can down and read the same information for different items. Now you can do it in tables, but in this plugin control+alt+down and up arrow don't move to next and previous item, and in fact now theese keystrokes are not attachet to scripts.
2. If control+alt+down arrow reads the current column, it can copy the text to the clipboard pressing two times, for instance. You always can copy the whole item, but sometimes you could want to save in a file a list of names shown in different items (in the first column), for example. Of course, I think that a screen reader should not have this feature in its core, but it can be developed in plugins.
3. Coming buck to the first column is useful now: if you are reading, for example, the tenth column in a list, using control+alt+right or left arrow, and then you go to a list that have 5 columns, the plugin produces a bug. I show it in the attached log file.

[nvaccessAuto]

Attachment enhancedListViewSupport.6.py added by pvagner on 2011-12-04 16:30
Description:
Updated version with an ability to report selection

[nvaccessAuto]

Comment 19 by pvagner on 2011-12-04 16:31
Some more updates this time even various internal changes as an addition to a new big feature:
* Up to now text of listview items was stored as a value property at the NVDA's side. Now changing that to name to be consistent with the mozilla table implementation.
* Implemented ability to report selection in listview controls and mozilla tables. You can use NVDA+Shift+up to report and review cursor to copy.
* Get rid of the global variable colNav. This is stored at the class level instead. Except of being nicer this fixes a bug reported in comment 18
* Use watchdog.cancellableSendMessage rather than winUser.sendMessage where possible.
* Cleaned a bit of code to be more readable hopefully.

Note: I've realized Windows explorer in Windows 7 uses UIAutomation rather than MSAA thus our column navigation does not apply in this case. UIA list views already implement column navigation so I am not sure I'll get into adding more features to those controls.

[nvaccessAuto]

Comment 20 by PZajda on 2011-12-04 17:15
I have a problem on explorer under XP.

Usually, when I move from each items, NVDA read headers too. For example : "file.txt; Size: 12 KO" and with the plugin, NVDA only read "File.txt; 12 KO".

[nvaccessAuto]

Comment 21 by norrumar on 2011-12-04 17:49
I think that making consistent navigation for lists and tables can be a good idea. And global values could be uggly if they are not into a class. But now, when you press down or Up arrow, colNavItem is set to 0, and reading the same column for different items is difficult. NVDA always reports the first column.

[nvaccessAuto]

Comment 34 by jteh on 2011-12-12 12:46
For the record, I don't think this idea or plugin will work for 64 bit applications. The !ListView structures are different for 64 bit. Even if you handle that, you will probably receive 64 bit pointers and you can't safely read remote memory from a 64 bit pointer using a 32 bit process.

[nvaccessAuto]

Comment 35 by pvagner (in reply to comment 34) on 2011-12-12 13:01
Replying to jteh:

For the record, I don't think this idea or plugin will work for 64 bit applications. The !ListView structures are different for 64 bit. Even if you handle that, you will probably receive 64 bit pointers and you can't safely read remote memory from a 64 bit pointer using a 32 bit process.

Huh, then I think this plugin actually makes more harm than adding enhancements.
Isn't this the case about all the pointers we are receiving from the sendmessage calls?
E.G. what about 64 bit compile of akelpad?

[nvaccessAuto]

Comment 36 by k_kolev1985 (in reply to comment 33) on 2011-12-12 15:15
Replying to pvagner:

Replying to k_kolev1985:

I don't know after witch update of the plugin it started (I've noticed it a few hours ago), but now NVDA doesn't read the first column in the list of devices in the USBDeview application (http://www.nirsoft.net/utils/usb_devices_view.html). This may occur in other applications as well,

If you explore further you will notice your first collumn is in fact reported but it's reported in the wrong order.

The update I've just posted fixes list item reporting when the order of columns is overridden.

Please test if this fixes your issue.

Norumar's resort to display text is a hack in this case because I believe it can be fixed properly.

Still if you think it's wrong please let me know and I'll try to fix it.

v10 of the plugin also fixes the issue in USBDeview. Thanks!

[nvaccessAuto]

Comment 37 by norrumar on 2011-12-12 16:41
I think that you could enhance this plugin, because now some information is repeated. I'm going to attach a log file for show it. I have change the order of two columns: description and Driver description, and they are repeated. I have disabled device name column, but it is spoken.
I think that using displaytext is not a very good idea, because headers can not be read, and application or specific windows have to be attached to another class.

[nvaccessAuto]

Attachment nvda.2.log added by norrumar on 2011-12-12 16:42
Description:
Changed columns are repeated.

[nvaccessAuto]

Comment 38 by jteh (in reply to comment 35) on 2011-12-12 22:10
Replying to pvagner:

Isn't this the case about all the pointers we are receiving from the sendmessage calls?

E.G. what about 64 bit compile of akelpad?

If you use !VirtualAllocEx, I suspect (though am not 100% certain) that it will only ever allocate memory within the range allowed by 32 bit. It should throw an error if it can't manage this, which is one of the reasons i recently added error checking to that function. The issue arises when you get back a struct which contains 64 bit pointers allocated by the remote process itself and not by NVDA. Perhaps this isn't an issue for your code. I confess I haven't tested it yet myself or read the code. It's certainly something to check for.

[nvaccessAuto]

Attachment enhancedListViewSupport.11.py added by pvagner on 2011-12-13 06:45
Description:
Fixed issue reported in comment:37

[nvaccessAuto]

Comment 39 by pvagner on 2011-12-13 06:48
Fixed issue reported in comment:37 however I am unable to fix the issue where hidden columns in the usb deview are reported.
There is no way on how to officially hide listview columns. It's either possible to remove / readd them when needed or use subclassing to do the work manually. I am unable to figure out which columns are hidden in the usbdeview.

[nvaccessAuto]

Comment 40 by norrumar on 2011-12-14 08:11
This last version is really great for me. We can hide or read columns subclassing in other programs too. I want to use this version, whith the script that announces the current subitem.

[nvaccessAuto]

Attachment enhancedListViewSupport.12.py added by norrumar on 2011-12-14 08:14
Description:
Just the previous version whith a script for reading the current subitem: control+alt+down arrow

[nvaccessAuto]

Attachment nvda.3.log added by k_kolev1985 on 2011-12-21 20:31
Description:
NVDA errors in uTorrent

[nvaccessAuto]

Comment 41 by k_kolev1985 on 2011-12-21 20:35
I've noticed from some days now, that even though this plugin has an exception for uTorrent, there still some issues with uTorrent's files selection list in uTorrent's "Add a new torrent" dialog. On each selection in that list, NVDA gives its error sound and does not read the selected item in the list. I'm attaching a saved NVDA log file after such an occasion with errors.

[nvaccessAuto]

Attachment enhancedListViewSupport.13.py added by norrumar on 2011-12-22 21:26
Description:
The plugin with a new exception for AVG and modified code for Utorrent, just for testing it

[nvaccessAuto]

Comment 42 by norrumar on 2011-12-22 21:29
I have found another issue using AVG. So I have added a new exception for this program too.

[nvaccessAuto]

Comment 43 by PZajda (in reply to comment 25) on 2011-12-24 13:39
Replying to pvagner:

How would you say which headers don't report?

E.G. consider list in the windows explorer window. Set it to display date in the first column. Which header you don't want to have reported in such a configuration?

Or are you saying no mather how the columns are ordered the first one should not have its header anounced? If yes, then this can be done.

Yes, it is what I am saying. It could be great to choose if we want or not the first header. In explorer window, if I decided the first column is date, I know the first header is date, so I don't want NVDA to announce it. Moreover, it makes more complicated to read the list with braille if the first header is written in braille too.

[nvaccessAuto]

Comment 44 by k_kolev1985 on 2012-01-05 19:47
I don't know why, but the column navigation does not seam to work as it should in Thunderbird's list of e-mails. I'm using Thunderbird 3.1.17 localized in bulgarian (but I doubt that the localization matters). The problem is that when I press any of the combinations for list column navigation in Thunderbird's list of e-mails, nothing happens. As far as I could test it, it works as it should in Thunderbird 9.0.1 (also localized in bulgarian), but I don't want to use That latest version of Thunderbird, due to accessibility issues in it.

[nvaccessAuto]

Attachment enhancedListViewSupport.zip added by norrumar on 2012-04-02 13:30
Description:
Plugin contained in a folder, with spanish translation.

[nvaccessAuto]

Comment 45 by pvagner on 2012-04-19 06:02
I have looked at crossing 32 bit app to 64 bit foreign processes and it requires hooking, enhancing our loader etc. I feel this is too complicated for me to continue development of this plugin.
I'm verry sorry to say this but I really feel unable to do this.

[nvaccessAuto]

Comment 46 by Loco O.G. on 2012-05-14 15:26
How likely is it this functionality will be included natively? I know there are complications due to each application having its own headers and some users not wanting specific headers to ve spoken or Brailled. Would it work if there is a new key combination key + another key that returns a list box of the headers with each of them being its own check box. When checked that means spoken/Brailled and when not checked means the opposite. Pressing Tab places focus to a set of 3 radio buttons only if the check box is checked: spoken, Brailled or both. Each header can be separately configured to fully meet the user's particular wants/needs.

I know this is probably not likely if likely at all but for basic native functionality theoretically I think it could work.

[nvaccessAuto]

Comment 47 by jteh (in reply to comment 45) on 2012-08-11 12:17
Replying to pvagner:

I have looked at crossing 32 bit app to 64 bit foreign processes and it requires hooking, enhancing our loader etc. I feel this is too complicated for me to continue development of this plugin.

Actually, it looks like you don't have to do this. All of the pointers are caller-allocated, so we just have to change the struct depending on the bit width of the remote process.

I'm going to take a stab at integrating this into the core alongside other work I'm doing on tables for NVDAObjects. Thanks for your great work so far; I'll be borrowing a lot of code. :)
Changes:
Milestone changed from None to near-term

[nvaccessAuto]

Comment 48 by jteh on 2012-08-17 12:12
Bzr branch: http://bzr.nvaccess.org/nvda/listTables/

This takes a slightly different approach in that cells are always exposed as objects inside the list item, as is done natively for Mozilla. One advantage of this is that you can review it just like any other object. Control+alt+left/rightArrow aren't really needed (as next and previous navigator object will do the same), though I've implemented them anyway. I've also implemented control+alt+up/downArrow to move focus and navigator object to the previous/next row, keeping the navigator object in the correct column. I'm beginning to have second thoughts about this, as this mixes focus and review in a weird way. It also doesn't change the selection.

This does support 64 bit list view controls.

[nvaccessAuto]

Comment 49 by jteh on 2012-08-21 06:02
Given that you can move by both column and row, is there really a need for NVDA+control+1-9? These seem like a waste of keys to me, as you can now move to a column and then read only that column for each item. It's worth noting that we don't provide this for tables in browse mode, even though we do provide table navigation.

[nvaccessAuto]

Comment 50 by jteh on 2012-09-05 07:50
Merged in 808c4c2. Any issues should be filed as new tickets.

I haven't implemented any specific exceptions for apps that reportedly crashed with the plugin code, so please file bugs if these start occurring.
Changes:
Milestone changed from near-term to 2012.3
State: closed

[nvaccessAuto]

Comment 51 by Simon818 on 2012-09-06 10:09
Is it possible to have the position information reported when moving to a specific list view column in NVDA? Right now, routing the mouse gives me an "object has no location" error.

Also, when I try to navigate with ctrl+alt+arrows in outlook express, I get this:
ERROR - scriptHandler.executeScript (03:08:10):
error executing script: <bound method Dynamic_MessageListItemListItemListItemWithReportViewIAccessible.script_moveToNextColumn of <NVDAObjects.Dynamic_MessageListItemListItemListItemWithReportViewIAccessible object at 0x052AA970>> with gesture u'ctrl+alt+right arrow'
Traceback (most recent call last):
File "scriptHandler.pyc", line 165, in executeScript
File "NVDAObjects\behaviors.pyc", line 414, in script_moveToNextColumn
File "baseObject.pyc", line 34, in get
File "baseObject.pyc", line 110, in _getPropertyViaCache
File "NVDAObjects\behaviors.pyc", line 504, in get_firstChild
File "NVDAObjects\behaviors.pyc", line 501, in makeCell
File "NVDAObjects__init.pyc", line 123, in call
File "appModules\msimn.pyc", line 52, in event_NVDAObject_init
AttributeError: '_FakeTableCell' object has no attribute 'windowControlID'

[nvaccessAuto]

Comment 52 by jteh (in reply to comment 51) on 2012-09-06 11:08
Replying to Simon818:

Is it possible to have the position information reported when moving to a specific list view column in NVDA? Right now, routing the mouse gives me an "object has no location" error.

Please file an enhancement for this. Thanks.

Also, when I try to navigate with ctrl+alt+arrows in outlook express, I get this:

AttributeError: '_FakeTableCell' object has no attribute 'windowControlID'

Should be fixed in c991128.