nvda doesn't refresh progressbars in braille #3258

Open
nvaccessAuto opened this Issue Jun 1, 2013 · 23 comments

2 participants

@nvaccessAuto

Reported by halim on 2013-06-01 17:48
Maybe I am doing something wrong but it seems that nvda displays progressbars but doesn't update these on the brailledisplay.
firefox downloadmanager or progressbar when copieing files are useful examples.
I tested it with nvda 2013.1 in windows xp pro.

@nvaccessAuto

Comment 1 by jteh on 2013-06-03 00:15
How are you displaying the progress bar in braille? Are you tethering to review and then moving to the progress bar with object navigation? If not, you're probably seeing the title of the window. In that case, perhaps events aren't being fired (or we aren't handling them properly) when the title changes.

@nvaccessAuto

Comment 2 by halim on 2013-06-03 08:35
Braille display was always tethered to review during my tests. No objectnavigation commands were used.
I.ll try to explain more detailed my tests.

  • In firefox: start a download and try to read the remaining time Currently nvda displaays the time but does not refresh that information
  • In windows explorer: Try to copy a large folder and try to read the remaining time with flat review. I was able to read it but again the information was not refreshed. Moving brailledisplay line up/down refreshs the copy progress properly.
@nvaccessAuto

Comment 3 by jteh on 2013-06-04 00:08
How were you reading the remaining time in Firefox? Were you using flat review or something else?

Flat review doesn't currently support refreshing, which explains the Explorer case. Unfortunately, it looks like Explorer doesn't fire events when it updates the remaining time, so object navigation won't update either. However, the percentage progress bar does fire events, so we can update for that if you move to the object (though I need to fix a bug there first).

@nvaccessAuto

Comment 4 by James Teh <jamie@... on 2013-06-04 01:06
In [16669b3]:
```CommitTicketReference repository="" revision="16669b30d35c54be659cf71bee6377284cdcb2a7"
If a progress bar is shown on a braille display, the braille display is updated when the progress bar changes.

The ProgressBar NVDAObject behavior overrides the valueChange event, but it wasn't notifying braille of the update.
Re #3258.

@nvaccessAuto

Comment 5 by aliminator on 2013-06-04 09:51
I could not figure out any changes since the fix. I tried to download a file with Firefox (internal download manager). As soon as you move up and down via arrow keys you can see its percentage but it still isn't updated.
This also occurs when going to the braille settings dialog and e.g. changing the flash message timeout. Before changing the value it is pre-selected and it is shown in braille correctly. When entering a number the display doesn't get updated.

Maybe the issue described can be related to some other problems other than progress bar as well.
Braille display was tethered to review, too.

@nvaccessAuto

Comment 6 by aliminator on 2013-06-04 10:00
Some corrections:
The last effect describes can be only reproduced in the braille settings dialogue when changing the message timeout. E.g. when a timeout of 4 is selected and one wants to enter 10, pressing 1 does not trigger the update on the braille display. But pressing 0 does trigger it as expected.
So one needs either to enter two digits or to move the cursor keys to get the updates. I think this is a completely different issue.

@nvaccessAuto

Comment 7 by jteh on 2013-06-04 10:52
The fix in comment:4 only applies when you've moved to a progress bar object with review. In Firefox download manager, I'm guessing events aren't being fired when the information is updated. Unfortunately, if that were added, you'd then hear the whole item spoken again every time it was updated.

@nvaccessAuto

Comment 8 by halim on 2013-06-04 10:57
Ok it works now when using object navigation commands. The fix has no effect to described problems of comment 3.
It should be possible to get the brailledisplay refreshed in flat review as well (eg. copy progress in explorer).
Maybe the issues are not the same but I see also no change in firefox downloadmanager.

@nvaccessAuto

Comment 9 by aliminator on 2013-06-04 11:25
I think the point here is to show progresses on a braille display only; meaning separating speech and braille handling.
While on a braille display it makes sense to update progress let's say every second - but when using speech an update frequency of one second disturbs the user and hence, is not sensible.

@nvaccessAuto

Comment 10 by jteh on 2013-06-04 11:32
The key point here is that the areas you're talking about aren't actually progress bars. They're just list items or text. We don't know whether they contain progress or something else and apps often don't fire events to notify us when they are updated.

@nvaccessAuto

Comment 11 by aliminator on 2013-06-06 07:16
NVDA seems to see this list item as progress bar. When triggering a download speech announces its progress every 10 %.
Here is the object info:
INFO - globalCommands.GlobalCommands.script_navigatorObject_devInfo (09:11:39):
Developer info for navigator object:
name: u'eclipse-scout-juno-SR2-win32-x86_64.zip 26% Pause Abbrechen 1 Minute, 6 Sekunden verbleibend \u2014 79,6 von 306 MB (3,1 MB/s)'
role: ROLE_LISTITEM
states: STATE_FOCUSABLE, STATE_SELECTABLE, STATE_FOCUSED, STATE_SELECTED
isFocusable: True
hasFocus: True
Python object:
Python class mro: (, , , , , , , , , )
description: u''
location: (20, 39, 380, 64)
value: None
appModule: <'firefox' (appName u'firefox', process ID 4960) at address 5843710>
TextInfo:
windowHandle: 590832L
windowClassName: u'MozillaWindowClass'
windowControlID: 0
windowStyle: 382664704
windowThreadID: 4964
windowText: u'26% von 1 Datei - Downloads'
displayText: u'\x00eclipse-scout-juno-SR2-win32-x86_64.zip\n\x001 Minute, 6 Sekunden verbleibend \u2014 79,6 von 306 MB (\u2026\n'
IAccessibleObject:
IAccessibleChildID: 0
IAccessible event parameters: windowHandle=590832, objectID=-4, childID=-196446688
IAccessible accName: u'eclipse-scout-juno-SR2-win32-x86_64.zip 26% Pause Abbrechen 1 Minute, 6 Sekunden verbleibend \u2014 79,6 von 306 MB (3,1 MB/s)'
IAccessible accRole: ROLE_SYSTEM_LISTITEM
IAccessible accState: STATE_SYSTEM_SELECTABLE, STATE_SYSTEM_SELECTED, STATE_SYSTEM_FOCUSED, STATE_SYSTEM_FOCUSABLE, STATE_SYSTEM_VALID (3145734)
IAccessible accDescription: u''
IAccessible accValue: None
IAccessible2 windowHandle: 590832
IAccessible2 uniqueID: -196446688
IAccessible2 role: ROLE_SYSTEM_LISTITEM
IAccessible2 states: IA2_STATE_ACTIVE, IA2_STATE_OPAQUE (1025)
IAccessible2 attributes: u'margin-left:0px;text-align:start;text-indent:0px;setsize:1;margin-right:0px;tag:richlistitem;id:dl1;margin-top:0px;posinset:1;margin-bottom:0px;display:-moz-box;'

@nvaccessAuto

Comment 12 by jteh (in reply to comment 11) on 2013-06-06 07:32
Replying to aliminator:

NVDA seems to see this list item as progress bar. When triggering a download speech announces its progress every 10 %.

That suggests there is a progress bar somewhere that NVDA is seeing. If you move to it with object navigation, it will update as you expect. I seem to remember that the progress bar is inside the list item, but I can't remember for sure.

@nvaccessAuto

Comment 13 by aliminator on 2013-06-06 08:17
True, this works. But using object navigation every time for progress bar updates is not very intuitive. Especially for those braille users not having this functionality integrated in their braille displays (as key strokes).

@nvaccessAuto

Comment 14 by jteh on 2013-06-06 08:47
I understand that, which is why the ticket remains open. The question is how to solve it. These list items aren't progress bars, so they aren't treated as such. Even if we did somehow detect the progress bar and just render that, you'd then lose the rest of the information about the item.

@nvaccessAuto

Comment 15 by aliminator on 2013-06-06 09:09
What about using the detection algorithm used to output speech for progress bars? I think it should be enough to braille the change.
For example, "downloading.... 10% " is shown on braille display. When speech gets its update just refresh "10%" and keep the original string by resending it to the display.

What do you think?

@nvaccessAuto

Comment 16 by jteh on 2013-06-07 05:26
Updating the currently focused item based on assumptions about text content and changes in a potentially unrelated progress bar is very error prone. You'd have to search for "%" in the string and assume it's the progress, but it might be entirely unrelated. For example, there could be multiple items or progress bars on the screen.

@nvaccessAuto

Comment 17 by aliminator on 2013-06-07 11:16
Yes, string searches are not reliable for this task.
Does the type (taken from the log):
Python object: <NVDAObjects.Dynamic_ListItemBrokenFocusedStateMozillaIAccessible object
not indicate a progress item in a list?
I think the issue is not to detect progress bars because NVDA already does this job. At the point in the source code, maybe there has to be written a statement which updates the display - at the same method where the progress is presented to speech.
So no complicated routines should be necessary.

@nvaccessAuto

Comment 18 by jteh on 2014-12-01 05:59
I believe Firefox Download Manager now fires events when the list items are updated. Can you confirm whether there are any outstanding issues here?

@nvaccessAuto

Comment 19 by halim on 2014-12-01 07:10
Confirmed. It's now ok in braille.

@nvaccessAuto

Comment 20 by jteh on 2014-12-01 10:06
Thanks. Just to be clear, are there any other outstanding issues here besides Firefox Download Manager?

@nvaccessAuto

Comment 21 by dkager on 2015-05-30 18:15
Another area where braille isn't refreshed is the NVDA update progress dialog. Probably explained by comment:3 and again not technically a problem with progress bar handling itself. It has been a while since I noticed this because updates download very quickly these days.
How involved a change would it be to allow dialogs like these to refresh braille? Simply updating the braille display when the progress bar changes isn't enough because you also have to keep track of display panning. If you don't the display will bounce back to whatever has the focus on every progress bar update. Also I'm guessing speech users won't care to have the entire dialog re-spoken. But for braille it makes sense to re-evaluate a dialog's contents when a progress bar changes. Scraping may be more accurate, but it seems safe to assume that an updated progress bar implies something else was also updated.

Long story short: even though this isn't strictly a progress bar issue I think it should be noted somewhere as an enhancement.

@nvaccessAuto

Comment 22 by jteh on 2015-05-30 22:00
Progress bars themselves don't appear in dialog text. In some cases, there can be other text associated with the progress bar (e.g. the elapsed/remaining times in the NVDA update dialog) and this is often included in dialog text. However, we have no way of knowing that this is the case. Fetching dialog text is very expensive from a performance point of view, so we want to be very sure it's necessary before we do it.

@nvaccessAuto

Comment 23 by dkager on 2015-05-31 10:25
Could this be done based on some threshold, i.e. every 5% of the progress bar or every 10 seconds? Or maybe optimize by fetching only the progress bar's siblings and maybe one level of ancestors? And of course only do this if braille is enabled and maybe only for controls that can actually be shown in braille. This implies a mapping from displayed braille regions to corresponding GUI widgets so you can fetch only that control's new text.
I'm not sure how much is possible without scraping, but I would like to see things like elapsed/remaining time update in braille if it can be done semi-efficiently.

@jcsteh jcsteh was assigned by nvaccessAuto Nov 10, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment