UIA seems to stall under certain conditions.

[stickbear2015]

Steps to reproduce:

1. start a process like a batch processing job in total recorder or goldwave.
2. open any listview, explorer, etc.

Expected behavior:

the listview should read

Actual behavior:

the listview doesn't read.> Tell us what happens instead.

System configuration:

NVDA version: 2017.2

NVDA Installed or portable: installed.
Other information:

Windows version: windows 10 pro creator update.

Other questions:

Does the issue still occur after restarting your PC? yes

Have you tried any other versions of NVDA? reproduceable from 2016 onward

[josephsl]

Hi, you say you can reproduce this from 2016 onward? What about versions such as 2015.2 or 2015.3? Thanks. CC @michaelDCurran

[stickbear2015]

if it's supported under windows 10, it's been tested thoroughly with the same results. It made no sense to list the 2015 versions when to be honest who's really using them now. But yes, they've been tested. I'm not getting enough log output to be useful, except WARNING - stdout (23:27:57): 1.0

[josephsl]

Hi, can you reproduce this problem while using Narrator and/or other screen readers? If this happens on other screen readers, chances are that another call to Redmond is in order. Thanks for checking into this.

[stickbear2015]

Hello, after testing with JAWS 16, last build available that started to support windows 10, 17, latest build, and 18, latest build, as well as narrator, this issue seems to be specific to NVDA. JAWS and Narrator read listviews like a champ even while running batch processing, saving, etc.

[josephsl]

Hi, can you run NVDA with all add-ons disabled and see if that makes a difference? Thanks.

[stickbear2015]

just restarted NVDA with all addons disabled, with same results.

[josephsl]

Hi, Thanks for all this. At this point, I think it might be best to see what @michaelDCurran asks us to do. Thanks.

[jcsteh]

Your steps to repro mention using any list view. Let's clarify what you mean by "list view" here. We actually don't use UIA for controls Windows calls ListView; e.g. the list on the Desktop. In contrast, Explorer lists these days actually aren't ListView controls. They're a custom list control specific to Explorer and we do use UIA for those. So, have you tested this with any lists other than Explorer? Please also provide a log at level debug showing the issue. This should give us some idea of where it's freezing. Note that I've definitely seen Narrator freeze when a background app is responding slowly. This is probably related, though it's interesting that you aren't seeing this with Narrator in your case.

[stickbear2015]

@jcsteh I didn't actually know that, and I'm sorry I wasn't completely clear, I thought I mentioned the specific list views. I was doing that thing where I was thinking faster than I was typing, and I thought I'd been clear. To be clear, yes, explorer listviews, or list views in open dialogs, save as dialogs, etc. The desktop is not effected how can I best get the necessary log output for you? I know how to use log viewer, etc, but I wouldn't know where to start copying and pasting, or should I just save the entire log.

[jcsteh]

Ah, so NVDA doesn't actually freeze; non-UIA stuff works fine. In that case, a log probably won't show much; this suggests UIA core just isn't firing events. I've definitely also seen this with Narrator. I wonder why you aren't seeing it with Narrator in this instance. With Narrator, can you try starting the slow background task and then holding down alt and repeatedly pressing tab? Does it eventually stop reading the alt+tab list? You may have to try switching back to the app running the slow task again and then repeating the whole process a few times.

[stickbear2015]

I wouldn't consider it a slow app, just an app that has stolen enough control away or causing something in the background to be a problem with UIA under certain conditions. The machine still chugs along and I can do other tasks except read explorer listviews. It's not one specific app, it's when apps, (mostly audio production apps, or apps that perform a lot), do their thing things start not behaving in explorer listviews. I retested with narrator after a reboot, and it is doing the same thing in list views when large batch processes, or saving operations in programs like goldwave, are in progress. You'd think on a 6th gen core I7 with 16GB of ram this wouldn't even be a thing! :-)

[jcsteh]

Just to be sure we're super clear here, you are saying you can now repro this with Narrator as well? If so, while not ideal, one plus is that I've seen this as well and think I might have a test case here we can send to Microsoft.

[stickbear2015]

James, on further checking, what happened last time when I initially tested it with narrator when josiph asked me to was the task itself completed and I didn't realize it and simply considered it unreproduceable with narrator and logged it as such. During this round of testing, I specifically started a situation that causes the issue in question with narrator running and performed my initial steps with the issue of listviews in explorer, this pc, etc. not reading, narrator as you said it would, refused to read them. I then performed the alt+tab sequence you asked me to perform and after a bit as you said it would, it stopped responding or reading application titles. Do you need anything else before we, as I'm suspecting we might have to do, send this to microsoft?

[jcsteh]

Unless you're able to check whether this happens on an earlier version of Windows, no, I think we've got all we can get for now. Thanks.

[michaelDCurran]

Filed as MS Bug 12608330: Narrator becomes unresponsive in all apps when a single app is continually slow to respond

[stickbear2015]

Can we get a link to track this bug, as i authored this bug here, I'd like to track it to resolution from the microsoft end.

[jcsteh]

I'm afraid not. Microsoft don't provide public access to this bug tracker.

[JavierJF]

Have anybody observed weird behavior regarding to UIA when the TaskManager is running? For me It caused a behavior very similar to this.

[a2hsh]

Hi!
I've seen the same issue when using Virtual Audio Cable. I guess it happens with all CPU-intensive programs.
I've used VAC with Windows 7 before with no problems what so ever.
Hope that Microsoft fix this ASAP.

[josephsl]

Hi,

Resolved in RS5 with IUIAutomation6::CoalesceEvents flag turned on. HOwever, what we really want is a solution that can cover earlier Windows 10 releases. CC @tspivey

[josephsl]

See also #8535

[michaelDCurran]

For Windows versions < Windows 10 RS5, here is a try build that registers UIA property change events per foreground, rather than once globally:
https://ci.appveyor.com/api/buildjobs/qg8hfsrt30xpry5a/artifacts/output%2Fnvda_snapshot_try-perForegroundUIAPropertyChangeEvents-16006%2C613a17ed.exe
this seems to solve the issues on my system. Please give it a go and let me know how it works.
I'm quite confident this should work okay on windows 10, and probably Windows 8.1. But the last time I tried anything like this on Windows 7 (in 2009) it caused freezes so we never continued with it.

[tspivey]

This works well so far, except for NVDA's own dialogs.
STR:

1. Open the About dialog.
   NVDA will freeze at this point.

[Simon818]

Whenever I load a page in Chrome, I get this:
nvda+f5 works fine, but loading a new page or refreshing the current one was throwing errors ... right until I came here to report it, actually. Now it's working fine. In case this is helpful, I'll still leave it here.
ERROR - RPC process 5556 (chrome.exe) (02:25:34.858):
Thread 1796, build\x86_64\vbufBase\backend.cpp, VBufBackend_t::terminate, 234:
Could not execute renderThread_terminate in UI thread

[beqabeqa473]

hi. i also got this message. killing all crhome processes, restarting nvda and then starting chrome again helped. 2018-09-11 14:16 GMT+04:00, Simon818 <notifications@github.com>:

…

Whenever I load a page in Chrome, I get this: nvda+f5 works fine, but loading a new page or refreshing the current one was throwing errors ... right until I came here to report it, actually. Now it's working fine. In case this is helpful, I'll still leave it here. ERROR - RPC process 5556 (chrome.exe) (02:25:34.858): Thread 1796, build\x86_64\vbufBase\backend.cpp, VBufBackend_t::terminate, 234: Could not execute renderThread_terminate in UI thread -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #7345 (comment)

-- with best regards beqa

[Simon818]

I can confirm the dialog issue mentioned by @tspivey. On my Windows 10 machine it seems to only happen some of the time. On my Windows 7 machine, NVDA's own installer froze after I clicked past the first screen, so I was not able to install and test it ... yet. I'll create a portable copy if necessary though.

[beqabeqa473]

i can also confirm, that nvda's own dialogs are freezing nvda 2018-09-11 14:19 GMT+04:00, Simon818 <notifications@github.com>:

…

I can confirm the dialog issue mentioned by @tspivey. On my Windows 10 machine it seems to only happen some of the time. On my Windows 7 machine, NVDA's own installer froze after I clicked past the first screen, so I was not able to install and test it ... yet. I'll create a portable copy if necessary though. -- You are receiving this because you commented. Reply to this email directly or view it on GitHub: #7345 (comment)

-- with best regards beqa

[michaelDCurran]

Here is a new try build that should address NVDA freezing in its own dialogs: https://ci.appveyor.com/api/buildjobs/5a563kep6293nq12/artifacts/output%2Fnvda_snapshot_try-perForegroundUIAPropertyChangeEvents-16008%2C583847a1.exe Please let me know how this works.

[zstanecic]

Hi, i am running the new try build, and i don't see freezing.

[LeonarddeR]

Reopened because #8742 has been reverted due to several reported crashes of Windows Explorer.

Probably related to these crashes are several errors I'm getting when typing into an explorer list view when the folder isn't fully loaded.

explorerCrash.log

[michaelDCurran]

I have tried an entirely alternative approach to solve this. In the unregisterUIAProxyWinEvents branch, I now instruct UI Automation client library to never map MSAA winEvents to UI Automation propertyChange events. NVDA does not need them as it listens to MSAA winEvents itself anyway.
Could people please test the new try build:
https://ci.appveyor.com/api/buildjobs/hcx6iof9p896lvnk/artifacts/output%2Fnvda_snapshot_try-unregisterUIAProxyWinEvents-16059%2Ce53d3adf.exe
Please let me know if:

1. Does it solve the original problems (I.e. Explorer and other UIA apps becoming unusable while an app such as an audio editor is doing heavy processing), and
2. Does this no longer cause crashes in File Explorer like the other pr did?

I'm hoping that this alternative will also address some very long-standing performance issues to do with Task manager, and also perhaps the slow downs in Windows 10 File Explorer after renaming or deleting a file.
However, the main thing trying to be solved here is this issue #7345.

[LeonarddeR]

The problem I see with this approach is that this also kills UIA property change events for objects that are natively MSAA but on which we enforce UIA (i.e. with appModule.isGoodUIAWindow).

No explorer crashes whatsoever.

[michaelDCurran]

WDAG definitely doesn't need these propertyChange events. What appModules currently use isGoodUIAWindow?

[LeonarddeR]

At the moment, None. However, there were two additional reasons I wanted to introduce isGoodUIAWindow, particular for scripting work places:

1. Some applications seem to fire spurious focus events (i.e. on graphics) which enforcing UIA seems to filter out.
2. More importantly, in some situations in turns out to be much quicker to use UIA cache request to work with children of an IAccessible object rather than creating IAccessible NVDA objects for every child and then fetch the desired property

Example code

class AppModule(appModuleHandler.AppModule):
def isGoodUIAWindow(self, hwnd):
	if winUser.getClassName(hwnd).startswith('Mamut'):
		return True
	return False

	...

class MamutBrokenComboBox(ComboBoxWithoutValuePattern):
def _get_value(self):
	# Directly talk to UIA to get the selection, as fetching NVDA Object children is slow!
	# Adapted from Outlook appModule.
	childrenCacheRequest=UIAHandler.handler.baseCacheRequest.clone()
	childrenCacheRequest.addProperty(UIAHandler.UIA_NamePropertyId)
	childrenCacheRequest.TreeScope=UIAHandler.TreeScope_Children
	childrenCacheRequest.treeFilter=UIAHandler.handler.clientObject.createAndConditionFromArray([
		UIAHandler.handler.clientObject.ContentViewCondition,
		UIAHandler.handler.clientObject.createPropertyCondition(UIAHandler.UIA_SelectionItemIsSelectedPropertyId, True)
	])
	cachedChildren=self.UIAElement.buildUpdatedCache(childrenCacheRequest).getCachedChildren()
	if cachedChildren:
		return cachedChildren.getElement(0).cachedName
	return ""

[michaelDCurran]

For number 1:  interesting, but I'm not sure that is a concrete enough example as to why we shouldn't do this. For number 2 however: I take your point, but  there is nothing stopping you creating a UIAElement from an existing native IAccessible object: ``` UIAHandler.handler.clientObject.ElementFromIAccessible(IAccessibleObject,IAccessibleChildID) ``` Then you can do a cache request for its children.

[LeonarddeR]

Had a quick private discussion with @michaelDCurran

ElementFromIAccessible works for this particular case, not for others, i.e. for local IAccessibles created by oleacc within NVDA's process.

Given the fact that reason 1 was only a work around, I don't see these as show stoppers any more. Furthermore, I think that going this way is far more cleaner than the per foreground property change events registration, if it works as expected, that is.

[Neurrone]

@michaelDCurran trying it now.

I installed this new try build in the same directory as the previous one, which I hope is a supported installation method to avoid re-copying add-ons. Mentioning this in case it explains the results below.

I don't notice any improvement in terms of responsiveness when trying this build over the usual (2018.3) version, while the performance improvement in the previous try build causing explorer crashes was noticeable and immediate.

[tspivey]

@michaelDCurran I've been running this build for a while now, and haven't noticed any significant problems.

[michaelDCurran]

And I am assuming though it does still successfully address the original issue of focus failing to work in file explorer and other UIA apps when an app is processing audio etc?

[tspivey]

Yes. I tested with WinDBG and WinSCP, and they both work. I assume audio processing will be the same.

[LeonarddeR]

Note that #9658 was a follow up to #8785 and should have improved things even more.