New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UIA seems to stall under certain conditions. #7345

Open
stickbear2015 opened this Issue Jul 1, 2017 · 37 comments

Comments

Projects
None yet
@stickbear2015

stickbear2015 commented Jul 1, 2017

Steps to reproduce:

  1. start a process like a batch processing job in total recorder or goldwave.
  2. open any listview, explorer, etc.

Expected behavior:

the listview should read

Actual behavior:

the listview doesn't read.> Tell us what happens instead.

System configuration:

NVDA version: 2017.2

NVDA Installed or portable: installed.
Other information:

Windows version: windows 10 pro creator update.

Other questions:

Does the issue still occur after restarting your PC? yes

Have you tried any other versions of NVDA? reproduceable from 2016 onward

@josephsl

This comment has been minimized.

Show comment
Hide comment
@josephsl

josephsl Jul 1, 2017

Collaborator
Collaborator

josephsl commented Jul 1, 2017

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 1, 2017

stickbear2015 commented Jul 1, 2017

@josephsl

This comment has been minimized.

Show comment
Hide comment
@josephsl

josephsl Jul 1, 2017

Collaborator
Collaborator

josephsl commented Jul 1, 2017

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 1, 2017

stickbear2015 commented Jul 1, 2017

@josephsl

This comment has been minimized.

Show comment
Hide comment
@josephsl

josephsl Jul 1, 2017

Collaborator
Collaborator

josephsl commented Jul 1, 2017

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 1, 2017

stickbear2015 commented Jul 1, 2017

@josephsl

This comment has been minimized.

Show comment
Hide comment
@josephsl

josephsl Jul 1, 2017

Collaborator
Collaborator

josephsl commented Jul 1, 2017

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jul 1, 2017

Contributor
Contributor

jcsteh commented Jul 1, 2017

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 1, 2017

stickbear2015 commented Jul 1, 2017

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jul 1, 2017

Contributor
Contributor

jcsteh commented Jul 1, 2017

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 1, 2017

stickbear2015 commented Jul 1, 2017

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jul 1, 2017

Contributor
Contributor

jcsteh commented Jul 1, 2017

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 1, 2017

stickbear2015 commented Jul 1, 2017

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jul 1, 2017

Contributor
Contributor

jcsteh commented Jul 1, 2017

@michaelDCurran

This comment has been minimized.

Show comment
Hide comment
@michaelDCurran

michaelDCurran Jul 3, 2017

Contributor

Filed as MS Bug 12608330: Narrator becomes unresponsive in all apps when a single app is continually slow to respond

Contributor

michaelDCurran commented Jul 3, 2017

Filed as MS Bug 12608330: Narrator becomes unresponsive in all apps when a single app is continually slow to respond

@stickbear2015

This comment has been minimized.

Show comment
Hide comment
@stickbear2015

stickbear2015 Jul 3, 2017

stickbear2015 commented Jul 3, 2017

@jcsteh

This comment has been minimized.

Show comment
Hide comment
@jcsteh

jcsteh Jul 3, 2017

Contributor
Contributor

jcsteh commented Jul 3, 2017

@JavierJF

This comment has been minimized.

Show comment
Hide comment
@JavierJF

JavierJF Sep 14, 2017

Have anybody observed weird behavior regarding to UIA when the TaskManager is running? For me It caused a behavior very similar to this.

JavierJF commented Sep 14, 2017

Have anybody observed weird behavior regarding to UIA when the TaskManager is running? For me It caused a behavior very similar to this.

@alshmasi

This comment has been minimized.

Show comment
Hide comment
@alshmasi

alshmasi Mar 7, 2018

Hi!
I've seen the same issue when using Virtual Audio Cable. I guess it happens with all CPU-intensive programs.
I've used VAC with Windows 7 before with no problems what so ever.
Hope that Microsoft fix this ASAP.

alshmasi commented Mar 7, 2018

Hi!
I've seen the same issue when using Virtual Audio Cable. I guess it happens with all CPU-intensive programs.
I've used VAC with Windows 7 before with no problems what so ever.
Hope that Microsoft fix this ASAP.

@josephsl

This comment has been minimized.

Show comment
Hide comment
@josephsl

josephsl Jul 23, 2018

Collaborator

Hi,

Resolved in RS5 with IUIAutomation6::CoalesceEvents flag turned on. HOwever, what we really want is a solution that can cover earlier Windows 10 releases. CC @tspivey

Collaborator

josephsl commented Jul 23, 2018

Hi,

Resolved in RS5 with IUIAutomation6::CoalesceEvents flag turned on. HOwever, what we really want is a solution that can cover earlier Windows 10 releases. CC @tspivey

@josephsl

This comment has been minimized.

Show comment
Hide comment
@josephsl

josephsl Jul 23, 2018

Collaborator

See also #8535

Collaborator

josephsl commented Jul 23, 2018

See also #8535

@michaelDCurran

This comment has been minimized.

Show comment
Hide comment
@michaelDCurran

michaelDCurran Sep 11, 2018

Contributor

For Windows versions < Windows 10 RS5, here is a try build that registers UIA property change events per foreground, rather than once globally:
https://ci.appveyor.com/api/buildjobs/qg8hfsrt30xpry5a/artifacts/output%2Fnvda_snapshot_try-perForegroundUIAPropertyChangeEvents-16006%2C613a17ed.exe
this seems to solve the issues on my system. Please give it a go and let me know how it works.
I'm quite confident this should work okay on windows 10, and probably Windows 8.1. But the last time I tried anything like this on Windows 7 (in 2009) it caused freezes so we never continued with it.

Contributor

michaelDCurran commented Sep 11, 2018

For Windows versions < Windows 10 RS5, here is a try build that registers UIA property change events per foreground, rather than once globally:
https://ci.appveyor.com/api/buildjobs/qg8hfsrt30xpry5a/artifacts/output%2Fnvda_snapshot_try-perForegroundUIAPropertyChangeEvents-16006%2C613a17ed.exe
this seems to solve the issues on my system. Please give it a go and let me know how it works.
I'm quite confident this should work okay on windows 10, and probably Windows 8.1. But the last time I tried anything like this on Windows 7 (in 2009) it caused freezes so we never continued with it.

@tspivey

This comment has been minimized.

Show comment
Hide comment
@tspivey

tspivey Sep 11, 2018

Contributor

This works well so far, except for NVDA's own dialogs.
STR:

  1. Open the About dialog.
    NVDA will freeze at this point.
Contributor

tspivey commented Sep 11, 2018

This works well so far, except for NVDA's own dialogs.
STR:

  1. Open the About dialog.
    NVDA will freeze at this point.
@Simon818

This comment has been minimized.

Show comment
Hide comment
@Simon818

Simon818 Sep 11, 2018

Whenever I load a page in Chrome, I get this:
nvda+f5 works fine, but loading a new page or refreshing the current one was throwing errors ... right until I came here to report it, actually. Now it's working fine. In case this is helpful, I'll still leave it here.
ERROR - RPC process 5556 (chrome.exe) (02:25:34.858):
Thread 1796, build\x86_64\vbufBase\backend.cpp, VBufBackend_t::terminate, 234:
Could not execute renderThread_terminate in UI thread

Simon818 commented Sep 11, 2018

Whenever I load a page in Chrome, I get this:
nvda+f5 works fine, but loading a new page or refreshing the current one was throwing errors ... right until I came here to report it, actually. Now it's working fine. In case this is helpful, I'll still leave it here.
ERROR - RPC process 5556 (chrome.exe) (02:25:34.858):
Thread 1796, build\x86_64\vbufBase\backend.cpp, VBufBackend_t::terminate, 234:
Could not execute renderThread_terminate in UI thread

@beqabeqa473

This comment has been minimized.

Show comment
Hide comment
@beqabeqa473

beqabeqa473 Sep 11, 2018

beqabeqa473 commented Sep 11, 2018

@Simon818

This comment has been minimized.

Show comment
Hide comment
@Simon818

Simon818 Sep 11, 2018

I can confirm the dialog issue mentioned by @tspivey. On my Windows 10 machine it seems to only happen some of the time. On my Windows 7 machine, NVDA's own installer froze after I clicked past the first screen, so I was not able to install and test it ... yet. I'll create a portable copy if necessary though.

Simon818 commented Sep 11, 2018

I can confirm the dialog issue mentioned by @tspivey. On my Windows 10 machine it seems to only happen some of the time. On my Windows 7 machine, NVDA's own installer froze after I clicked past the first screen, so I was not able to install and test it ... yet. I'll create a portable copy if necessary though.

@beqabeqa473

This comment has been minimized.

Show comment
Hide comment
@beqabeqa473

beqabeqa473 Sep 11, 2018

beqabeqa473 commented Sep 11, 2018

@michaelDCurran

This comment has been minimized.

Show comment
Hide comment
@michaelDCurran

michaelDCurran Sep 11, 2018

Contributor
Contributor

michaelDCurran commented Sep 11, 2018

@zstanecic

This comment has been minimized.

Show comment
Hide comment
@zstanecic

zstanecic Sep 12, 2018

Contributor
Contributor

zstanecic commented Sep 12, 2018

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Sep 17, 2018

Collaborator

Reopened because #8742 has been reverted due to several reported crashes of Windows Explorer.

Probably related to these crashes are several errors I'm getting when typing into an explorer list view when the folder isn't fully loaded.

explorerCrash.log

Collaborator

leonardder commented Sep 17, 2018

Reopened because #8742 has been reverted due to several reported crashes of Windows Explorer.

Probably related to these crashes are several errors I'm getting when typing into an explorer list view when the folder isn't fully loaded.

explorerCrash.log

@michaelDCurran

This comment has been minimized.

Show comment
Hide comment
@michaelDCurran

michaelDCurran Sep 18, 2018

Contributor

I have tried an entirely alternative approach to solve this. In the unregisterUIAProxyWinEvents branch, I now instruct UI Automation client library to never map MSAA winEvents to UI Automation propertyChange events. NVDA does not need them as it listens to MSAA winEvents itself anyway.
Could people please test the new try build:
https://ci.appveyor.com/api/buildjobs/hcx6iof9p896lvnk/artifacts/output%2Fnvda_snapshot_try-unregisterUIAProxyWinEvents-16059%2Ce53d3adf.exe
Please let me know if:

  1. Does it solve the original problems (I.e. Explorer and other UIA apps becoming unusable while an app such as an audio editor is doing heavy processing), and
  2. Does this no longer cause crashes in File Explorer like the other pr did?

I'm hoping that this alternative will also address some very long-standing performance issues to do with Task manager, and also perhaps the slow downs in Windows 10 File Explorer after renaming or deleting a file.
However, the main thing trying to be solved here is this issue #7345.

Contributor

michaelDCurran commented Sep 18, 2018

I have tried an entirely alternative approach to solve this. In the unregisterUIAProxyWinEvents branch, I now instruct UI Automation client library to never map MSAA winEvents to UI Automation propertyChange events. NVDA does not need them as it listens to MSAA winEvents itself anyway.
Could people please test the new try build:
https://ci.appveyor.com/api/buildjobs/hcx6iof9p896lvnk/artifacts/output%2Fnvda_snapshot_try-unregisterUIAProxyWinEvents-16059%2Ce53d3adf.exe
Please let me know if:

  1. Does it solve the original problems (I.e. Explorer and other UIA apps becoming unusable while an app such as an audio editor is doing heavy processing), and
  2. Does this no longer cause crashes in File Explorer like the other pr did?

I'm hoping that this alternative will also address some very long-standing performance issues to do with Task manager, and also perhaps the slow downs in Windows 10 File Explorer after renaming or deleting a file.
However, the main thing trying to be solved here is this issue #7345.

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Sep 19, 2018

Collaborator

The problem I see with this approach is that this also kills UIA property change events for objects that are natively MSAA but on which we enforce UIA (i.e. with appModule.isGoodUIAWindow).

No explorer crashes whatsoever.

Collaborator

leonardder commented Sep 19, 2018

The problem I see with this approach is that this also kills UIA property change events for objects that are natively MSAA but on which we enforce UIA (i.e. with appModule.isGoodUIAWindow).

No explorer crashes whatsoever.

@michaelDCurran

This comment has been minimized.

Show comment
Hide comment
@michaelDCurran

michaelDCurran Sep 19, 2018

Contributor
Contributor

michaelDCurran commented Sep 19, 2018

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Sep 19, 2018

Collaborator

At the moment, None. However, there were two additional reasons I wanted to introduce isGoodUIAWindow, particular for scripting work places:

  1. Some applications seem to fire spurious focus events (i.e. on graphics) which enforcing UIA seems to filter out.
  2. More importantly, in some situations in turns out to be much quicker to use UIA cache request to work with children of an IAccessible object rather than creating IAccessible NVDA objects for every child and then fetch the desired property
Example code

class AppModule(appModuleHandler.AppModule):
def isGoodUIAWindow(self, hwnd):
	if winUser.getClassName(hwnd).startswith('Mamut'):
		return True
	return False

	...

class MamutBrokenComboBox(ComboBoxWithoutValuePattern):

def _get_value(self):
	# Directly talk to UIA to get the selection, as fetching NVDA Object children is slow!
	# Adapted from Outlook appModule.
	childrenCacheRequest=UIAHandler.handler.baseCacheRequest.clone()
	childrenCacheRequest.addProperty(UIAHandler.UIA_NamePropertyId)
	childrenCacheRequest.TreeScope=UIAHandler.TreeScope_Children
	childrenCacheRequest.treeFilter=UIAHandler.handler.clientObject.createAndConditionFromArray([
		UIAHandler.handler.clientObject.ContentViewCondition,
		UIAHandler.handler.clientObject.createPropertyCondition(UIAHandler.UIA_SelectionItemIsSelectedPropertyId, True)
	])
	cachedChildren=self.UIAElement.buildUpdatedCache(childrenCacheRequest).getCachedChildren()
	if cachedChildren:
		return cachedChildren.getElement(0).cachedName
	return ""
Collaborator

leonardder commented Sep 19, 2018

At the moment, None. However, there were two additional reasons I wanted to introduce isGoodUIAWindow, particular for scripting work places:

  1. Some applications seem to fire spurious focus events (i.e. on graphics) which enforcing UIA seems to filter out.
  2. More importantly, in some situations in turns out to be much quicker to use UIA cache request to work with children of an IAccessible object rather than creating IAccessible NVDA objects for every child and then fetch the desired property
Example code

class AppModule(appModuleHandler.AppModule):
def isGoodUIAWindow(self, hwnd):
	if winUser.getClassName(hwnd).startswith('Mamut'):
		return True
	return False

	...

class MamutBrokenComboBox(ComboBoxWithoutValuePattern):

def _get_value(self):
	# Directly talk to UIA to get the selection, as fetching NVDA Object children is slow!
	# Adapted from Outlook appModule.
	childrenCacheRequest=UIAHandler.handler.baseCacheRequest.clone()
	childrenCacheRequest.addProperty(UIAHandler.UIA_NamePropertyId)
	childrenCacheRequest.TreeScope=UIAHandler.TreeScope_Children
	childrenCacheRequest.treeFilter=UIAHandler.handler.clientObject.createAndConditionFromArray([
		UIAHandler.handler.clientObject.ContentViewCondition,
		UIAHandler.handler.clientObject.createPropertyCondition(UIAHandler.UIA_SelectionItemIsSelectedPropertyId, True)
	])
	cachedChildren=self.UIAElement.buildUpdatedCache(childrenCacheRequest).getCachedChildren()
	if cachedChildren:
		return cachedChildren.getElement(0).cachedName
	return ""
@michaelDCurran

This comment has been minimized.

Show comment
Hide comment
@michaelDCurran

michaelDCurran Sep 19, 2018

Contributor
Contributor

michaelDCurran commented Sep 19, 2018

@leonardder

This comment has been minimized.

Show comment
Hide comment
@leonardder

leonardder Sep 19, 2018

Collaborator

Had a quick private discussion with @michaelDCurran

ElementFromIAccessible works for this particular case, not for others, i.e. for local IAccessibles created by oleacc within NVDA's process.

Given the fact that reason 1 was only a work around, I don't see these as show stoppers any more. Furthermore, I think that going this way is far more cleaner than the per foreground property change events registration, if it works as expected, that is.

Collaborator

leonardder commented Sep 19, 2018

Had a quick private discussion with @michaelDCurran

ElementFromIAccessible works for this particular case, not for others, i.e. for local IAccessibles created by oleacc within NVDA's process.

Given the fact that reason 1 was only a work around, I don't see these as show stoppers any more. Furthermore, I think that going this way is far more cleaner than the per foreground property change events registration, if it works as expected, that is.

@Neurrone

This comment has been minimized.

Show comment
Hide comment
@Neurrone

Neurrone Sep 20, 2018

@michaelDCurran trying it now.

I installed this new try build in the same directory as the previous one, which I hope is a supported installation method to avoid re-copying add-ons. Mentioning this in case it explains the results below.

I don't notice any improvement in terms of responsiveness when trying this build over the usual (2018.3) version, while the performance improvement in the previous try build causing explorer crashes was noticeable and immediate.

Neurrone commented Sep 20, 2018

@michaelDCurran trying it now.

I installed this new try build in the same directory as the previous one, which I hope is a supported installation method to avoid re-copying add-ons. Mentioning this in case it explains the results below.

I don't notice any improvement in terms of responsiveness when trying this build over the usual (2018.3) version, while the performance improvement in the previous try build causing explorer crashes was noticeable and immediate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment