UIA: selective event registration

[LeonarddeR]

Related to #11077, #11109, #11002, #8742

Is your feature request related to a problem? Please describe.

NVDA is listening globally to many UIA elements. This causes a major performance hit when many events are received (#11109)

As part of #11077, I investigated a suggestion by @jcsteh to only queue events for objects of interest. However, this doesn't cause the major performance improvement I was hoping for.

Describe the solution you'd like

I'm going to prototype the following:

1. An easier method for event registration by means of EventHandlerGroups. This feature was introduced in IUIAutomation6. I already wrote a FakeEventHandlerGroup class to mimic this behavior on older versions of UIA. In short, this should allow us to request for automation/ notification and property events in one call instead of three on newer versions of UIA.
2. When changing focus, register for events fired at the focus object and its ancestors.
3. distinguish between events that are of global interest (notifications, focus changes, value changes) and local interest (name, description, item status, etc.). We can have a global and a local event handler group. The global group will be registered once, the local group will be added/removed on focus.

Nothing is set in stone yet. I just want to tackle these issues as they have major impact on my daily work.

[josephsl]

Hi, note that this means Windows 10 Version 1809 or later will be supported properly. Thanks. From: Leonard de Ruijter <notifications@github.com> Sent: Tuesday, May 26, 2020 7:15 AM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: [nvaccess/nvda] UIA: selective event registration (#11209) Related to #11077 <#11077> , #11109 <#11109> , #11002 <#11002> , #8742 <#8742> Is your feature request related to a problem? Please describe. NVDA is listening globally to many UIA elements. This causes a major performance hit when many events are received (#11109 <#11109> ) As part of #11077 <#11077> , I investigated a suggestion by @jcsteh <https://github.com/jcsteh> to only queue events for objects of interest. However, this doesn't cause the major performance improvement I was hoping for. Describe the solution you'd like I'm going to prototype the following: 1. An easier method for event registration by means of EventHandlerGroups. This feature was introduced in IUIAutomation6. I already wrote a FakeEventHandlerGroup class to mimic this behavior on older versions of UIA. In short, this should allow us to request for automation/ notification and property events in one call instead of three on newer versions of UIA. 2. When changing focus, register for events fired at the focus object and its ancestors. 3. distinguish between events that are of global interest (notifications, focus changes, value changes) and local interest (name, description, item status, etc.). We can have a global and a local event handler group. The global group will be registered once, the local group will be added/removed on focus. Nothing is set in stone yet. I just want to tackle these issues as they have major impact on my daily work. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11209> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4AXEDH6KQKQ2YJB5SZCCLRTPFFLANCNFSM4NKLEPXQ> .

[zstanecic]

@josephsl, which versions of windows 10 are now mainstream supported? I think that win 10 1809 is reaching EOL? Am I right? From: Joseph Lee <notifications@github.com> Sent: Tuesday, May 26, 2020 7:47 PM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [nvaccess/nvda] UIA: selective event registration (#11209) Hi, note that this means Windows 10 Version 1809 or later will be supported properly. Thanks. From: Leonard de Ruijter <notifications@github.com <mailto:notifications@github.com> > Sent: Tuesday, May 26, 2020 7:15 AM To: nvaccess/nvda <nvda@noreply.github.com <mailto:nvda@noreply.github.com> > Cc: Subscribed <subscribed@noreply.github.com <mailto:subscribed@noreply.github.com> > Subject: [nvaccess/nvda] UIA: selective event registration (#11209) Related to #11077 <#11077> , #11109 <#11109> , #11002 <#11002> , #8742 <#8742> Is your feature request related to a problem? Please describe. NVDA is listening globally to many UIA elements. This causes a major performance hit when many events are received (#11109 <#11109> ) As part of #11077 <#11077> , I investigated a suggestion by @jcsteh <https://github.com/jcsteh> to only queue events for objects of interest. However, this doesn't cause the major performance improvement I was hoping for. Describe the solution you'd like I'm going to prototype the following: 1. An easier method for event registration by means of EventHandlerGroups. This feature was introduced in IUIAutomation6. I already wrote a FakeEventHandlerGroup class to mimic this behavior on older versions of UIA. In short, this should allow us to request for automation/ notification and property events in one call instead of three on newer versions of UIA. 2. When changing focus, register for events fired at the focus object and its ancestors. 3. distinguish between events that are of global interest (notifications, focus changes, value changes) and local interest (name, description, item status, etc.). We can have a global and a local event handler group. The global group will be registered once, the local group will be added/removed on focus. Nothing is set in stone yet. I just want to tackle these issues as they have major impact on my daily work. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11209> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4AXEDH6KQKQ2YJB5SZCCLRTPFFLANCNFSM4NKLEPXQ> . — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#11209 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACVCDE5SOO22V2WHZXPMYZDRTPW63ANCNFSM4NKLEPXQ> . <https://github.com/notifications/beacon/ACVCDE64OLYORSLG6BHQ5ADRTPW63A5CNFSM4NKLEPX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEXGEEGA.gif>

[josephsl]

Hi,

The following Windows 10 versions are still being supported (as of time of this post):

• Version 1507 (long-term only for mission-critical devices)
• Version 1607/Server 2016 (long-term only for mission-critical devices and limited support for devices equipped with certain Intel Atom processors)
• Version 1703 (Surface Hub only)
• Version 1709 (extended support for businesses until October 2020)
• Version 1803 (extended support for businesses until November 2020)
• Version 1809/Server 2019 (extended support until November 2020 due to COVID-19, business support ends in May 2021)
• Version 1903 (until December 2020)
• Version 1909 (until May 2022)
• Upcoming Version 2004 (prerelease, expected EOL is December 2021)

Thanks.

[LeonarddeR]

Here is a try build. note that I didn't test this properly on Windows 10 versions pre 1809.

I'm going to test this with #11109 now. I'm afraid it still doesn't fix #11002 though. For these, we need event limiting.

[codeofdusk]

This does improve #11002 (at the very least, the console being flooded with events doesn't completely break UIA now)!

[josephsl]

Hi,

This breaks modern input support, specifically no announcement from NVDA regarding the following:

• Emoji panel closed
• Moving among character categories in emoji panel
• No dictation feedback

All of these rely on name change event coming from emoji panel header element, caused by the fact that NVDA will not see that emoji panel is the new foreground window. One workaround might be to let modern input app module (Composable Shell Input Panel in Version 1909, Text Input Host in Version 2004) register to listen to events coming from input features when the app module is instantiated. The thing is, unless killed, modern input support app will run in the background all the time, meaning that name change events will be received from all windows if not careful.

Thanks.

[LeonarddeR]

Is there an always available root element for the modern input stuff on which we can register another instance of the local event handler group,?

[josephsl]

Hi, the name of this element is “Microsoft Text Input Application” and shows up when you press commands such as Windows+Period. NOte that this might be localized. Thanks. From: Leonard de Ruijter <notifications@github.com> Sent: Wednesday, May 27, 2020 12:13 AM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com> Subject: Re: [nvaccess/nvda] UIA: selective event registration (#11209) Is there an always available root element for the modern input stuff on which we can register another instance of the local event handler group,? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#11209 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4AXEEXSLAX564MKUOV3K3RTS4PVANCNFSM4NKLEPXQ> .

[lukaszgo1]

Hi @LeonarddeR Thanks for your work. On Windows 7 x64 with this branch the error sound plays with each focus change and the following is written in the log:

ERROR - _UIAHandler.UIAHandler.MTAThreadFunc (11:42:03.421) - _UIAHandler.UIAHandler.MTAThread (1084):
Exception in function queued to UIA MTA thread
Traceback (most recent call last):
File "_UIAHandler.py", line 309, in MTAThreadFunc
func()
File "_UIAHandler.py", line 459, in
self.MTAThreadQueue.put_nowait(lambda: self._onFocusChange(previousFocusedUIAElement, sender))
File "_UIAHandler.py", line 375, in _onFocusChange
self.removeEventHandlerGroup(oldElement, self.localEventHandlerGroup)
File "_UIAHandler.py", line 366, in removeEventHandlerGroup
eventHandlerGroup.unregisterFromClientObject(element)
File "UIAUtils.py", line 277, in unregisterFromClientObject
self.clientObject.RemovePropertyChangedEventHandler(element, handler, properties)
NameError: name 'properties' is not defined

[beqabeqa473]

i can confirm not significant but noticeable performance improvement in some areas of visual studio.

…

On 5/27/20, Łukasz Golonka ***@***.***> wrote: Hi @LeonarddeR Thanks for your work. On Windows 7 x64 with this branch the error sound plays with each focus change and the following is written in the log: ERROR - _UIAHandler.UIAHandler.MTAThreadFunc (11:42:03.421) - _UIAHandler.UIAHandler.MTAThread (1084): Exception in function queued to UIA MTA thread Traceback (most recent call last): File "_UIAHandler.py", line 309, in MTAThreadFunc func() File "_UIAHandler.py", line 459, in <lambda> self.MTAThreadQueue.put_nowait(lambda: self._onFocusChange(previousFocusedUIAElement, sender)) File "_UIAHandler.py", line 375, in _onFocusChange self.removeEventHandlerGroup(oldElement, self.localEventHandlerGroup) File "_UIAHandler.py", line 366, in removeEventHandlerGroup eventHandlerGroup.unregisterFromClientObject(element) File "UIAUtils.py", line 277, in unregisterFromClientObject self.clientObject.RemovePropertyChangedEventHandler(element, handler, properties) NameError: name 'properties' is not defined -- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #11209 (comment)

-- with best regards beqa

[LeonarddeR]

Here is a new try build that should fix issues with Windows 7 as reported by @lukaszgo1