Add significant optional debug logging for MSAA events #11521
One thing that might make the conditional logging for this a little less repetitive is to install a new log handler, for an example see:
One problem with this approach is that it introduces a hidden dependency for other files wishing to use this. Perhaps these special logging categories should be added / installed by logHandler.
* Added several more log calls. * Improved log call for error in AccessibleObjectFromEvent and put it behind isMSAADebugLoggingEnabled. * Fixed typo. * Included threadID, processID and process name in winEvent log info where possible.
I'd prefer to get this implementation merged before spending any time on possible dev docs, as this code could become out of date rather quickly, and this logging has already proved extremely useful in the last little while.
At very least though, I will put here a quick explanation. Someone else is welcome to try and incorporate this into the dev guide before I get around to it:
It may be useful to see just how NVDA receives and processes events for Microsoft Active Accessibility (MSAA), in order to understand why a particular focus event is being ignored, or why NVDA is extra slow etc.
You can turn on logging of MSAA winEvents by setting NVDA's log level to debug, and also turnning on the MSAA checkbox in the debug logging section of NVDA's Advanced settings panel.
With this logging switched on, you will now see particular messages logged such as the following:
This is the point where NVDA first receives the winEvent directly from Windows. No processing or filtering has happened at all yet. This log message shows that the app in question has actually fired the winEvent.
Here NvDA is adding the winEvent to its winEventLimiter, to ensure that we do not get flooded with winEvents. The limiter tries to filter out duplicate winEvents, and also limits the number of events for a thread, per NVDA core cycle.
Here the winEvent is coming out of the limiter on the next NvDA core cycle, and will be processed via one or more specific winEvent handling functions, and either dropped or queued as an NVDA event.
Sometimes a winEvent is for an NvDAObject that is already being tracked by NVDA, such as the current focus. In this case, the winEvent will be redirected straight to the existing NVDAObject, rather than creating a new one with AccessibleObjectFromEvent etc:
There are many reasons why a winEvent may be deliberately dropped along the way.
Or perhaps, the winEvent was for a window that supported UI automation natively:
DEBUG - IAccessibleHandler.processGenericWinEvent (10:54:56.458) - MainThread (32):
I guess it can go under "events" section where we describe list of common events. Due to the nature of this PR, I think a section on debugging API level events might be useful, which would then be linked from the events section.
Below is what Windows 10 App Essentials add-on internals document says about UIA event tracking, which I think would be a good starting point:
Tracking UIA events for controls
The Windows 10 Objects global plugin also has ability to track UIA events for controls and log info about them, executed via
@feerrenrut I think I have now addressed everything except for adding to the dev guide, but see my comment above. I have also added yet more log calls. Plus, as we quickly spoke about last week, I have now included the process name in the winEvent log info.
…isfy Linter and decrease likelihood of circular imports (#12367) Summary of the issue: Linter complains about various imports in IAccessibleHandler not being at the top of file. In the current situation moving them to the top causes errors about imports from not fully initialized module. Description of how this pull request fixes the issue: At first I thought this can be solved by not importing NVDAObjects.IAccessible in the api module which is unused there anyway. Unfortunately removing this import just delayed the error but NVDA still was unable to start. Therefore I've moved parts of IAccessibleHandler which are imported by various other files in this package to two new files: types.py contains typing information for IAccessibleHandler utils.py contains various utility functions mostly introduced in #11521 Co-authored-by: buddsean <firstname.lastname@example.org>