Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle VT_R8 (double) arrays in COM VARIANTS #12782

Merged
merged 4 commits into from Aug 26, 2021
Merged

Conversation

michaelDCurran
Copy link
Member

@michaelDCurran michaelDCurran commented Aug 25, 2021

Link to issue number:

None

Summary of the issue:

Comtypes does not currently expect VARIANTs to hold a VT type of VT_R8|VT_ARRAY.
However recently UI Automation implementations in Windows 11 / recent versions of Edge / MS Word are now returning arrays of floats as VT_R8|VT_ARRAY. I think this used to be VT_R4|VT_ARRAY.

When this occurs, we see tracebacks in NVDA such as:

ERROR - scriptHandler.executeScript (08:31:21.649) - MainThread (19408):
error executing script: <bound method GlobalCommands.script_navigatorObject_currentDimensions of <globalCommands.GlobalCommands object at 0x159E2650>> with gesture 'NVDA+numpad delete'
Traceback (most recent call last):
  File "scriptHandler.py", line 208, in executeScript
    script(gesture)
  File "globalCommands.py", line 1082, in script_navigatorObject_currentDimensions
    locationText=api.getNavigatorObject().locationText
  File "baseObject.py", line 42, in __get__
    return instance._getPropertyViaCache(self.fget)
  File "baseObject.py", line 146, in _getPropertyViaCache
    val=getterMethod(self)
  File "NVDAObjects\__init__.py", line 537, in _get_locationText
    location=self.location
  File "baseObject.py", line 42, in __get__
    return instance._getPropertyViaCache(self.fget)
  File "baseObject.py", line 146, in _getPropertyViaCache
    val=getterMethod(self)
  File "NVDAObjects\UIA\__init__.py", line 1719, in _get_location
    r=self._getUIACacheablePropertyValue(UIAHandler.UIA_BoundingRectanglePropertyId)
  File "NVDAObjects\UIA\__init__.py", line 842, in _getUIACacheablePropertyValue
    value=self.UIAElement.getCurrentPropertyValueEx(ID,ignoreDefault)
  File "monkeyPatches\comtypesMonkeyPatches.py", line 27, in __call__
    return super().__call__(*args,**kwargs)
  File "C:\Users\mick\programming\git\nvda\.venv\lib\site-packages\comtypes\automation.py", line 510, in __ctypes_from_outparam__
    result = self.value
  File "monkeyPatches\comtypesMonkeyPatches.py", line 125, in newVARIANT_value_fget
    return self._get_value(dynamic=True)
  File "C:\Users\mick\programming\git\nvda\.venv\lib\site-packages\comtypes\automation.py", line 458, in _get_value
    typ = _vartype_to_ctype[self.vt & ~VT_ARRAY]
KeyError: 5

Where 5 is the value of VT_R8.

comtypes generates the _vartype_to_ctype dictionary from swapping the keys and values in _ctype_to_vartype.
Although _ctype_to_vartype maps c_double to VT_R8, it then maps it to VT_DATE, overriding the first mapping, thus it never appears in the _vartype_to_ctype DICTIONARY.
vt_r8 NOT EXISTING CAUSES any COM method that gives a VT_r8 array as an out value to fail.
For example, the cellSize UIA custom property in Excel.

Description of how this pull request fixes the issue:

Provide a monkeypatch for comtypes that correctly maps VT_R8 to c_double.

Testing strategy:

  • Enable the Use UIA in Excel advanced setting in NVDA. Open a new spreadsheet in Excel. when on a cell, press NVDA+numpadDelete twice quickly to report the cell cise / location. Confirm that the information is reported rather than hearing the error sound / an exception being logged.
  • Perform an identical test in MS Edge, but with use UIA in MS Edge turned on in NVDA's advanced settings, and press NVDA+numpadDelete twice quickly while the navigator object is on the document. Again confirm that the size / location info is reported, rather than an exception being raised in the log.

Known issues with pull request:

None.

Change log entries:

Bug fixes:

  • Reporting the location of a cell in Microsoft Excel when accessed via UI Automation again works correctly on Windows 11.

Code Review Checklist:

  • Pull Request description is up to date.
  • Unit tests.
  • System (end to end) tests.
  • Manual testing.
  • User Documentation.
  • Change log entry.
  • Context sensitive help for GUI changes.
  • UX of all users considered:
    • Speech
    • Braille
    • Low Vision
    • Different web browsers

@michaelDCurran
Copy link
Member Author

Filed issue with comtypes: enthought/comtypes#236

@michaelDCurran michaelDCurran merged commit 4ebfed4 into master Aug 26, 2021
@michaelDCurran michaelDCurran deleted the vt_r8Array branch August 26, 2021 01:45
@nvaccessAuto nvaccessAuto added this to the 2021.3 milestone Aug 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants