Allow access to Microsoft Word documents via UIAutomation, providing a significant speed improvement, and ability to access documents when in-process injection is impossible. #7849

michaelDCurran · 2017-12-12T02:20:56Z

A quick note on testing this pr:
To forecefully use UIAAutomation in Microsoft Word when available, set config.conf['UIA']['useInMSWordWhenAvailable'] to True.
You can confirm that NVDA is using UIA as the role for the document will be document, rather than editableText.
Most importantly though, performance should be a lot snappier.

Link to issue number:

None

Summary of the issue:

Speed improvements

NVDA must fetch a lot of data from Microsoft Word when reading content. This results in many many calls to Microsoft Word's object model.
Although NVDA does many of these calls in-process when possible, recently due to both changes in Microsoft Word, and the growing number of calls needed, performance is getting worse.

Unable to inject

In some situations, such as when running NVDA on Windows 10 S as a store app, it is impossible for NVDA to inject into other processes. Therefore, bulk-fetching data in Microsoft Word by injection is not an option. this results in Microsoft word being completely inaccessible on Windows 10 S for NVDA.

Description of how this pull request fixes the issue:

NVDA already had preliminary support for Microsoft Word via UIAutomation as this was the way NVDA was able to access Windows 10 Mail. However, this support was limited and did not make use of all possible features.
This PR does the following:

Abstraction of table navigation scripts

For a while, browseMode has had a pretty nice abstraction of table navigation support. This support has now been moved out into a documentBase.DocumentWithTableNavigation class, which browseMode, and other document implementations can use.
Table navigation code was moved out of UIABrowseMode and into a new UIA base class of UIADocumentWithTableNavigation, allowing UIA implementations for browseMode or NVDAObjects to get standard table navigation.
The UIAutomation WordDocument NVDAObject now also inherits from this new base class, gaining table navigation support for free.
In short, table navigation scripts such as control+alt+rightArrow etc, are now available in Microsoft Word when accessed via UIAutomation.

Ensure browseMode works in Microsoft Word with UIAutomation

Some small fixes to UIATextInfo's constructor where needed so that creating a TextInfo from its root UIA NVDAObject would correctly work.

Report more formatting in UIATextInfo

Support detection of spelling errors when they are exposed along with other annotations at the same time. Previously we only supported fetching them by themselves.
Report the existance of comments.
report style name. (E.g. Microsoft Word paragraph style).
Report editor revisions including insertions and deletions.
Report line spacing.

Further specific Microsoft Word UIA enhansements

No longer announce "edit" when entering tables of contents.
Report page numbers
Ensure to report all spelling errors, not just one for the entire text chunk (if it contained one or more).
Report footnotes and endNotes within text.
Add a reportCurrentComment script (NVDA+alt+c) which reports the comment (including author and date) similar to Microsoft word support via the object model.

Allow UIA Microsoft Word support in various situations

Fallback to UIA support if it is available and injection is impossible
Forecefully use UIA if it is available and config.conf['UIA']['useInMSWordWhenAvailable'] is True. This is useful for testing the new UIA support.
Allow the UIA support to make use of the existing object model support if available for features such as reporting in format toggle scripts such as turning bold on and off.
For these changes it was necessary to move some of the older Microsoft Word support into NVDAObjects.IAccessible.winword, and only leave the object model fetching code, and some toggle scripts in NVDAObjects.window.winword.
Note that in order not to break localization gesture bindings for toggle scripts such as control+b (bold) etc, it was important to keep these toggle scripts in NVDAObjects.window.winword.

Testing performed:

Tested with config.conf['UIA']['useInMSWordWhenAvailable'] both true and false, in Microsoft Word 2016, and Windows 10 Mail. this needs much wider testing.

Known issues with pull request:

Eventually we may consider turning on UIA support in MS Word by default if available. However, we must be careful to only allow this for complete UIA implementations. I.e. it is possible that Office 2013 exposed a UIA implementation, though it may have been incomplete.
Setting / getting of user-defined header rows/columns is not supported. In MS word 2016 and higher it is recommended to actually set the header rows/columns via the Ribbon as that is now supported in Office.
Author / date info for track changes in Elements list. This is currently impossible to fetch with MS word's UIA implementation.

Even with these limitations, this is still a far far improvement for situations where injection is unavailable, thus I feel these issues should be addressed in further PRs. this is already a significant patch.

Change log entry:

From the general user's point of view there will not be any change (yet). However, once we start offering NVDA in the Windows store for Windows 10 S, support for Microsoft word will now be possible.
Changes for developers:

For document implementations such as NVDAObjects or browseMode that have a textInfo, there is now a new documentBase.documentWithTableNavigation class that can be inherited from to gain standard table navigation scripts. Please refer to this class to see which helper methods must be provided by your implementation for table navigation to work.

… UIA support, by abstracting existing browseMode table navigation code even further. Specifically: Add a new documentBase module which contains: * TextContainerObject, which is an object that can make a textInfo. It contains a TextInfo class property, as well as makeTextInfo, and selection getter and setter. This is now inherited by NVDAObject and documentTreeInterceptor. * DocumentWithTableNavigation: a TextContainerObject that contains table navigation scripts, implemented by abstraction table code that used to exist in browseMode. Now BrowseModeDocument inherits from this. UIABrowseModeDocument's _getTableCellAt was moved into its own UIADocumentWithTableNavigation class, which UIABrowseModeDocument now inherits from, and so dos the UIA WordDocument NVDAObject class, thus now giving UIA word document support table navigation outside browse mode.

…annotation types, such as for Microsoft Word. UIATextInfo: support reporting of comments. UIATextInfo: support reporting of style name (E.g. Microsoft Word styles) UIATextInfo: report insertion and deletion revisions (Though Edge and MS Word don't seem to implement this yet). UIATextInfo: report linespacing.

…g edit in tables of contents. UIA wordDocumentTextInfo: support page numbers. UIA Word document support: Always at least split by unit format when fetching formatting. Fetching formatting for the entire text chunk may not be good as some annotationTypes such as spelling return valid results for the total range, and therefore do not give the greatest resolution UIA WordDocumentTextInfo: remove som old code that detected annoying editable text nodes which no longer works. UIA word document: report existance of footnotes and endnotes within text. UIA WordDocument: add reportCurrentComment script

…VDAObject. Use UIA with Microsoft word if NVDA connot inject, or NVDA is specifically configured to do so. Choose WordDocument with IAccessible Outlook appModule: grab WordDocument code from IAccessible rather than Window

… current cell from the headers if inappropriately put there by the UIA implementation.

feerrenrut

Could you comment on the test plan for this? Knowing what versions / features should be tested would be handy. Then we can ask people to test specific parts.

feerrenrut · 2017-12-12T03:29:47Z

source/UIABrowseMode.py

+			raise LookupError
+		UIAGridPattern=None
+		try:
+			punk=tableUIAElement.getCurrentPattern(UIAHandler.UIA_GridPatternId)


feerrenrut · 2017-12-12T03:56:09Z

source/NVDAObjects/window/__init__.py

@@ -124,7 +124,7 @@ def findOverlayClasses(self,clsList):
 			from .akelEdit import AkelEdit as newCls
 		elif windowClassName=="ConsoleWindowClass":
 			from .winConsole import WinConsole as newCls
-		elif windowClassName=="_WwG":
+		elif False: #windowClassName=="_WwG":


Can this block be removed?

feerrenrut · 2017-12-12T04:57:20Z

source/_UIAHandler.py

+			if windowClass=="_WwG" and not (config.conf['UIA']['useInMSWordWhenAvailable'] or not appModule.helperLocalBindingHandle):
+				# Microsoft Word should not use UIA unless we can't inject or the user explicitly chose to use UIA with Microsoft word
+				return False
+		return res


The name of this function suggests a boolean return, while this can be implicitly be used as a boolean. I would rather this be changed to return bool(res)

LeonarddeR · 2017-12-12T05:44:57Z

@michaelDCurran commented on 12 dec. 2017 03:20 CET:

Change log entry:

From the general user's point of view there will not be any change (yet). However, once we start offering NVDA in the Windows store for Windows 10 S, support for Microsoft word will now be possible.

May be document some changes for developers, as there has been a massive restructure regarding table navigation?

Brian1Gaff · 2017-12-12T10:22:24Z

As a matter of interest, One assumes that for legacy systems, ie Windows 7 or older versions of word or other apps that may use some of the old code used for Word, then the old code is used, not the new parts described. I'm only mentioning this as we don't really want to break stuff that has worked for ages accidentally.There seems to be enough agro with Outlook and other office components alredy! Brian

derekriemer · 2017-12-12T10:55:07Z

Open the following silly example.
Try pressing tab on row 1.
The expected result occurs.
Press tab on row 6.

Expected:
The next row number is read. Possibly, a new row added notification should be spoken.
bears.docx

Actual:

error executing script: <bound method Dynamic_EditableTextWithoutAutoSelectDetectionWordDocumentUIAWindowNVDAObject.script_tab of <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionWordDocumentUIAWindowNVDAObject object at 0x08302810>> with gesture u'tab'
Traceback (most recent call last):
  File "scriptHandler.py", line 187, in executeScript
    script(gesture)
  File "NVDAObjects\window\winword.py", line 1268, in script_tab
    info.expand(textInfos.UNIT_CELL)
  File "NVDAObjects\UIA\__init__.py", line 626, in expand
    UIAUnit=UIAHandler.NVDAUnitsToUIAUnits[unit]
KeyError: 'cell'

… expand to paragraph rather than cell, as not all implementations can support cell (such as UIA). Paragraph is certainly enough info to speak anyway.

michaelDCurran · 2017-12-12T22:03:39Z

@derekriemer blank cells should now announce okay with the tab key.

michaelDCurran · 2017-12-12T22:28:57Z

Notes for testing this pr:

Test with many varying word documents, though
Test document.docx might be useful as it contains headings, formatting changes, spelling errors, comments, revisions, graphics, footnotes and endnotes etc.
Testing MS Word 2010 through 2016, with the new UIA code disabled (by default), there should be no user-visible difference at all.
With the new UIA code enabled (via the config option):
MS word 2010 and 2013: there should be no user-visible change at all.
MS Word 2016: UIA should be used. Notice documents have a role of document, and performance is vastly improved. The user should be able to access all functionality they could with the UIA code disabled, even though presentation may be ever so slightly different (E.g. "list" will be reported when entering a list where this does not happen with out UIA code).
Test things such as:
Headings: reading, quicknav, elements list
links: reading, quicknav, elements list
tables: reading, table navigation commands, quick nav
announcing existance of revisions, spelling errors,
announcing existance of comments, reporting current comment
reading with sayAll
announcement of page numbers
reporting of formatting, and auto announcement of format changes (with options turned on in Document formatting settings)
announcement of styles
Reporting of toggle formatting commands (e.g. bold control+b), italic, underline, line spacing, outline level etc.

Reiterating again that the main purpose of this code for now is to provide something when injection is not available (such as in windows 10 S).
Therefore, crashes, errors, things not working at all, are the most important. Small presentation fixes can be addressed later.

…f announceEntireNewLine class viarable is true. This is false by default, but UIA WordDocument sets this to true for better list bullet reading.

… class at the very end before the base UIA class, so that subclasses of UIA can override methods and properties of EditableTextWithoutAutoSelectDetection (such as for controlling of announcement of new line text).

…ng lists. this ensures compatibility with the older MS Word implementation, and stops confusion when pressing enter for new bullets where NVDA was announcing exiting and then entering a new list.

derekriemer · 2017-12-21T02:19:25Z

source/NVDAObjects/IAccessible/winword.py

 from displayModel import EditableTextDisplayModelTextInfo
 from NVDAObjects.window import DisplayModelEditableText
+from ..behaviors import EditableTextWithoutAutoSelectDetection
+from NVDAObjects.window.winword import *
+


Would a comment here for explanation of what this file is for be worth putting?

derekriemer · 2017-12-21T02:35:35Z

source/NVDAObjects/UIA/wordDocument.py

+		elif obj.UIAElement.cachedControlType==UIAHandler.UIA_CustomControlTypeId and obj.name:
+			# Include foot note and endnote identifiers
+			field['content']=obj.name
+			#field['alwaysReportName']=True


Is there a reason this is commented?

derekriemer · 2017-12-21T02:37:59Z

source/NVDAObjects/UIA/wordDocument.py

+				# To stay compatible with the older MS Word implementation, don't expose lists in word documents as actual lists. This suppresses announcement of entering and exiting them.
+				# Note that bullets and numbering are still announced of course.
+				# Eventually we'll want to stop suppressing this, but for now this is more confusing than good (as in many cases announcing of new bullets when pressing enter causes exit and then enter to be spoken).
+				field['role']=controlTypes.ROLE_EDITABLETEXT
 		if obj.role==controlTypes.ROLE_GRAPHIC:
 			# Label graphics with a description before name as name seems to be auto-generated (E.g. "rectangle")
 			field['value']=field.pop('description',None) or obj.description or field.pop('name',None) or obj.name


Will this break Microsoft's AI generated image descriptions possibly?

derekriemer · 2017-12-21T02:40:36Z

source/NVDAObjects/UIA/wordDocument.py

+		try:
+			UIAElementArray=val.QueryInterface(UIAHandler.IUIAutomationElementArray)
+		except COMError:
+			print "not an array"


Debugging code.

derekriemer · 2017-12-21T02:55:09Z

source/NVDAObjects/__init__.py

@@ -139,7 +140,7 @@ def clearDynamicClassCache(cls):
 		"""
 		cls._dynamicClassCache.clear()

-class NVDAObject(baseObject.ScriptableObject):
+class NVDAObject(documentBase.TextContainerObject,baseObject.ScriptableObject):
 	"""NVDA's representation of a single control/widget.
 	Every widget, regardless of how it is exposed by an application or the operating system, is represented by a single NVDAObject instance.
 	This allows NVDA to work with all widgets in a uniform way.


Does that change need a docs update?

derekriemer · 2017-12-21T03:03:58Z

source/_UIAHandler.py

@@ -312,7 +312,13 @@ def _isUIAWindowHelper(self,hwnd):
 		if appModule and appModule.isBadUIAWindow(hwnd):
 			return False
 		# Ask the window if it supports UIA natively
-		return windll.UIAutomationCore.UiaHasServerSideProvider(hwnd)
+		res=windll.UIAutomationCore.UiaHasServerSideProvider(hwnd)


Just curious, why is this being changed for this project?

derekriemer · 2017-12-21T03:04:43Z

source/_UIAHandler.py

-		return windll.UIAutomationCore.UiaHasServerSideProvider(hwnd)
+		res=windll.UIAutomationCore.UiaHasServerSideProvider(hwnd)
+		if res:
+			# the window does support UIA natively, but


That comment isn't a sentence.

derekriemer · 2017-12-21T03:20:24Z

source/documentBase.py

+import ui
+import controlTypes
+
+class TextContainerObject(AutoPropertyObject):


There is no docstring here.

derekriemer · 2017-12-21T03:21:07Z

source/documentBase.py

+		info.updateSelection()
+
+class DocumentWithTableNavigation(TextContainerObject,ScriptableObject):
+


…ng errors.

…currently under annotations like old MS Word).

…lements list.)

josephsl · 2018-01-03T06:01:27Z

Hi,

For @michaelDCurran: please keep an eye on #7456, as your latest commit fixes it for now. Thanks.

josephsl · 2018-01-04T23:07:24Z

Hi,

Reported by a user (paraphrased): When opening Start menu in Windows 7, an error is recorded.

STR:

From Windows 7, install latest next snapshot.
Open Start menu.

Expected: no errors.
Actual:

ERROR - eventHandler.executeEvent (21:12:00.062):
error executing event: gainFocus on <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x05DFBEF0> with extra args of {}
Traceback (most recent call last):
File "eventHandler.pyc", line 152, in executeEvent
File "eventHandler.pyc", line 92, in init
File "eventHandler.pyc", line 100, in next
File "C:\Users\Angelo\AppData\Roaming\nvda\addons\remote\globalPlugins\remoteClient_init_.py", line 410, in event_gainFocus
File "eventHandler.pyc", line 100, in next
File "appModules\explorer.pyc", line 280, in event_gainFocus
File "eventHandler.pyc", line 100, in next
File "NVDAObjects_init_.pyc", line 951, in event_gainFocus
File "NVDAObjects_init_.pyc", line 839, in reportFocus
File "speech.pyc", line 396, in speakObject
File "speech.pyc", line 752, in speakTextInfo
File "NVDAObjects\UIA_init_.pyc", line 621, in getTextWithFields
File "NVDAObjects\UIA_init_.pyc", line 611, in getTextWithFieldsForUIARange
File "NVDAObjects\UIA_init.pyc", line 446, in getTextWithFields_text
File "NVDAObjects\UIA_init.pyc", line 241, in _getFormatFieldAtRange
UnboundLocalError: local variable 'fetchAnnotationTypes' referenced before assignment

Thanks.

zstanecic · 2018-01-04T23:22:28Z

@josephsl Has someone tested this? This implementation in windows 10 works great Wysłane z aplikacji Poczta dla Windows 10 Od: Joseph Lee Wysłano: piątek, 5 stycznia 2018 00:07 Do: nvaccess/nvda DW: Subscribed Temat: Re: [nvaccess/nvda] Allow access to Microsoft Word documents viaUIAutomation, providing a significant speed improvement, and ability to accessdocuments when in-process injection is impossible. (#7849) Hi, Reported by a user (paraphrased): When opening Start menu in Windows 7, an error is recorded. STR: 1. From Windows 7, install latest next snapshot. 2. Open Start menu. Expected: no errors. Actual: ERROR - eventHandler.executeEvent (21:12:00.062): error executing event: gainFocus on <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x05DFBEF0> with extra args of {} Traceback (most recent call last): File "eventHandler.pyc", line 152, in executeEvent File "eventHandler.pyc", line 92, in init File "eventHandler.pyc", line 100, in next File "C:\Users\Angelo\AppData\Roaming\nvda\addons\remote\globalPlugins\remoteClient_init_.py", line 410, in event_gainFocus File "eventHandler.pyc", line 100, in next File "appModules\explorer.pyc", line 280, in event_gainFocus File "eventHandler.pyc", line 100, in next File "NVDAObjects_init_.pyc", line 951, in event_gainFocus File "NVDAObjects_init_.pyc", line 839, in reportFocus File "speech.pyc", line 396, in speakObject File "speech.pyc", line 752, in speakTextInfo File "NVDAObjects\UIA_init_.pyc", line 621, in getTextWithFields File "NVDAObjects\UIA_init_.pyc", line 611, in getTextWithFieldsForUIARange File "NVDAObjects\UIA_init.pyc", line 446, in getTextWithFields_text File "NVDAObjects\UIA_init.pyc", line 241, in _getFormatFieldAtRange UnboundLocalError: local variable 'fetchAnnotationTypes' referenced before assignment Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

josephsl · 2018-01-04T23:30:50Z

Hi, yes, someone is testing this on Windows 7. Thanks. From: zstanecic [mailto:notifications@github.com] Sent: Thursday, January 4, 2018 3:23 PM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com> Subject: Re: [nvaccess/nvda] Allow access to Microsoft Word documents via UIAutomation, providing a significant speed improvement, and ability to access documents when in-process injection is impossible. (#7849) @josephsl Has someone tested this? This implementation in windows 10 works great Wysłane z aplikacji Poczta dla Windows 10 Od: Joseph Lee Wysłano: piątek, 5 stycznia 2018 00:07 Do: nvaccess/nvda DW: Subscribed Temat: Re: [nvaccess/nvda] Allow access to Microsoft Word documents viaUIAutomation, providing a significant speed improvement, and ability to accessdocuments when in-process injection is impossible. (#7849) Hi, Reported by a user (paraphrased): When opening Start menu in Windows 7, an error is recorded. STR: 1. From Windows 7, install latest next snapshot. 2. Open Start menu. Expected: no errors. Actual: ERROR - eventHandler.executeEvent (21:12:00.062): error executing event: gainFocus on <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x05DFBEF0> with extra args of {} Traceback (most recent call last): File "eventHandler.pyc", line 152, in executeEvent File "eventHandler.pyc", line 92, in init File "eventHandler.pyc", line 100, in next File "C:\Users\Angelo\AppData\Roaming\nvda\addons\remote\globalPlugins\remoteClient_init_.py", line 410, in event_gainFocus File "eventHandler.pyc", line 100, in next File "appModules\explorer.pyc", line 280, in event_gainFocus File "eventHandler.pyc", line 100, in next File "NVDAObjects_init_.pyc", line 951, in event_gainFocus File "NVDAObjects_init_.pyc", line 839, in reportFocus File "speech.pyc", line 396, in speakObject File "speech.pyc", line 752, in speakTextInfo File "NVDAObjects\UIA_init_.pyc", line 621, in getTextWithFields File "NVDAObjects\UIA_init_.pyc", line 611, in getTextWithFieldsForUIARange File "NVDAObjects\UIA_init.pyc", line 446, in getTextWithFields_text File "NVDAObjects\UIA_init.pyc", line 241, in _getFormatFieldAtRange UnboundLocalError: local variable 'fetchAnnotationTypes' referenced before assignment Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7849 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkM0iHaUCGLsaIVLteqf4zIOqLa4Qks5tHV02gaJpZM4Q-UoF> .

zstanecic · 2018-01-05T00:53:32Z

Well, conclusion, If we will have an opportunity to squash these bugs, we will have fast MS Word experience Even in the large documents. The performance with Braille is notably increased, and it’s good to se it. Wysłane z aplikacji Poczta dla Windows 10 Od: Joseph Lee Wysłano: piątek, 5 stycznia 2018 00:31 Do: nvaccess/nvda DW: zstanecic; Comment Temat: Re: [nvaccess/nvda] Allow access to Microsoft Word documents viaUIAutomation, providing a significant speed improvement, and ability to accessdocuments when in-process injection is impossible. (#7849) Hi, yes, someone is testing this on Windows 7. Thanks. From: zstanecic [mailto:notifications@github.com] Sent: Thursday, January 4, 2018 3:23 PM To: nvaccess/nvda <nvda@noreply.github.com> Cc: Joseph Lee <joseph.lee22590@gmail.com>; Mention <mention@noreply.github.com> Subject: Re: [nvaccess/nvda] Allow access to Microsoft Word documents via UIAutomation, providing a significant speed improvement, and ability to access documents when in-process injection is impossible. (#7849) @josephsl Has someone tested this? This implementation in windows 10 works great Wysłane z aplikacji Poczta dla Windows 10 Od: Joseph Lee Wysłano: piątek, 5 stycznia 2018 00:07 Do: nvaccess/nvda DW: Subscribed Temat: Re: [nvaccess/nvda] Allow access to Microsoft Word documents viaUIAutomation, providing a significant speed improvement, and ability to accessdocuments when in-process injection is impossible. (#7849) Hi, Reported by a user (paraphrased): When opening Start menu in Windows 7, an error is recorded. STR: 1. From Windows 7, install latest next snapshot. 2. Open Start menu. Expected: no errors. Actual: ERROR - eventHandler.executeEvent (21:12:00.062): error executing event: gainFocus on <NVDAObjects.Dynamic_EditableTextWithoutAutoSelectDetectionUIA object at 0x05DFBEF0> with extra args of {} Traceback (most recent call last): File "eventHandler.pyc", line 152, in executeEvent File "eventHandler.pyc", line 92, in init File "eventHandler.pyc", line 100, in next File "C:\Users\Angelo\AppData\Roaming\nvda\addons\remote\globalPlugins\remoteClient_init_.py", line 410, in event_gainFocus File "eventHandler.pyc", line 100, in next File "appModules\explorer.pyc", line 280, in event_gainFocus File "eventHandler.pyc", line 100, in next File "NVDAObjects_init_.pyc", line 951, in event_gainFocus File "NVDAObjects_init_.pyc", line 839, in reportFocus File "speech.pyc", line 396, in speakObject File "speech.pyc", line 752, in speakTextInfo File "NVDAObjects\UIA_init_.pyc", line 621, in getTextWithFields File "NVDAObjects\UIA_init_.pyc", line 611, in getTextWithFieldsForUIARange File "NVDAObjects\UIA_init.pyc", line 446, in getTextWithFields_text File "NVDAObjects\UIA_init.pyc", line 241, in _getFormatFieldAtRange UnboundLocalError: local variable 'fetchAnnotationTypes' referenced before assignment Thanks. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7849 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AHgLkM0iHaUCGLsaIVLteqf4zIOqLa4Qks5tHV02gaJpZM4Q-UoF> . — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

Brian1Gaff · 2018-01-05T09:10:59Z

Which is the oldest version of Office now supported. in general at the moment I see that at least master snaps still work on 2002/3 versions of Word, though I know the former is not officially supported.

zstanecic · 2018-01-05T21:54:49Z

Hi, I think that i found a Bug. 1. Open Microsoft word, opening an existing document. 2. Open elements list with insert +f7 3. From the filter elements radio button, select grammar errors 4. As a result, you will see, that MS word crashes, and elements list leaves opened. Using latest next, Windows 10 1709 16299 Installed NVDA

ruifontes · 2018-01-06T00:46:30Z

Curiously, with: NVDA last Next Windows version 10.0.15063 workstation Word 2013 I can't open the Elements list! IO - inputCore.InputManager.executeGesture (00:38:01.766): Input: kb(desktop):NVDA+f7 IO - speech.speak (00:38:05.506): Speaking [u'Spelling Tool bar'] IO - speech.speak (00:38:05.913): Speaking [u'Ignore button I'] Do you want me to test something? Regards, Rui Às 21:54 de 05/01/2018, zstanecic escreveu:

…

Hi, I think that i found a Bug. 1. Open Microsoft word, opening an existing document. 2. Open elements list with insert +f7 3. From the filter elements radio button, select grammar errors 4. As a result, you will see, that MS word crashes, and elements list leaves opened. Using latest next, Windows 10 1709 16299 Installed NVDA — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7849 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APIHxp2ujY76SuEasXDa7uXsggCgxYP4ks5tHposgaJpZM4Q-UoF>.

zstanecic · 2018-01-06T00:49:16Z

@ruifontesare you in browse mode? Wysłane z aplikacji Poczta dla Windows 10 Od: ruifontes Wysłano: sobota, 6 stycznia 2018 01:46 Do: nvaccess/nvda DW: zstanecic; Comment Temat: Re: [nvaccess/nvda] Allow access to Microsoft Word documents viaUIAutomation, providing a significant speed improvement, and ability to accessdocuments when in-process injection is impossible. (#7849) Curiously, with: NVDA last Next Windows version 10.0.15063 workstation Word 2013 I can't open the Elements list! IO - inputCore.InputManager.executeGesture (00:38:01.766): Input: kb(desktop):NVDA+f7 IO - speech.speak (00:38:05.506): Speaking [u'Spelling Tool bar'] IO - speech.speak (00:38:05.913): Speaking [u'Ignore button I'] Do you want me to test something? Regards, Rui Às 21:54 de 05/01/2018, zstanecic escreveu:

Hi, I think that i found a Bug. 1. Open Microsoft word, opening an existing document. 2. Open elements list with insert +f7 3. From the filter elements radio button, select grammar errors 4. As a result, you will see, that MS word crashes, and elements list leaves opened. Using latest next, Windows 10 1709 16299 Installed NVDA — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#7849 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APIHxp2ujY76SuEasXDa7uXsggCgxYP4ks5tHposgaJpZM4Q-UoF>.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

LeonarddeR · 2018-01-06T11:57:47Z

I found another browse mode bug regarding finding:

Enter the following lines of text:
```
Hello!
How are you?
```
Go to the top of the document
Enable browse mode
Search for how

It seems that the cursor does move one cursor position to the right, but it does not move to the actual result.

fernando-jose-silva · 2018-01-08T17:52:24Z

Sorry for opening a separate call to this discussion.
As I reported in the call I opened 7903, I would just like to report that I can not read characters, words or lines when editing a cell in excel 2013 on windows 7.
I saw that as far as the menu to start this had already been approached in this discussion.
Thank you.

…AnnotationTypes variable before any point where it can be used. Specifically: If the IUIAutomationTextRange3 interface was missing, fetchAnnotationTypes was never set. It should be set to False by default.

michaelDCurran · 2018-01-11T20:49:49Z

The issue with find in browse mode is clearly a bug in MS Word's UI automation implementation. If it finds text, it returns the original text range, rather than the position of the found string. There is no way to work around this in NVDA.

michaelDCurran · 2018-01-11T20:50:30Z

Re grammar errors crash with Elements list: a test word document file would be great.

michaelDCurran · 2018-01-11T20:51:31Z

and the Windows 7 attributeError in the start menu has been fixed in the last commit to this branch.

zstanecic · 2018-01-11T21:00:10Z

Here is the document in the attachment. A sample one which shows you the crash And, in the elements list just press left arrow t select errors radio buton, and crach happens, and there we go. Wysłane z aplikacji Poczta dla Windows 10 Od: Michael Curran Wysłano: czwartek, 11 stycznia 2018 21:50 Do: nvaccess/nvda DW: zstanecic; Comment Temat: Re: [nvaccess/nvda] Allow access to Microsoft Word documents viaUIAutomation, providing a significant speed improvement, and ability to accessdocuments when in-process injection is impossible. (#7849) Re grammar errors crash with Elements list: a test word document file would be great. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

zstanecic · 2018-01-11T21:41:23Z

here is the attachment
Zvonimir Stanecic rjecnik Govorne vj 2017.-2018.docx

Brian1Gaff · 2018-01-12T15:13:32Z

Yes I know. I only posted the other search error out of completeness. Brian

ferhatbkr · 2018-12-29T13:00:54Z

Hi, I activated the this feature. But, I think I found an issue.
When using NVDA in mouse tracking mode, NVDA doesn't report the main text of the Word document. NVDA reports header and footer text, buttons, menus, etc. NVDA says"<current_page> content" instead reporting the text.
I'm using Office 365 ProPlus on Windows 10 64-bit Home. Windows and Office both are up to date.

LeonarddeR · 2018-12-29T14:58:52Z

@ferhatbkr: What version of NVDA did you test this? Mouse tracking should work in latest Alpha builds.

ferhatbkr · 2018-12-30T00:26:44Z

@LeonarddeR I didn't try on alpha version, only on latest stable.

zstanecic · 2018-12-30T08:11:04Z

Hi to all, I can say, that for me it works perfectly. From: ferhatbkr <notifications@github.com> Sent: Sunday, December 30, 2018 1:27 AM To: nvaccess/nvda <nvda@noreply.github.com> Cc: zstanecic <zvonimirek222@yandex.com>; Comment <comment@noreply.github.com> Subject: Re: [nvaccess/nvda] Allow access to Microsoft Word documents via UIAutomation, providing a significant speed improvement, and ability to access documents when in-process injection is impossible. (#7849) @LeonarddeR <https://github.com/leonardder> I didn't try on alpha version, only on latest stable. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#7849 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AKohk5fJ6zBOmf-0GjyPNkf8BjpMcnmvks5u-AhGgaJpZM4Q-UoF> . <https://github.com/notifications/beacon/AKohk5CIPrtgevwdlHyKrKbqe5YLELUCks5u-AhGgaJpZM4Q-UoF.gif>

michaelDCurran added 5 commits December 12, 2017 10:48

Allow browse mode in Microsoft Word using UIA

915bb2b

michaelDCurran requested a review from feerrenrut December 12, 2017 02:20

UIA NVDAObject: rowHeaderText and columnHeaderText now filter out the…

f588bd2

… current cell from the headers if inappropriately put there by the UIA implementation.

feerrenrut approved these changes Dec 12, 2017

View reviewed changes

michaelDCurran added 2 commits December 13, 2017 07:59

Address review actions

3b714db

Winword's script_tab: when we have a collapsed range on a table cell,…

cbe86d7

… expand to paragraph rather than cell, as not all implementations can support cell (such as UIA). Paragraph is certainly enough info to speak anyway.

michaelDCurran added 3 commits December 13, 2017 08:58

Allow editableText's newLine script to announce the entire new line i…

7c8bb29

…f announceEntireNewLine class viarable is true. This is false by default, but UIA WordDocument sets this to true for better list bullet reading.

UIA WordDocument: for now suppress announcement of entering and exiti…

31c9019

…ng lists. this ensures compatibility with the older MS Word implementation, and stops confusion when pressing enter for new bullets where NVDA was announcing exiting and then entering a new list.

michaelDCurran force-pushed the wordDocumentsWithUIA branch from 90e09de to 31c9019 Compare December 12, 2017 23:15

Remove blank line

ecfdaf1

michaelDCurran mentioned this pull request Dec 13, 2017

Package NVDA for the Windows Store #7851

Merged

michaelDCurran added 2 commits December 21, 2017 10:19

Merge branch 'master' into wordDocumentsWithUIA

b33b2a5

Merge branch 'master' into wordDocumentsWithUIA

139bf0a

derekriemer reviewed Dec 21, 2017

View reviewed changes

michaelDCurran added 3 commits December 21, 2017 17:39

UIA browseMode: add support for quicknav and elements list for spelli…

0fcdc2e

…ng errors.

UIA WordDocument support: add quicknav / elements list for comments (…

92738d7

…currently under annotations like old MS Word).

Address review actions: add docstrings, remove some debugging code.

a16927b

LeonarddeR mentioned this pull request Dec 21, 2017

Report row and column span for table cells in speech and braille #7872

Merged

Add support for grammar errors to UIA (inline reading, quicknav and e…

1a732cd

…lements list.)

josephsl mentioned this pull request Jan 3, 2018

nvda and windows mail: backspace issue #7456

Closed

LeonarddeR mentioned this pull request Jan 3, 2018

Enhanced support for form controls in MS Word #6944

Open

josephsl mentioned this pull request Jan 8, 2018

sound error when opening windows 7 start menu #7903

Closed

UIATextInfo: fix an AttributeError on Windows 7 by defining the fetch…

10d6df3

…AnnotationTypes variable before any point where it can be used. Specifically: If the IUIAutomationTextRange3 interface was missing, fetchAnnotationTypes was never set. It should be set to False by default.

michaelDCurran added a commit that referenced this pull request Jan 11, 2018

Incubates #7849

fa79787

michaelDCurran merged commit 38d28f9 into master Jan 17, 2018

nvaccessAuto removed the incubating label Jan 17, 2018

nvaccessAuto added this to the 2018.1 milestone Jan 17, 2018

josephsl mentioned this pull request Mar 23, 2018

Switch to UIA for Microsoft Word 2016 and later #7409

Closed

		info.updateSelection()

		class DocumentWithTableNavigation(TextContainerObject,ScriptableObject):

Allow access to Microsoft Word documents via UIAutomation, providing a significant speed improvement, and ability to access documents when in-process injection is impossible. #7849

Allow access to Microsoft Word documents via UIAutomation, providing a significant speed improvement, and ability to access documents when in-process injection is impossible. #7849

Conversation

michaelDCurran commented Dec 12, 2017 • edited

Link to issue number:

Summary of the issue:

Speed improvements

Unable to inject

Description of how this pull request fixes the issue:

Abstraction of table navigation scripts

Ensure browseMode works in Microsoft Word with UIAutomation

Report more formatting in UIATextInfo

Further specific Microsoft Word UIA enhansements

Allow UIA Microsoft Word support in various situations

Testing performed:

Known issues with pull request:

Change log entry:

feerrenrut left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeonarddeR commented Dec 12, 2017

Change log entry:

Brian1Gaff commented Dec 12, 2017 via email

derekriemer commented Dec 12, 2017

michaelDCurran commented Dec 12, 2017

michaelDCurran commented Dec 12, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

josephsl commented Jan 3, 2018

josephsl commented Jan 4, 2018

zstanecic commented Jan 4, 2018 via email

josephsl commented Jan 4, 2018 via email

zstanecic commented Jan 5, 2018 via email

Brian1Gaff commented Jan 5, 2018 via email

zstanecic commented Jan 5, 2018 via email

ruifontes commented Jan 6, 2018 via email

zstanecic commented Jan 6, 2018 via email

LeonarddeR commented Jan 6, 2018

fernando-jose-silva commented Jan 8, 2018

michaelDCurran commented Jan 11, 2018

michaelDCurran commented Jan 11, 2018

michaelDCurran commented Jan 11, 2018

zstanecic commented Jan 11, 2018 via email

zstanecic commented Jan 11, 2018

Brian1Gaff commented Jan 12, 2018 via email

ferhatbkr commented Dec 29, 2018

LeonarddeR commented Dec 29, 2018

ferhatbkr commented Dec 30, 2018

zstanecic commented Dec 30, 2018 via email

michaelDCurran commented Dec 12, 2017 •

edited