Improve working with message lists and calendars in Outlook #7949

LeonarddeR · 2018-01-31T20:31:33Z

Link to issue number:

Fixes #6911
Fixes #8028
Reverts #6219

Summary of the issue:

In the Outlook messages list, replied to or forwarded status is not announced by NVDA.
In outlook's calendar view, the month view is overly verbose, announcing redundant information (e.g. 00.00AM to next day 00.00AM
When a message has an associated draft, this is not reported.

Description of how this pull request fixes the issue:

This pr gets the announcement of replied/forwarded state from the UIA value of rows in the Outlook messages list. NVDA will now also announce meeting requests based on this information, as well as appointment cancellations.

The initial implementation of this pr relied on the Outlook object model. However, this caused security warnings on some systems with a tight security policy.
When an appointment or time slot is now exactly one day (86400 seconds), and starts at 00.00, the time slot is announced as 26 february 2018 (entire day) instead of 26 februari 2018 00.00 to 27 februari 2018 00.00. While looking at this code, I noticed that Microsoft strongly discourages the use of kernel32 GetTimeFormat and GetDateFormat functions, so I changed this implementation to use the *Ex alternatives for these functions.
The associated draft column in Outlook 2016 is no longer silenced. After implementing Outlook 2016: NVDA reads "Associated draft, no associated drafts" for every message #6219, it seems Microsoft decided to present an empty text node when there is no associated draft, rather than spamming everyone with the "no associated draft" message. Therefore when this particular pr is merged, everyone who gets "no associated draft" for almost every message is advised to update Outlook 2016 to the most recent build.

Testing performed:

Made sure that the current implementation doesn't cause security warnings when setting outlook's program access settings to the most secure option available.
Tested replied/forwarded messages in Outlook 2016 and 2013, the state was reported correctly. The replied state did even report for messages I replied to using Thunderbird. Also tested this in Outlook 2010, and as Outlook 2010 doesn't expose this information using UIA, we fall back to the object model for the unread state.
Tested the announcement of a whole day appointment in the calendar

Known issues with pull request:

The replied or forwarded state is localized by Outlook itself, not by NVDA. This shouldn't be a problem though, as the headers are also localized by Outlook itself.
In theory, Outlook could expose every piece of relevant or irrelevant information using the UIA value. I've only seen the following strings until now:
- Message: usually the first word in the value of a message
- Contact: usually the first word in the value of a contact when the contacts are displayed in a list view
- Appointment: usually the first word in the value of a calendar item when the calendar is displayed in a list view
- Read/Unread: usually the last word in the value of a message, contact or appointment
- Forwarded/Replied: usually the second word in a string containing three words, e.g. Message Replied Read
I'm quite sure JAWS also relies on the value here, as replied/forwarded state is localized using Outlook, not using JAWS. The wild should make clear whether parsing the value this way is too non-restrictive.
The replied to all state is not reported as it is not exposed using the object's value.
When the root of a conversation in conversation view has focus, NVDA reads the headers of every single message in the conversation. There is currently no suitable workaround for this. Note that this was already the case before this pr. This issue is caused by a bug in UIA, where the tree scope is ignored when providing a custom tree filter for a cache request.

Change log entry:

New features
- In Microsoft Outlook message lists, NVDA now reports if a message has been replied to or forwarded. (NVDA not inform about if an e-mail in Outlook 2013 is answered or forwarded #6911)
Changes
- NVDA is now less verbose when an appointment or time slot in the Outlook calendar covers an entire day. (Improve working with message lists and calendars in Outlook #7949)

LeonarddeR · 2018-02-02T20:31:37Z

Blocked by #7953

…ssage's UIAGridRow object

Implementation had to be changed completely.

LeonarddeR · 2018-02-03T12:14:04Z

I had to change this into a completely different implementation due to #7953. @michaelDCurran. It would be great if you could review this as soon as possible to fix #7953.

michaelDCurran · 2018-02-04T22:09:29Z

source/appModules/outlook.py

+			# The first valuePart is the type of the selection, e.g. Message, Contact.
+			# The last valuePart indicates whether the message is read or unread.
+			if len(valueParts)>=3:
+				textList.extend(valueParts[1:-1])


Can we be sure that the first part (message, contact etc) is never more than one word in all languages?

Good question, which also applies to read/unread I belief.

Though JAWS uses these translated strings, I"m pretty sure they do some string matching on it, as the read state is ignored when Both Outlook and JAWS are set to Dutch, but the state is preserved when Outlook is Dutch and JAWS is English. So that's definitely not a clean solution.

I belief both Message and Unread are part of the NVDA translation, so it should be doable to check whether these words contain spaces in other languages without intervention of every single translator.

LeonarddeR · 2018-02-26T12:57:47Z

@michaelDCurran: I made the scope of this pr somewhat broader and also addressed some other concerns there were with the current way of dealing with Outlook. The initial message of the pr covers all relevant information.

dkager · 2018-03-04T17:23:06Z

source/appModules/outlook.py

-}
+#: When in a list view, the message classes which should not be announced for an item.
+#: For these, it should be safe to assume that their names consist of only one word.
+silentMessageClasses = [


So there is no such class for a message/email?

Yes, that's the note one. I added a comment that clarifies this.

dkager · 2018-03-04T17:25:21Z

source/appModules/outlook.py

 		CalendarView._lastStartDate=startDate
+		if endDate!=startDate:
+			if (endDate - startDate).total_seconds()==SECONDS_PER_DAY:


This also catches events that start at noon on one day and end at noon on the next. Do we want to call these all day events or should we check for the start and end times being midnight as well?

dkager · 2018-03-04T17:26:19Z

source/appModules/outlook.py

 		CalendarView._lastStartDate=startDate
+		if endDate!=startDate:
+			if (endDate - startDate).total_seconds()==SECONDS_PER_DAY:
+				# Translators: a message reporting the date of a whole day Outlook calendar entry


Outlook calls them "All day event", not "whole day".

dkager · 2018-03-04T17:26:42Z

source/appModules/outlook.py

+		if endDate!=startDate:
+			if (endDate - startDate).total_seconds()==SECONDS_PER_DAY:
+				# Translators: a message reporting the date of a whole day Outlook calendar entry
+				return _("{date} (entire day)").format(date=startDateText)


Again, "all day" may be better here.

dkager · 2018-03-04T17:30:48Z

source/appModules/outlook.py

+				# Do not expose the read state
+				lastPart = valueCount if unread else valueCount-1
+				# The first valuePart is the type of the selection, e.g. Message, Contact, meeting request.
+				# We can safely assume that the message class of a regular message or appointment is one word.


Yet the example you give above is "Meeting request". Is this not regular? If so, 2 words should be filtered out.

I tried to improve the clarity of the comments. Meeting request is regular, but it isn't part of silentMessageClasses and therefore always announced.

dkager · 2018-03-04T17:32:10Z

source/appModules/outlook.py

-			# Thus we filter it out.
-			if self.appModule.outlookVersion>=16 and e.cachedClassName=="DraftFlagField":
-				continue
+			isFlag = e.cachedClassName=="FlagField"


Does this handle the "DraftFlagField" case?

Good point, I forgot to mention this in the initial pr description, I surely should have done that. It is still open to discussion.

MS no longer exposes "no associated draft" as a text node by default. Instead, the text node is empty. Furthermore, the "no associated draft" patch meant that it would be never announced when a message has an associated draft. Therefore, I propose people who have the "no associated draft" issue updating their outlook to fix this, rather than keeping the old patch. CC @bramd

…nd at noon time

…appointments

derekriemer · 2018-03-05T14:53:20Z

what about saying "all day" instead of "entire day"

LeonarddeR · 2018-03-05T15:12:51Z

@derekriemer commented on 5 mrt. 2018 15:53 CET:

what about saying "all day" instead of "entire day"

It is now saying "all day" as per the last commits from today morning.

LeonarddeR · 2018-03-06T07:19:26Z

@michaelDCurran, this pr has been overhauled and is ready for another review I think. I'm afraid there's no way to work around relying on the localized contents of the UIA value.

…ey ought to be silent.

LeonarddeR · 2018-03-15T11:54:29Z

@josephsl commented on 15 mrt. 2018 06:18 CET:

I wonder if this is the right PR for this.

Looks like it is. It turns out it wasn't a good idea to remove the tree filter from the cache request that forced the fetched children to be text controls. I now added a new tree filter that requires the children to be content children. After that, I couldn't reproduce the issue any more.

I also noticed another bug in the code that tried to access the object model for flag information when there was no selection. That bug was actually obfuscated by the treeFilter one. We are now ignoring flags when there is no object model access, just as we did before this pr.

LeonarddeR · 2018-03-28T04:05:58Z

Currently blocked by #8129. When getting the name for message list items, a NULL COM pointer access error is raised because the custom UIA query fails. This is a bit strange, since until now, it fails only on two systems. @bramd has a system where it fails and another system where it works correctly, same windows and office builds.

Note that such an error has been reported by @josephsl in #7949 (comment), the fix for that has yet not been committed to Next. However, testing by @bramd tells that the fix for #7949 (comment) does not aplly to this particular case of the null com pointer access error.

I guess we can do an if check on cachedChildren, but I'd like to no why getting cached children would reveal a null com pointer in the first place. @michaelDCurran: Have you seen these cases earlier?

… rows

michaelDCurran · 2018-03-28T06:39:11Z

I think I have seen a NULL COM pointer for cached children before. But I can't remember exactly where. We could argue it is a Microsoft bug, but in the end, we will still have to handle it if it is happening in the wild.

…e the ContentViewCondition as tree filter instead of creating a property condition for content elements.

LeonarddeR · 2018-04-04T14:20:25Z

I reverted the announcement of importance, attachments and flags based on UIA children, as it is unreliable in conversation view. The announcement of these fields was based on the assumption that the UIA implementation treats relevant fields as content (i.e. with the IsContentElement property) whereas irrelevant fields (no attachment, no reminder) are non content. However, when conversation view is enabled, none of the relevant children have the IsContentElement property set.

… mixed with tree view items (e.g. in conversation view)

LeonarddeR · 2018-04-12T17:44:55Z

@michaelDCurran: Could you please review the last few commits?

Show replied/forwarded status for messages in Outlook

673dc9c

LeonarddeR requested a review from michaelDCurran January 31, 2018 20:32

LeonarddeR added app/microsoft-office quick fix labels Jan 31, 2018

michaelDCurran previously approved these changes Jan 31, 2018

View reviewed changes

michaelDCurran added a commit that referenced this pull request Jan 31, 2018

Incubates #7949

5c23fec

nvaccessAuto added the incubating label Jan 31, 2018

nvaccessAuto assigned michaelDCurran Jan 31, 2018

LeonarddeR mentioned this pull request Feb 2, 2018

nvda causing the outlook security message to be displayed #7953

Closed

LeonarddeR added the blocked label Feb 2, 2018

New, completely different implementation based on the value of the me…

98b4417

…ssage's UIAGridRow object

LeonarddeR removed the blocked label Feb 3, 2018

michaelDCurran reviewed Feb 4, 2018

View reviewed changes

michaelDCurran removed the incubating label Feb 9, 2018

Leonard de Ruijter added 2 commits February 23, 2018 11:50

Add getDateFormatEx and getTimeFormatEx to winKernel

1bd4a5b

Improvements to the message list and calendar date announcements

89e7367

LeonarddeR changed the title ~~Show replied/forwarded status for messages in Outlook~~ Improve working with message lists and calendars in Outlook Feb 26, 2018

Fixed entire day announcements for calendar

d5e3752

dkager reviewed Mar 4, 2018

View reviewed changes

Leonard de Ruijter added 3 commits March 5, 2018 07:12

Only announce appointments and slots as all day when they start and e…

db2ffc8

…nd at noon time

All day appointments are now called all day and no longer entire day …

0393e1d

…appointments

Improved clarity of some comments

fa3b252

Decrease the likelyhood of announcing partial message classes when th…

78b5cae

…ey ought to be silent.

Leonard de Ruijter added 2 commits March 15, 2018 12:43

Work around null pointer access value error

d3d1392

Ignore flags when the object model couldn't be accessed

aef52b8

LeonarddeR mentioned this pull request Mar 28, 2018

NVDA does not read the message list in Outlook 2016 with the latest Next release #8129

Closed

Catch null com pointer access when fetching UIA children for UIA grid…

be37594

… rows

LeonarddeR force-pushed the outlookReplyStatus branch from ac0c603 to be37594 Compare March 28, 2018 06:36

michaelDCurran added a commit that referenced this pull request Mar 28, 2018

Incubates #7949

d39c7aa

When getting children to construct the name of an Outlook message, us…

8e06454

…e the ContentViewCondition as tree filter instead of creating a property condition for content elements.

LeonarddeR force-pushed the outlookReplyStatus branch from 37b866c to 8e06454 Compare March 29, 2018 13:00

Leonard de Ruijter added 2 commits April 4, 2018 15:00

Merge remote-tracking branch 'origin/master' into outlookReplyStatus

b117cbb

Revert relying on UIA for Attachment, importance and flag info

8206c57

LeonarddeR mentioned this pull request Apr 4, 2018

NVDA reports "has attachment" for messages without visible attachments in Outlook #8028

Closed

Properly get the value for grid rows, also when they are dynamic type…

45add3e

… mixed with tree view items (e.g. in conversation view)

LeonarddeR mentioned this pull request Apr 9, 2018

Add a appModule for taskMan, to fix some bugs in windows ten. #8147

Merged

michaelDCurran approved these changes Apr 15, 2018

View reviewed changes

michaelDCurran added a commit that referenced this pull request Apr 15, 2018

Incubates #7949

c47db1b

PratikP1 mentioned this pull request Apr 18, 2018

Outlook: When selecting multiple messages in 2016, NVDA uses the read/unread characteristics of the first message in the selection #6819

Open

michaelDCurran merged commit 758a116 into nvaccess:master Apr 30, 2018

nvaccessAuto removed the incubating label Apr 30, 2018

nvaccessAuto added this to the 2018.2 milestone Apr 30, 2018

michaelDCurran mentioned this pull request May 31, 2018

MS Outlook, answered e-mails are reported as "read" #8351

Closed

LeonarddeR mentioned this pull request Aug 8, 2018

Outlook 2016: report category and flagged status for messages. #8608

Merged

LeonarddeR added the BabbageWork Pull requests filed on behalf of Babbage B.V. label Oct 11, 2019

seanbudd mentioned this pull request Mar 5, 2021

Deprecations for 2021.1 #12123

Closed

18 tasks

seanbudd added the deprecated/2021.1 Label used to track deprecations due for removal in the 2021.1 release label Mar 5, 2021

seanbudd mentioned this pull request Mar 9, 2021

remove winKernel.GetTimeFormat, winKernel.GetDateFormat and usages #12139

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve working with message lists and calendars in Outlook #7949

Improve working with message lists and calendars in Outlook #7949

LeonarddeR commented Jan 31, 2018 •

edited

LeonarddeR commented Feb 2, 2018

LeonarddeR commented Feb 3, 2018

michaelDCurran Feb 4, 2018

LeonarddeR Feb 5, 2018

LeonarddeR commented Feb 26, 2018

dkager Mar 4, 2018

LeonarddeR Mar 5, 2018

dkager Mar 4, 2018

dkager Mar 4, 2018

dkager Mar 4, 2018

dkager Mar 4, 2018

LeonarddeR Mar 5, 2018

dkager Mar 4, 2018

LeonarddeR Mar 5, 2018

dkager Mar 5, 2018

derekriemer commented Mar 5, 2018

LeonarddeR commented Mar 5, 2018

LeonarddeR commented Mar 6, 2018

LeonarddeR commented Mar 15, 2018

LeonarddeR commented Mar 28, 2018

michaelDCurran commented Mar 28, 2018 via email

LeonarddeR commented Apr 4, 2018

LeonarddeR commented Apr 12, 2018

Improve working with message lists and calendars in Outlook #7949

Improve working with message lists and calendars in Outlook #7949

Conversation

LeonarddeR commented Jan 31, 2018 • edited

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing performed:

Known issues with pull request:

Change log entry:

LeonarddeR commented Feb 2, 2018

LeonarddeR commented Feb 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeonarddeR commented Feb 26, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

derekriemer commented Mar 5, 2018

LeonarddeR commented Mar 5, 2018

LeonarddeR commented Mar 6, 2018

LeonarddeR commented Mar 15, 2018

LeonarddeR commented Mar 28, 2018

michaelDCurran commented Mar 28, 2018 via email

LeonarddeR commented Apr 4, 2018

LeonarddeR commented Apr 12, 2018

LeonarddeR commented Jan 31, 2018 •

edited