office365 setServerEncoding US_ASCII mb_list_encodings

[blu-towers]

I recently updated to the latest version and office365 is not working corrrectly.

It seems that the US-ASCII needed for office365 is not in the list of mb_list_encodings(), PHP 7.2.16 and mbstrings, iconv installed.

On line 76 in Mailbox.php i put a bit of a hack in to fix the issue for me:

if(in_array($serverEncoding, mb_list_encodings()) || $serverEncoding = 'US-ASCII') {

It now seems to be working again with no issues but would like to take that hack out any ideas?

[Sebbo94BY]

Thanks for the feedback and possible solution. I'll look into those Exchange / Office 365 specific issues the rest of this week.

@blu-towers: Is there a way to get a test mail inbox for debugging and testing this with a Microsoft mail server? As far as I could see, the Exchange / Mail feature is only included in the company / business paid plans.

If not, I need to push the changes into develop and someone else (eg. you) needs to test it.

[Sebbo94BY]

Related to #101, #128, #184 and #274.

[blu-towers]

i dont have a test mail server but i can check any updates on my server, thanks for the response on this.

[blu-towers]

I think you can also get away with setting the encoding to null, https://github.com/freescout-helpdesk/freescout/issues/176

[Sebbo94BY]

Thanks, I'll check this.

I just found, that this may can solve the issue as well:

$mailbox->setConnectionArgs(0, 0, array('DISABLE_AUTHENTICATOR' => 'GSSAPI'));
// OR
// $mailbox->setConnectionArgs(0, 0, array('DISABLE_AUTHENTICATOR' => 'PLAIN'));

[Sebbo94BY]

@blu-towers I could create an account at https://outlook.live.com/owa/ for free.

This account allows IMAP with these settings:

Setting	Value
Protocol	IMAP
Server	outlook.office365.com
Port	993
Encryption	TLS

So the host connection string for new Mailbox() should look like this: {outlook.office365.com:993/imap/ssl}

Using my test script, I'm able to connect and fetch ALL emails without any issues. It doesn't matter, if I change the server encoding from UTF-8 to US-ASCII or leave it at UTF-8:

$ php outlook.php
Trying to connect to '{outlook.office365.com:993/imap/ssl}'...
Found 2 email(s)...
+------ P A R S I N G ------+
From: Outlook-Team <no-reply@microsoft.com>
Subject: Welcome to your new Outlook.com account
+------ P A R S I N G ------+
From: Sebastian Krätzig <sebastian.kraetzig@example.com>
Subject: PHP-IMAP Test on outlook.office365.com

But as soon as I try to fetch for example UNSEEN SINCE 01-May-2019 emails, it fails with the error message IMAP method imap_search() failed with error: [BADCHARSET (US-ASCII)] The specified charset is not supported.:

$ php outlook.php
Trying to connect to '{outlook.office365.com:993/imap/ssl}INBOX'...
PHP Fatal error:  Uncaught PhpImap\Exception: IMAP method imap_search() failed with error: [BADCHARSET (US-ASCII)] The specified charset is not supported. in /home/sebastian/imap-test/vendor/php-imap/php-imap/src/PhpImap/Mailbox.php:941
Stack trace:
#0 /home/sebastian/imap-test/vendor/php-imap/php-imap/src/PhpImap/Mailbox.php(302): PhpImap\Mailbox->imap('search', Array)
#1 /home/sebastian/imap-test/outlook.php(24): PhpImap\Mailbox->searchMailbox('UNSEEN SINCE 01...')
#2 {main}
  thrown in /home/sebastian/imap-test/vendor/php-imap/php-imap/src/PhpImap/Mailbox.php on line 941

This is the reported issue, so I'm able to debug and test it. I'll update you as soon as I could solve the issue.

---

References:

• https://social.technet.microsoft.com/Forums/en-US/00bb2ec5-9951-40b4-8ec3-97e5a8f1a59f/imap-search-with-utf8?forum=exchangesvrsecuremessaging
• https://stackoverflow.com/questions/40436674/exchange2013-imap-search-can-support-any-other-charset
• https://docs.microsoft.com/en-us/previous-versions/office/exchange-server-2010/bb310794(v=exchg.141)
• https://docs.microsoft.com/en-us/exchange/mail-flow-best-practices/remote-domains/supported-character-sets
• https://social.technet.microsoft.com/Forums/en-US/4b19df5f-7d41-449f-8b78-f446561455ac/exchange-mime-character-set?forum=exchange2010
• https://busylog.net/imap-telnet-example-test-imap-with-telnet/
• https://tools.ietf.org/html/rfc5255#section-4.5
• https://tools.ietf.org/html/rfc5738#section-3

[blu-towers]

Yes i can confim i am using searchMailbox() with a SINCE query with the same fatal error

[Sebbo94BY]

Yeah, I'm currently testing, how we can support UTF-8 characters, because when you simply set no encoding at all, you won't find strings (emails) with non-ascii characters.

For example, the search will work with FROM "sebastian" and FROM "kr", but NOT with FROM "krätzig". Email From field has "Sebastian Krätzig" in this test-case for example.

So, this is a bit tricky and I couldn't find a solution yet, but I'm sure, that I'll find one. :)

[Sebbo94BY]

Setting the charset / server encoding to US-ASCII allows the usage of other search criteria such as UNSEEN and SINCE, but you're still unable to search for UTF-8 characters like äöü or any other language specific characters like あ.

I would like to solve both of these issues, so you can use all criterias and search for non-ASCII characters.

I've tried to convert the search string into any other charset then UTF-8 (eg. ISO-8859-1, UTF-7, 8bit, ...), but all what I've tested didn't found my expected emails.

I've asked a question at stack overflow, where I've explained everything more detailed. Seems like as the Exchange servers have only implemented the RFC 3501 (US-ASCII must be supported) and will not support a charset based search.

However, I've already committed and pushed some changes into develop branch:

• Added support for US-ASCII server encoding (charset)
• Added ability to disable serverEncoding for searchMailbox()

By default the charset used by imap_search is set to NULL, but the source code also mentions, that it will be NIL in some cases.

// serverEncoding is set to 'UTF-8'

// php-imap v3.0.10 and older
$mailsIds = $mailbox->searchMailbox('UNSEEN'); // will fail with Microsoft Exchange servers

// php-imap v3.0.x and newer (not released yet)
$mailsIds = $mailbox->searchMailbox('UNSEEN', true); // will work with Microsoft Exchange servers

So the only missing change is the fix for being able to search for non-ASCII characters as well.

Let's test, if we can build a function / some logic to do the search by our own instead by the IMAP server.

[EndikaRamos]

Hi,

I don't know if it will help, but I've done some testing in an office365 account:

Subject: "Hi Käfer"

Testing with:
$mailsIds = imap_search($mbox_connection, 'SUBJECT "Käfer"', SE_UID, "US-ASCII");
Results:
Trying to connect to '{outlook.office365.com:993/imap/ssl}INBOX'...
No emails found!

Testing with:
$mailsIds = imap_search($mbox_connection, 'TEXT "Kafer"', SE_UID, "US-ASCII");

Results:
Trying to connect to '{outlook.office365.com:993/imap/ssl}INBOX'...
Found 1 email(s)...
+------ P A R S I N G ------+
From: Endika <Endika ------------@hotmail.com>
Subject: =?iso-8859-1?Q?Hi_K=E4fer?=

It seems, that if I replace, the ä letter with the same letter without the diaeresis (a), and instead of using the SUBJECT parameter I use the TEXT parameter, the server returns a correct result.

So, why don't we try to "translate" the non-ascii characters with the same letter but without the umlaut mark (For example: ä to a) and using a "TEXT" search instead of "Subject" ?

One again, forgive my English, is not my primary language.

[Sebbo94BY]

Thanks for testing!

There are unfortunately a few issues with this solution:

• You're still unable to search for chinese / japanese characters
• Searching in TEXT instead of SUBJECT will be much slower as it will search the string in more and may much longer mail fields

You usually want to use those other search criterias to speed up the search. :)

I believe, we really need to fetch all headers, decode them and then return matching mails.

[Sebbo94BY]

Closing this issue since it is now possible to set and use the server encoding (charset) US-ASCII.

The other issue regarding the failing IMAP search is handled in the issue #250.

Please note, that this is an issue of Microsoft Exchange servers, not this library. So we are trying to build a workaround to be able to do the searches even although the mail server doesn't support it.