Support newer Exchange servers

[jgoguen]

With WebDAV support gone in newer Exchange versions, EWS support should be added so newer Exchange servers can be used as well.

[cketti]

Exchange ActiveSync might be easier to add.
We have an old branch with initial EAS support: https://github.com/k9mail/k-9/tree/ms-eas

For more information on EWS vs. EAS:
https://msdn.microsoft.com/en-us/library/office/dn144954%28v=exchg.140%29.aspx

[jgoguen]

Fair enough, I suggested EWS since ActiveSync need a patent license. Or is that not true for open source projects?

[ocdtrekkie]

Please ensure this gets fixed before any sort of destructive UI overhaul is committed.

[cketti]

The patent issue requires further research. Where I live there are no software patents, so personally I'm not terribly concerned. But the project is using a lot of services hosted in the US, so this might become an issue.

@ocdtrekkie If you feel the need to provide comments like this, please use the mailing list. Let's use Github issues for technical discussions.

[ocdtrekkie]

@cketti Aside from my general disagreement that Github isn't the place to discuss when PRs should or shouldn't happen and in what order, I would suggest that if you want to direct people to "the mailing list", a link to the mailing list should likely be available somewhere easily identifiable on Github, or linked in your comment. The repo doesn't seem to have a README file, which would be a good place for such links. Also, the organization here on Github has a Google Code link as it's web URL, which may be worth replacing since Google Code is about to go the way of the dodo.

[ashh87]

This:
https://github.com/OfficeDev/ews-java-api

and to a lesser extent perhaps, this:
https://github.com/OfficeDev/Office-365-SDK-for-Android

may be helpful. As far as I can see, MIT and Apache licensing respectively.

[danielcra]

The standard Android Email app seems to handle Exchange servers just fine in cases where K9 fails. Wouldn't it be possible to re-use the code from the AOSP?
http://grepcode.com/file/repository.grepcode.com/java/ext/com.google.android/android-apps/4.4_r1/com/android/exchange/ExchangeService.java

Support of Exchange as alternative protocol to IMAP seems important in order to get push mail to work; for those cases where IMAP idle does not work or is not supported by the server. (I am just now suffering from such a case).

[philipwhiuk]

Just re-using the code is no longer an option. We're too far apart now. But we can and do take bits and pieces where it makes sense.

In fact the ms-eas branch already does this. It has a file called Eas.java which is ported directly (it includes the copyright headers). We probably want to review the entirety of that codebase for improvements at some point and if we had a large team it would be something someone could do.

Anyway, back on topic:

I created a diff between the ms-eas branch and the last point it merged from master, to work out the changes that were done. That gives you https://github.com/philipwhiuk/k-9/pull/2

4 years of changes is too complex to try and rebase with git rebase. The entire codebase has been re-organised into a library module and an app module for example.

Instead I've started manually rebasing changes and trying to work out what needs to be brought in and what we can do without.

I want to minimise the amount of changes being done in a single PR to reduce risk. That means I want to submit PRs which change common code and other stores before I submit the main PR for EAS.

• Introduce the concept of remote names.
• Refactor the MessagingController

Finally, let's not confuse the issue by discussing EWS. If there is a use case for EWS we can create a new issue for that. But from the overview, EAS would seem to be a better choice for an e-mail client: https://msdn.microsoft.com/en-us/library/office/dn144954(v=exchg.140).aspx

Regarding the patents, both Germany and the UK have seen licensing action related to ActiveSync... This is something they appear to care about and enforce: https://www.microsoft.com/en-us/legal/intellectualproperty/mtl/technologylicensing.aspx

I don't particularly want to get sued, so if it turns out this is something we have to pay to license we may have to drop the feature. I expect either the person who merged the code or the person who uploaded the infringing APK would be liable... If we're lucky it's a percentage of revenue ;)

[philipwhiuk]

I've still not got a response from Microsoft :(

However my work is moving to the Microsoft cloud platform so I will probably have a test platform (and a large incentive to develop this) soon...

[philipwhiuk]

The response from Microsoft is 'sign the NDA'..

Just to summarise the situation:

• Implementation of EAS is available in Android source code under a reasonable copyright licence
• There's a branch of code implementing much of the functionality
• EAS is however protected by patents
• All Western handset manufacturers have licenced the patent for the email app.
• This is one of the main way Microsoft makes money on Android.
• Expecting a free licence for EAS on Android is therefore unlikely - the handset manufacturers might be tempted to drop their apps.
• In any case asking for a patent licence involves signing an NDA saying you won't disclose the licensing terms outside your company.
• This can't really reasonably apply to an open source team, especially one like this where there's no actual incorporated company
• Hence it's legally complicated at best to sign the NDA let alone any subsequent patent license.

I live in the UK. @cketti lives in Germany. In both of these countries EAS-related patent activity has been enforced. As such I can't imagine us being able to merge the code and ship an app based on it.

If individuals want to maintain forks of K-9 that support EAS that's fine but I am sure Microsoft would shut down an unlicensed app.

I'm proposing to close this Won't Fix until the patents expire

[cketti]

Do you have any evidence that EAS-related patents have been enforced in court anywhere in the EU?

[philipwhiuk]

This was discussed on IRC - there is some enforcement of an EAS patent outside the EU, and a license to the patent is referenced in a successful defence against another patent lawsuit

Most hardware manufacturers license to the patents - there's commentary on tech blogs that this makes Microsoft much money. There's also comment that it's part of a broader anti-Apple litigation in some places.

We've not yet decided on a direction for this I think...

[philipwhiuk]

As a note - Office 365 supports IMAP (but not IMAP push). So this has slipped down my priority list (because work email still worked).

[phansson]

Not sure I understand this ticket. It suggests to add support for EWS (Exchange Web Services) but very quickly moves on to discuss various issues around EAS (Exchange Active Sync). The latter is riddled with licensing questions, the former has a liberal license.

As far as I understand the only downside to using EWS instead of EAS is that it is a little heavier on the bandwidth. Still this would be better than the current scenario where K-9 is left practically without support for corporate-type Exchange-based email.

[philipwhiuk]

I don't much mind which way whoever picks this up decides to take it. If we ended up with both in the long run that would not be a bad thing if they were both maintained. But it needs time from a developer who has a motivation to develop it (i.e. has an Exchange set-up that doesn't support IMAP as well).

PS: This is essentially the competitor in the space: https://play.google.com/store/apps/details?id=com.ninefolders.hd3&hl=en

[Massimo-B]

I would very appreciate EWS support in K-9 someday. I left M$ for business completely especially Outlook after I got Evolution with EWS working to browse my Exchange account.
For testing purpose and implementing EWS into K-9 you could also get an office365.com account. There are offers per year or per month, basic "Personal" account is 7 EUR per month, for free for testing. Maybe that is useful.
You can also ask the Evolution devs in #evolution on irc.gimp.org

[philipwhiuk]

Hmm, given Evolution supports EWS we should probably go down that route first. I do have an Office 365 work account.

Of course building a new store in K-9 from scratch will take a decently long time. I likely won't have time to work on this for a fair while.

[smccloud]

Given that the upgrade to Exchange 2016 at work has left the stock Gmail app to not work correctly for me, I would like to start working on this. I'm guessing Ubuntu would be the best platform to work on (although I only have Windows at my disposal right now due to work requirements).

[ghost]

How about re-using DavMail's code for EWS?
https://sourceforge.net/p/davmail/news/2016/11/davmail-473-released/

[Frankkkkk]

Gnome Evolution supports quite well exchange > 2010 (email, tasks, calendar, …). Maybe some code could be taken from here ?

[derekatkins]

Actually, I suspect K-9 would be better off starting from https://github.com/alipov/ews-android-api which is a fork and port to Android of https://github.com/OfficeDev/ews-java-api -- I suspect this would be much easier to use (and start with) than the C code of EVO.
The ews-android-api is released under the MIT License, as is its upstream.

[philipwhiuk]

I started looking at this: https://github.com/k9mail/k-9/tree/ews

I've encountered OfficeDev/ews-java-api#353 while testing locally. I'm probably going to have to fork the repo, then fork the android repo which submodules the main repo, then use that...

[mh166]

Just curious: what's the current state? The last few comments do really sound promising. 😃

I know that you guys are devoting your free-time to this and we better be patient. I'm very thankful for your hard work! I just want to kindly ask wether there's any further progress as this has become a necessary feature for me as well.

(I really don't want to have that MS Outlook app sitting around on my phone, if possible... 😅)

[philipwhiuk]

I spent a while trying to work around the bug I mentioned above which kind of stymied my progress. I think you have to replace the character stream reader and filter out the invalid characters which is a horribly involved bit of work. It's kind of lame that the repository doesn't get more support from Microsoft (but not that surprising given they really want you to use EAS).

[AlexisJAnderson]

@philipwhiuk Did you have any more success since December ?

[ro0ter]

Is this feature aborted? I am really interested in having k-9 work with my company`s email... they will all switch to m$...

[ilikenwf]

Could we pull in the EWS for Android code to make this happen?

https://github.com/faisons/EWS-For-Android

[fodinabor]

Some nice update..
EWS for Exchange Online is deprecated so (will be turned off by Oct. 2020).. to support Exchange Online & On-Premise either EAS has to be implemented or two new store providers: EWS & MS Graph

[fodinabor]

Reference:
https://blogs.technet.microsoft.com/exchange/2018/07/03/upcoming-changes-to-exchange-web-services-ews-api-for-office-365/

[derekatkins]

fodinabor <notifications@github.com> writes:

Some nice update.. EWS for Exchange Online is deprecated so (will be turned off by Oct. 2020).. to support Exchange Online & On-Premise either EAS has to be implemented or two new store providers: EWS & MS Graph

Actually, you did not properly read the reference. Yes, EWS is deprecated, but it will continue to be run in production and get security updates. It's only BASIC AUTHENTICATION that is being turned off by October 2020 (in lieu of OAuth tokens). They specifically say in their FAQ that if your app is using OAuth then it will continue to work after October 2020.

[fodinabor]

yeah, sorry.. I remember .. but if I'm right, then OAuth is not supported for on-premise?
So it's just two different auth providers to use..
But yeah.. still EWS is deprecated... and as far as I know MS pushing the Graph API really hard (had some discussions about Graph with MS employees), I could imagine, the next step will be disabling EWS for online totally.. but yeah - we'll have some time until then ;-)

so.. any way one could get the begun work fixed up and working? Or should one start from the base again?

[pingu8007]

There are lots of company still using old-school Exchange server and EWS still an option for they. It would be better to put this into another issue, i.g. OAuth2 support for Outlook, or MS Graph support for Office 365. BTW, AFAIK, OAuth2 support for Gmail still in a very early stage, right?

…

[fodinabor]

afaik /e/'s fork implemented this, leading to being tested by some users right now: https://gitlab.e.foundation/e/apps/Mail/

[fodinabor]

https://azure.microsoft.com/en-us/blog/microsoft-joins-open-invention-network-to-help-protect-linux-and-open-source/

guess it will need some more research... but... possibly we could support active sync now?

[chromy234test]

Hi , Does k9 have any branch that has working activesync code ?

[yoshimo]

afaik /e/'s fork implemented this, leading to being tested by some users right now: https://gitlab.e.foundation/e/apps/Mail/

Which commit would that be? Active sync and eas as search parameters do not yield useful results.

[fodinabor]

afaik /e/'s fork implemented this, leading to being tested by some users right now: https://gitlab.e.foundation/e/apps/Mail/

Which commit would that be? Active sync and eas as search parameters do not yield useful results.

I meant OAuth2

[wiomoc]

@chromy234test
There is an old branch with EAS support:
https://github.com/k9mail/k-9/blob/ms-eas/src/com/fsck/k9/mail/store/EasStore.java

I'm also currently playing a little bit with this:
https://github.com/wiomoc/k-9/tree/ExchangeASBackend

[cketti]

For a contract I integrated the EAS implementation from AOSP into the K-9 code base. Parts of it are available under the Apache 2.0 license here: https://github.com/cketti/k-9/tree/eas_backend

In practice we had lots of issues with EAS. That, and the fact that EAS is not really an open standard made me decide not to include it in K-9 Mail.
I'm not generally opposed to adding support for semi-open protocols (although, in hindsight, EWS is probably a better choice for Exchange servers). But currently there are a lot of other issues in K-9 Mail that need to be addressed before I want to start a project like this.
I don't see this changing anytime soon. So I'm closing this issue in the hope that nobody else will start to do a substantial amount of work only to be told it won't get merged afterwards.