Add hooks for email poller plugins

[AlessioC31]

Hello,

This PR is child of #20445: while it would be better to support OAUTH2 with pop3, that PR can't be merged until pop3 merges the support for XOAUTH (ruby/net-pop#16).

For this reason, I would like to add in discourse the support for mail pollers plugins. Doing so, it would be possible to write a plugin which then uses other ways (microsoft graph sdk for example) to poll emails from a mailbox.

The idea is that a plugin would define a class which inherits from Email::Poller and defines a poll_mailbox static method which returns an array of strings. Then the plugin could call register_mail_poller(<class_name>) to have it registered. All the configuration (oauth2 tokens, email, etc) could be managed by sitesettings defined in the plugin.

Please let me know if this solution works for you (I tried to have a look at other "plugin hooks" and based my solution on those) and if you think I should change something, thanks!

[AlessioC31]

I've slightly changed the design of the API. Now discourse core gives a callback to the plugin to call each time to process an inbound mail. I thought this was the best idea to easily give the control over the lifecycle of an email to the plugin.

[SamSaffron]

the biggest thing missing here is some tests, we are adding an extensibility point here, if we don't cover it with tests we are more likely to mistakenly remove or break it in future.

[AlessioC31]

the biggest thing missing here is some tests, we are adding an extensibility point here, if we don't cover it with tests we are more likely to mistakenly remove or break it in future.

I've added a couple of tests, let me know if you think we should add more, thanks!

[CLAassistant]

All committers have signed the CLA.

[AlessioC31]

I saw that the PR failed, the error didn't look like it was something related to my changes. I rebased again and pushed

[AlessioC31]

Hello, I've added the fact that now the validator for reply_by_email_enabled checkes also custom mail pollers.
So that it's possible to enable the reply by email feature while having manual polling / pop3 disabled but at least one custom mail poller

[trobiyo]

With the following configuration, we've been able to reply by mail (private message included) and send an email to a category with successful results:

Settings (core)

• reply by mail enabled: checked
• reply by email address: account+%{reply_key}@example.com
• email_in: checked
• manual polling enabled: unchecked
• pop3 polling enabled: unchecked

Settings (plugin)

• msgraph polling enabled: checked
• msgraph polling mailbox: account@example.com
• msgraph polling client id: <uuid>
• msgraph polling tenant id: <uuid>
• msgraph polling oauth2 refresh token: <obtained token as per instructions in the plugin>
• msgraph polling login endpoint: https://login.microsoftonline.com
• msgraph polling graph endpoint: https://graph.microsoft.com/v1.0

Emails are automatically auto-purged from Exchange Online, which is great.

Unless Discourse Team gives us some more feedback, this is ready.

[discoursebot]

This pull request has been mentioned on Discourse Meta. There might be relevant details there:

https://meta.discourse.org/t/microsoft-graph-mail-poller/268611/1

[nattsw]

Just need a small test in https://github.com/discourse/discourse/blob/main/spec/lib/discourse_plugin_registry_spec.rb. Ideally we want to test the empty set.

If you could add an explicit empty test in poll_mailbox_spec.rb describe "poller plugin" that would be good too - though I get that it would be covered implicitly in the tests above.

[AlessioC31]

Thanks, I added the test for the empty set!

[nattsw]

Heya @AlessioC31, took this for a spin locally! This generally looks very good, thanks. I'll be happy to approve this with one small addition. Will keep this PR open 👀 so we can get this in this week.

[nattsw]

Very funny/shady but the test failure is mailer related but has got nothing to do with your change. Will re-run

  1) UserNotifications.user_replied number of tags shown in subject line max_tags_per_email_subject siteSetting disabled should match max_tags_per_topic
     Failure/Error: expect(mail.subject).to match(/Taggie/)
     
       expected "[Discourse] [India] Teggo Taggo Super cool topic" to match /Taggie/
       Diff:
       @@ -1 +1 @@
       -/Taggie/
       +"[Discourse] [India] Teggo Taggo Super cool topic"