smtp forward dest split routing

[msimerson]

This is progress on #1472 and it improves upon a longstanding mail routing issue that affects users of per-domain mail forwarding.

I have a bunch of [domain.com] blocks in smtp_forward.ini and most domains have differing forward hosts. When a transaction has recipients in domains with different forward hosts, then those subsequent recipients are sent to the wrong destination host. The one_message_per_rcpt option in smtp_forward just does an RSET between transactions to the forward host of the first recipient.

The previous workaround was to forward transactions with multiple destination hosts to the default forward host and let the smart host sort it out. See #573 and #671. That works reasonably well the vast majority of the time. The edge case is split transactions with undeliverable recipients. They end up being queued and some of those recipients are ultimately undeliverable and bounce.

This PR removes the "forward multiple destination transactions to smarthost and let it sort it out" behavior and adds recipient validation to smtp_forward. Example:

 -> MAIL FROM:<matt@****.tnpi.net>
<-  250 sender <matt@****.tnpi.net> OK
 -> RCPT TO:<matt@tnpi.net>
<-  250 recipient <matt@tnpi.net> OK
 -> RCPT TO:<ryan@example.net>
<** 450 [7E794F58-35C@haraka] Queue error, try again soon
 -> DATA
<-  354 go ahead, make my day

Within a transaction, it defers subsequent recipients if they have a different forward host than the first recipient. This is possible because smtp_forward.ini already has the forward host information. Other recipient/queue plugins can cooperate by setting txn.notes.queue.

Changes proposed in this pull request:

• smtp_forward: add check_sender() & check_recipient()
  • disabled by default
  • when enabled, if a domain has a definition block in smtp_forward.ini:
    • validate sender & recipients
      • for myself, the host_list plugin is no longer necessary
    • for relaying clients, set queue to txn.notes.queue:outbound
    • for inbound recipients, set txn.notes.queue=smtp_forward:hostname
    • if subsequent recipients on a transaction have differing forward routes:
      • DENYSOFT
      • it's the only safe/sane way to split the transaction
  • remove the "forward to default host when conflicting forward hosts" behavior added in #671
• if txn.notes.queue is defined but doesn't start with smtp_forward, then smtp_forward doesn't queue.
• moved the AUTH block out of hook_queue to its own function
• add get_mx hook, so that recipients in connections where relaying=true that are destined to a local domain (defined in smtp_forward.ini and for which we are the MX for) can be queued to outbound and still be deliverable.
• qmail_deliverable
  • config, add manually specified queue option
  • like smtp_forward, qmd already supports domain routing
    • specify a queue (for myself: smtp_forward), supporting the smtp_forward queue splitting
  • moved qmd options into get_qmd_response (was plugin global)

Checklist:

• docs updated
• tests updated

[codecov-io]

Codecov Report

Merging #1847 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1847   +/-   ##
=======================================
  Coverage   46.23%   46.23%           
=======================================
  Files          22       22           
  Lines        5864     5864           
  Branches     1474     1474           
=======================================
  Hits         2711     2711           
  Misses       3153     3153

Impacted Files	Coverage Δ
outbound.js	13.45% <ø> (ø)	✅
smtp_client.js	6.8% <0%> (-0.03%)	❌

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51589b3...d6dcf10. Read the comment docs.

[codecov-io]

Codecov Report

Merging #1847 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #1847   +/-   ##
=======================================
  Coverage   46.23%   46.23%           
=======================================
  Files          22       22           
  Lines        5864     5864           
  Branches     1474     1474           
=======================================
  Hits         2711     2711           
  Misses       3153     3153

Impacted Files	Coverage Δ
outbound.js	13.45% <ø> (ø)	✅
smtp_client.js	6.8% <0%> (-0.03%)	❌

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51589b3...d6dcf10. Read the comment docs.