Return Path detection for "system" domain doesn't work

[anhofmann]

when I create new domains in postal the DNS check detects if there is a return path. This doesn't work for the domain that postal has in the dns settings of the postal.yml. Below are the DNS query answers (in redacted form) for different domains.

This is a return path that postal detects:

;QUESTION
psrp.someotherdomain.com. IN MX
;ANSWER
psrp.someotherdomain.com. 299 IN CNAME rp.postal.service.systemdomain.com.
rp.postal.service.systemdomain.com. 299 IN MX 10 mx.postal.service.systemdomain.com.

This is a return path that postal doesn't detect:

;QUESTION
psrp.systemdomain.com. IN MX
;ANSWER
psrp.systemdomain.com. 299 IN CNAME rp.postal.service.systemdomain.com.
rp.postal.service.systemdomain.com. 299 IN MX 10 mx.postal.service.systemdomain.com.

This is the dns config in my postal.yml:

mx_records:
- mx.postal.service.systemdomain.com
smtp_server_hostname: postal.service.systemdomain.com
spf_include: spf.postal.service.systemdomain.com
return_path: rp.postal.service.systemdomain.com
route_domain: routes.postal.service.systemdomain.com
track_domain: track.postal.service.systemdomain.com
helo_hostname: # By default, this will be the same as the smtp_server_hostname
dkim_identifier: postal
domain_verify_prefix: postal-verification
custom_return_path_prefix: psrp

my postal is on commit 9a76538f

Is this behavior in postal a problem, or is this just a minor bug? Did I made something wrong in the postal.yml?

[ro78]

My bug #149 might be related, as I'm also trying to get return-path working with the same domain than in my config.

[anhofmann]

After reading #149 I made some tests:

• I cannot send emails to something@rp.postal.service.systemdomain.com (Error 530 from Postal like in RP record - route needed? #149)
• I can send email to something@psrp.someotherdomain.com
• I can send mail to something@psrp.systemdomain.com but Postal puts the mail into a mailserver that has the domain postalintern.systemdomain.com which is wrong.

So I assume that postal should really use another domain for internal use because everything else just creates a lot of confusion.

BTW: my domain systemdomain.com is not tied to a mailserver but is configured on the organisation level. So maybe this is a problem, I don't know. Generally I find it confusing that I'm able to configure domains on two different levels (organisation and mail server).

[ro78]

I tried to send an email to something@psrp.borezo.info and it works as an incoming message recognized as hard bounce but can't link to a message. But Postal doesn't use psrp record in return path.

This message was a bounce but we couldn't link it with any outgoing message and there was no route for it.

[adamcooke]

Messages sent to a return path can only be linked back to an original message if the bounce includes the message ID of the original message. Linking actual bounces back to their original message is a little tricky and the current method isn't ideal. An alternative is sending every message from it's own address but this can cause delays with greylisting.

[ro78]

Ok, but why bounces sent to @rp.postal.borezo.info are not handled?
If it should be handled by @psrp.borezo.info, why Postal don't use this domain as returnpath?

[ro78]

Is this situation normal? https://www.dropbox.com/s/7k4i4mchz1ed1t6/2017-05-16_144056.png?dl=0
I don't know what I must do to change this "There is no return path record at psrp.borezo.info" line.

[anhofmann]

rp.postal.borezo.info doesn't point to any of your mail servers in postal. So postal has to throw away mails for this domain. Otherwise postal would be a open relay: https://en.wikipedia.org/wiki/Open_mail_relay
So this is expected and desired behavior. The only open question that remains: can postal detect that a return path is correct when its checking for itself (aka the system domain)?

[ro78]

If Postal uses rp address as Return-Path value, then he must handle bounces received on this domain. It's the expected behavior of email servers. I didn't configured psrp and bounce is handled, so the issue is not about open relay.

[anhofmann]

there is a difference between rp.postal.borezo.info and psrp.borenzo.info and I tried to explain the difference. I'm sorry that I didn't made it understandable and I will try again:
rp.postal.borezo.info and psrp.borenzo.info are domains in the DNS that point to the Server where Postal is running.

In Postal psrp.borenzo.info is a domain that is linked to a mailserver inside of Postal that has routes, rules and so on. Postal knows what to do with mails that use the psrp.borenzo.info domain: mails from this domain are bounce mails that belong to the borenzo.info domain.

The domain rp.postal.borezo.info has no function in Postal, it is not linked to any mailserver in Postal therefore there are no rules, or routes that Postal could use. So Postal has nothing else to do than to throw mails away that use rp.postal.borezo.info as the domain part. This is correct behavior. If you really want to receive mails for this domain, you need to create a mailserver with the domain rp.postal.borezo.info but this would be really confusing for everyone.

[ro78]

Ok i got it.
psrp is automatically configured by postal to handle bounces to it accepts incoming emails, and it's not the case for rp.postal, right?
So, why postal sends emails using rp.postal domain as return path value, where bounces are expected to be sent ?
Should I change the postal config file to ask Postal to use psrp? I just followed the install guide asking to use rp.postal in configuration.

[adamcooke]

@anhofmann So... rp.postal.borezo.info looks like it's configured correctly and psrp.borezo.info is a CNAME to it which is also correct. The check will check that psrp.borezo.info is a CNAME and that it matches the value from the dns.return_path value in your config file.

If the DNS checks fail for psrp.borezo.info, Postal will use whatever is configured in dns.return_path for the return path. The return path is made up from the server's unique token at any return path domain.

I have just spotted a bug though which would cause some issues here. Postal will identify a message as being sent to a return path based solely on the prefix of the domain it is sent to. If the domain starts with dns.custom_return_path_prefix then it will be treated as a return path. The documentation says to use psrp and point it to rp. They should both be psrp. I'm just about to push a fix to this to match the base domain too which should fix this and stop any confusion.

[adamcooke]

I have pushed a fix in 4106ac6. Hopefully this will fix these issues people have been having with return paths.

[adamcooke]

I haven't had any issues with the DNS verification of these records myself. It all works as intended. If you're still having issues with getting the "Return Path" check to go green, we may need to investigate further.

[ro78]

Thank you. Can I update postal upgrade to get this fix? Or I should wait for something else or do something else?

[adamcooke]

You can just upgrade following the instructions. At present, everyone is just using a copy of postal directly from the repository because we haven't released an official v1.0 yet.

[ro78]

I used "postal auto-upgrade" and for the moment my return path setting is still grey.

Config:
dns:

Specifies the DNS record that you have configured. Refer to the documentation at

https://github.com/atech/postal/wiki/Domains-&-DNS-Configuration for further

information about these.

mx_records:
- mx.postal.borezo.info
smtp_server_hostname: postal.borezo.info
spf_include: spf.postal.borezo.info
return_path: rp.postal.borezo.info
route_domain: routes.postal.borezo.info
track_domain: track.postal.borezo.info
dkim_identifier: postal
domain_verify_prefix: postal-verification
custom_return_path_prefix: psrp

[ro78]

(but messages to rp.postal.borezo.info are accepted)

[adamcooke]

It should be psrp.postal.borezo.info not rp.

[ro78]

The domain section still says "There is no return path record at psrp.borezo.info", not "psrp.postal.borezo.info".
And emails sent through Postal still uses "rp.postal.borezo.info" as return-path.

[adamcooke]

They are sending from that default domain (rp.postal.borezo.info) because the return path on the domain hasn't been verified. If it had checked successfully it would start using the correct domain. psrp.borezo.info does appear to be pointed to your CNAME though. Have you pressed retest?

[ro78]

Yes, I used the test button many times. No change.
On postal:

root@postal:~# host psrp.borezo.info
psrp.borezo.info is an alias for rp.postal.borezo.info.
rp.postal.borezo.info has address 149.202.163.26
rp.postal.borezo.info has IPv6 address 2001:41d0:302:1000::3:ce38
rp.postal.borezo.info mail is handled by 10 mx.postal.borezo.info.

[adamcooke]

Well... the Ruby Resolver refuses to see those CNAME records so that's quite puzzling but probably has a quite simple explanation.

Resolv::DNS.new.getresources("psrp.borezo.info", Resolv::DNS::Resource::IN::CNAME)
# => []

[adamcooke]

Actually, even using dig doesn't return a CNAME record.

dig -t cname psrp.borezo.info +trace

[adamcooke]

Are you using some sort of CNAME flattening from Cloudflare which might be causing this?

[ro78]

I'm not using anything particular : https://www.dropbox.com/s/0q41x3bbqjxfh5l/2017-05-16_184945.png?dl=0
I'll contact CloudFlare. Can I bypass this issue by using direct A/MX records instead of CNAME?

[adamcooke]

CloudFlare don't return a CNAME when asked for one by the looks of it. If you do a request for the A or MX you'll get the A/MX along with the CNAME record. For example:

dig -t a psrp.borezo.info +trace
# psrp.borezo.info.	120	IN	CNAME	rp.postal.borezo.info.
# rp.postal.borezo.info.	120	IN	A	149.202.163.26

Postal, at present anyway, checks that it can get a CNAME record back when making a query for the CNAME. We might be able to change that to check for the associated MX record instead. For now, there is no easy workaround I'm afraid.

[adamcooke]

This might be useful: https://support.cloudflare.com/hc/en-us/articles/200169056-CNAME-Flattening-RFC-compliant-support-for-CNAME-at-the-root

[ro78]

Thank you, I was not aware of it.
Can I publish the same A/MX record than rp.postal as psrp to get it working? Or Postal will only check CNAME?

[ro78]

They answered it's a propagation issue (https://www.whatsmydns.net/#CNAME/psrp.borezo.info), I need to wait.

[adamcooke]

Postal will only check CNAMEs. It shouldn't really be a propagation issue though. Postal queries your domain's own nameservers so unless Cloudflare's own internal propagation is very slow (their website claims < 5 seconds) it wouldn't have thought it'll be that.

That website demonstrates the same thing as my dig examples. Cloudflare's nameserver aren't returning CNAME records when the queried for a CNAME.

[ro78]

I had 6 servers returning a CNAME when they answered and now it's 0. I think there is an issue on their side. When fixed, I'll check if postal still have the issue.

[ro78]

Just to be clear. What I understand and saw on how CloudFlare works.
If you have a "something CNAME somethingelse", and "somethingelse A X.X.X.X", CloudFlare returns "A X.X.X.X" to the CNAME instead of the cname itself.
So, in my case, I should have something in answer to the CNAME, even if it's a A. Here, I get nothing.

[ro78]

Ok I'm wrong. It returns something when doing a A request but not CNAME as explained in the link you found. I need to wait for you to fix the code to allow a A check instead of CNAME.

[adamcooke]

I don't have any urgent plans to change the check I'm afraid. We'll look into the appropriate RFCs to see what the DNS servers should be returning and whether we need to make any changes.

[catphish]

CloudFlare have confirmed that this is a bug in their DNS servers.

Thank you for contacting Cloudflare Support. I am sorry to hear that you are experiencing some difficulties here. Since you are not the zone owner for borezo.info I cannot share any details with you however I can confirm that this is a bug and that our engineering team is currently looking into the issue.
We will be sure to let you know as soon as we have any updates.

[ro78]

Ok. I'm waiting for a CloudFlare update, they confirmed it's a bug on their side. But while I read their FAQ about flattening, I'm not sure it's a bug.
Anyway, since your last submit @adamcooke , bounces sent by email after delivery are now correctly handled and recognized as bounces, and linked to the correct message.

[adamcooke]

Everything should work, it's just the verification that'll be broken. Hopefully CloudFlare will fix that soon.

[catphish]

CloudFlare have now fixed this issue.

[ro78]

Issue where CNAME record was not returned has been fixed by CloudFlare today.