OpenSMTPd should accept alias rules in relay declarations

[wxcafe]

It is sometimes useful to be able to use alias rules in relay declarations, for example if one tries to redirect mail from one user that doesn't exist on the system to another one, using a spam check (like amavisd). As of now, what happens is that the email arrives on the system, and OpenSMTPd immediately rejects it as the user doesn't exist, even though there's an alias on this user, while with a direct delivery it would work (since one can apply aliases to deliver declarations)

[poolpOrg]

Doesn't this generate a bounce sent back to the original sender ?

[wxcafe]

Yep, there is a bounce.
It's not the desired behavior though.

[wxcafe]

Pushing this up. The desired behavior is not to send a bounce telling the sender that the recipient doesn't exist, it should be evaluating the aliases and then forwarding to the (aliased) recipient address.

[poolpOrg]

No need to push this up, we have not forgotten about this but it is a VERY invasive change with profound implications with regard to the ruleset matching, the aliases expansion and the entire code path taken by an accepted envelope.

It'll be worked out but it will take some time.

[ArchangeGabriel]

So, that’s probably what I need.

I’m currently trying to redirect mails received for pseudoml@mydomain.org to a list of users from other domains (e.g. user1@otherdomain1.org,user2@otherdomain2.org).

My main mail is delivered with this rule:

accept from any for domain <vdoms> virtual <vusers> deliver to lmtp "/var/run/dovecot/lmtp" rcpt-to

So, as mydomain.org is in , I’ve added a rule above for my redirection:

accept from any for domain <vdoms> recipient <forward_list> virtual <forwards> forward-only

Where forward_list contains currently pseudoml@mydomain.org and forwards for now only pseudoml@mydomain.org user1@otherdomain1.org,user2@otherdomain2.org.

This does work, but breaks DKIM/SPF. I’ve tried to use relay as to solve this, but to no avail because I can’t use alias/virtual in a relay rule (also this would probably not scale well depending on what you intend to put as the as). I’ve even tried torelay as to opensmtpd itself on a specific port without the virtual parameter, and then use a tag to match the virtual+forward-only rule, but it doesn’t work (same 550 error).

So, unless I’ve missed a much more simpler config to setup a pseudo-ml, I’m gonna need this too. ;)

[PhLe]

I have the same goals: DKIM signing only for mails originating from @mydomain.org, but not for mails that only pass through my server for taking profit of redirections to a list of mail addresses in various domains. Furthermore, I have to cope with a cornercase: mail originated from @laposte.net and redirected to @laposte.net are rejected due to strict SPF restrictions by their SMTP server. I have this setting

accept from any for domain mydomain.org virtual <val>
accept tagged DKIM_OUT for any relay pki ... hostname ....
accept sender "@laposte.net" for any recipient "@laposte.net" relay as "@mydomain.org"
accept sender "@mydomain.org" for any relay via smtp://127.0.0.1:10027 # send to dkimproxy_out

All work as expected, only local issued mail are DKIM signed. However, due to severe troubles within laposte.net, I have not yet been able to test if they are now happy with this new setting. I had to test with *another mail provider and I could verify that relay as "@mydomain.org" works as expected.

[poolpOrg]

This is going to be fixed with #772

I'm closing this ticket because #772 now has a reference to it and we can discuss this in the commands of that new ticket.

[wagnerflo]

Sorry for resurrecting an old issue, but this feature is exactly what I'm looking for. Neither reading smptd.conf(5) nor #772 nor trying different setups, I can find a way to do this. Any pointers?

[hmht]

isn't it just match for rcpt-to <these_need_forwarding> action "spamcheckandforward"

[wagnerflo]

isn't it just match for rcpt-to <these_need_forwarding> action "spamcheckandforward"

Is this an answer to my question? If so then, I'm sorry to say, it's a bit too cryptic for me to understand. What's "spamcheckandforward"? Why spam? Where do I specify the alias file to apply before forwarding?

[hmht]

the original issue mentioned it wanted to do a spam check, so that's what I used as example action name.

not sure what your use case is, but:

1. define your forward action with an action declaration. ex: action "a" relay host "smtp+notls://10.8.0.2"
2. declare a table (man 5 table) ex: table "aa" "file:/aaa/aaaa"
3. match the rcpt-tos in the table to use the action. ex: match for rcpt-to <aa> action "a"

the example config shows some of this already, so it'll make a good starting point.

there's also an alias you can stick onto the relay action to do aliases expansion, all explained in the manpage.

[wagnerflo]

the original issue mentioned it wanted to do a spam check, so that's what I used as example action name.

Ah right. Sorry I missed that.

not sure what your use case is, but:

1. define your forward action with an action declaration. ex: action "a" relay host "smtp+notls://10.8.0.2"
2. declare a table (man 5 table) ex: table "aa" "file:/aaa/aaaa"
3. match the rcpt-tos in the table to use the action. ex: match for rcpt-to <aa> action "a"

I dont' think you can use alias/mapping tables in match ... for rcpt-to ... (or any other match tests probably) only lookup/list tables, can you? My tests, modeled after your suggestion, support that:

listen on localhost
table "relay_aliases" { test=florian@wagner-flo.net }
action "relay" relay
match from local for rcpt-to <relay_aliases> action "relay"

But smtpd doesn't like it...

~ smtpd -d -f /tmp/smtpd.conf
/tmp/smtpd.conf:4: table "relay_aliases" may not be used for for lookups
warn: no rules, nothing to do

Using just table "relay_aliases" { test } works, of course. But then it will not alias anything, obviously.

the example config shows some of this already, so it'll make a good starting point.

I seem to be missing, where exactly it "shows some of this". I can only find an alias table that is used for local delivery but nowhere else.

OpenSMTPD/usr.sbin/smtpd/smtpd.conf

Line 15 in 5800ede

action "local" maildir alias <aliases>

there's also an alias you can stick onto the relay action to do aliases expansion, all explained in the manpage.

Quoting from the manpage you mentioned (important emphasis added by me):

The local delivery methods support additional options:
alias < table >
[...]

Quick testing confirms this

listen on localhost
table "relay_aliases" { test=florian@wagner-flo.net }
action "relay" relay alias <relay_aliases>
match from local for any action "relay"

No dice...

~ smtpd -d -f /tmp/smtpd.conf
/tmp/smtpd.conf:3: syntax error

It will not let you apply a table "onto the relay action to do aliases expansion". Remove the alias <relay_aliases> from the end of line 3 and smtpd starts, but again will not alias anything obviously.

This is all on OpenSMTPD 6.8.0p2 if that makes any difference. But judging form that version having been released 4 month ago and this feature supposedly having being integrated nearly 3 years ago I don't think this is a version problem.

[wagnerflo]

I went a step further, cloned the OpenSMTP repository and modified the configuration parser. Copy these lines

https://github.com/openbsd/src/blob/85f491dae6d227f10e75412df0c572c8e5b04442/usr.sbin/smtpd/parse.y#L611-L631

paste them as a |-branch of dispatcher_remote_option here

https://github.com/openbsd/src/blob/85f491dae6d227f10e75412df0c572c8e5b04442/usr.sbin/smtpd/parse.y#L968

That'll make smtpd parse my second example, start and not even crash (I'm a bit astonished). But it'll still not alias before relaying:

2c4a7e03d912ba55 smtp connected address=[::1] host=localhost.localdomain
2c4a7e03d912ba55 smtp message msgid=7e6ddd5c size=320 nrcpt=1 proto=SMTP
2c4a7e03d912ba55 smtp envelope evpid=7e6ddd5c8432e886 from=<REDACTED> to=<test@REDACTED>
2c4a7e073af57334 mta connecting address=smtp://REDACTED:25 host=REDACTED
2c4a7e073af57334 mta connected
2c4a7e073af57334 mta delivery evpid=7e6ddd5c8432e886 from=<REDACTED> to=<test@REDACTED> rcpt=<-> source="REDACTED" relay="REDACTED (REDACTED)" delay=9s result="Ok" stat="250 2.0.0 13EAGO3Q047826 Message accepted for delivery"
2c4a7e073af57334 mta disconnected reason=quit messages=1

[wagnerflo]

Oh right making the parser just assign the alias table to dsp->u.local will not make it magically work for remote delivery. But otherwise

../../usr.sbin/smtpd/parse.y: In function 'yyparse':
../../usr.sbin/smtpd/parse.y:957:19: error: 'struct dispatcher_remote' has no member named 'table_alias'
  957 |  if (dsp->u.remote.table_alias) {
      |                   ^
../../usr.sbin/smtpd/parse.y:962:19: error: 'struct dispatcher_remote' has no member named 'table_virtual'
  962 |  if (dsp->u.remote.table_virtual) {
      |                   ^
../../usr.sbin/smtpd/parse.y:973:15: error: 'struct dispatcher_remote' has no member named 'table_alias'
  973 |  dsp->u.remote.table_alias = strdup(t->t_name);
      |               ^

So... am I allowed to go so far to say that there must have been an error and this feature is actually not implemented? Reopen this issue? Create a new one?

[ArchangeGabriel]

Yeah, I don’t see how this has been fixed: from the documentation, virtual and alias are only under the section The local delivery methods support additional options:.

So my old rule, that was:

accept from any for domain mydomain.org recipient pseudoml@mydomain.org virtual <forwards> forward-only

where forwards was a table like pseudoml@mydomain.org user1@otherdomain1.org,user2@otherdomain2.org seems not even possible nowadays. The issue before was we could not use relay as in such a rule, but now we have SRS and I would need something like

action "aliasedrelay" relay helo mydomain.org srs virtual <forwards> forward-only
match from any for rcpt-to pseudoml@mydomain.org action "aliasedrelay"

But virtual would now even be illegal in this context?

[ArchangeGabriel]

Also I’m wondering how forward-only works in the new system if we cannot expand when relaying…

[poolpOrg]

Chiming in to give some insight:

Sorry for resurrecting an old issue, but this feature is exactly what I'm looking for. Neither reading smptd.conf(5) nor #772 nor trying different setups, I can find a way to do this. Any pointers?

It is not the right mechanism for doing that which is why this issue was closed.

Aliases and virtuals are local mechanism intended to convert a session RCPT address into a local user account, with a uid, a gid, a homedir, and so on. It can end up resolving an address into another address but this is a special case of the mechanism which always tries to find a local user account, does cross rules expansions, converts an address to another one just so it can match it against the configuration file again to find a local user, etc...

For relay rules, this doesn't make sense and may have undesireable side effects, you really want a rewrite mechanism that can ONLY convert an e-mail address into another one, not trying to resolve a local user, not trying to do cross-rules things, just mapping foo@bar.org to bar@baz.org and be done.

This mechanism doesn't exist in the ruleset yet, though I proposed it a while back.

It is possible to do it through filters or built-in filters since they have a "rewrite" action which allows replacing a mail-from or rcpt-to parameter with a new value, it's just not doable from the ruleset itself at this point.

Oh right making the parser just assign the alias table to dsp->u.local will not make it magically work for remote delivery. But otherwise

[...]

So... am I allowed to go so far to say that there must have been an error and this feature is actually not implemented? Reopen this issue? Create a new one?

Wrong mechanism which is why it can't work and even if you managed to plug it correctly, it would hit other issues because the aliasing does stuff specifically for local systems, which may not work on a relay rule.

Everything needed to support the rewrite already exists, if you open an issue it might just be implemented shortly.

Yeah, I don’t see how this has been fixed: from the documentation, virtual and alias are only under the section The local delivery methods support additional options:.

virtual and alias have always been a local only mechanism, the old configuration syntax was ambiguous and accepted them n relay rules ... without actually doing the aliasing, it used the key to similarly to what rcpt-to option does today, with no ambiguity anymore.

So my old rule, that was:

accept from any for domain mydomain.org recipient pseudoml@mydomain.org virtual <forwards> forward-only

where forwards was a table like pseudoml@mydomain.org user1@otherdomain1.org,user2@otherdomain2.org seems not even possible nowadays. The issue before was we could not use relay as in such a rule, but now we have SRS and I would need something like

forward-only is a local rule, its purpose is for systems that have local users but want them to have their mails redirected elsewhere (ie: poolp.org may host local users but want them to redirect their mail through .forward files).

action "aliasedrelay" relay helo mydomain.org srs virtual <forwards> forward-only
match from any for rcpt-to pseudoml@mydomain.org action "aliasedrelay"

But virtual would now even be illegal in this context?

Only because your action is invalid, the new syntax does not change behaviours:

forward-only is still a local action that makes no sense in a relay rule:

action "forward_only" forward-only virtual <forwards>

What is illegal is using virtual with a "relay" action because as stated above, aliases and virtuals are a local mechanism doing local operations which make no sense for relay rules.

A different mechanism is needed that only does address rewrites without looking up local users and such.

Also I’m wondering how forward-only works in the new system if we cannot expand when relaying…

the same way it has always worked:

forward-only matches a local rule, does aliases expanding which results in remote addresses, which are all matched individually against the ruleset to find a matching relay rule.

[poolpOrg]

To better highlight ONE of the many issues of using aliases in relay rules:

table aliases { foo@bar.org = gilles }

action "bleh" relay aliases <aliases>
match from any for any action "bleh'

What are your expecting the relay action to do with the local user "gilles" here ?
Any decision you take for this case leads to many other issues which do not have solutions because ... a local username does not make sense in a relay rule and therefore there can't be a sensible action to do.

[ArchangeGabriel]

So my old rule, that was:

accept from any for domain mydomain.org recipient pseudoml@mydomain.org virtual <forwards> forward-only

where forwards was a table like pseudoml@mydomain.org user1@otherdomain1.org,user2@otherdomain2.org seems not even possible nowadays. The issue before was we could not use relay as in such a rule, but now we have SRS and I would need something like

forward-only is a local rule, its purpose is for systems that have local users but want them to have their mails redirected elsewhere (ie: poolp.org may host local users but want them to redirect their mail through .forward files).

Right, makes sense and I now remember how this rule worked exactly.

action "aliasedrelay" relay helo mydomain.org srs virtual <forwards> forward-only
match from any for rcpt-to pseudoml@mydomain.org action "aliasedrelay"

But virtual would now even be illegal in this context?

Only because your action is invalid, the new syntax does not change behaviours:

Right, for one moment I thought I had a relay keyword in the old one but actually not, relaying can be part of “local delivery”, that’s what I failed to remember.

forward-only is still a local action that makes no sense in a relay rule:

action "forward_only" forward-only virtual <forwards>

What is illegal is using virtual with a "relay" action because as stated above, aliases and virtuals are a local mechanism doing local operations which make no sense for relay rules.

A different mechanism is needed that only does address rewrites without looking up local users and such.

Also I’m wondering how forward-only works in the new system if we cannot expand when relaying…

the same way it has always worked:

forward-only matches a local rule, does aliases expanding which results in remote addresses, which are all matched individually against the ruleset to find a matching relay rule.

Right, so of course the translation of my old rule now is:

action "aliasedrelay" forward-only virtual <forwards>
match from any for rcpt-to pseudoml@mydomain.org action "aliasedrelay"

with forwards still being a table with pseudoml@mydomain.org user1@otherdomain1.org,user2@otherdomain2.org as content. Correct? And that upon parsing this file, smtpd will actually relay to user1@otherdomain1.org and user2@otherdomain2.org, and I should use something like this:

action "relaysrs" relay srs
match from local for any action "relaysrs"

so that those emails are relayed using SRS (or, for the old relay as version, using mail-from pseudoml@mydomain.org instead of SRS, but I think SRS is the proper way now here).

Did I understood things correctly? If so this seems actually solved to me.

[wagnerflo]

This mechanism doesn't exist in the ruleset yet, though I proposed it a while back.

It is possible to do it through filters or built-in filters since they have a "rewrite" action which allows replacing a mail-from or rcpt-to parameter with a new value, it's just not doable from the ruleset itself at this point.

Ah right. Someone pointed that out on IRC, I played around a bit with it, but didn't manage to expand a single address to multiple ones. Didn't continue looking there and came here...

Everything needed to support the rewrite already exists, if you open an issue it might just be implemented shortly.

Since I didn't manage to get it working with a custom filter, would "implemented shortly" be still on the table if I wanted rewrite to turn a match into multiple recipients?

To better highlight ONE of the many issues of using aliases in relay rules:

table aliases { foo@bar.org = gilles }

action "bleh" relay aliases <aliases>
match from any for any action "bleh'

What are your expecting the relay action to do with the local user "gilles" here ?

I'd expect this to apply aliasing/rewriting recursively and if no more matches are found to bounce the mail. If on the other hand its

action "bleh" relay aliases <aliases> host smtp://...

I'd hope that forwarding would be tried in hopes of the smarthost actually accepting that user. Or at least I'd like to be able to configure my MTA to work that way.

Maybe that'd be out of scope of OpenSMTPD and I'm better served with some other piece of software?

[ArchangeGabriel]

@wagnerflo Did you try my proposition? If yes and there is an issue, please tell me. If not, please do. If I did not understand what you are trying to do exactly, please explain your exact needs/setup. ;)