lmtp delivery to subfolder fails

[brong]

From: Alberto Sentieri
Bugzilla-Id: 3163
Version: 2.4.8
Owner: Bron Gondwana

[brong]

From: Alberto Sentieri

I am trying to deliver emails to subfolders using lmtp and getting strange
errors.

A regular delivery works fine, as could be seen on the attempt below:

$ telnet 192.168.44.70 24
Trying 192.168.44.70...
Connected to 192.168.44.70 (192.168.44.70).
Escape character is '^]'.
220 localhost.localdomain LMTP Cyrus v2.3.7-Invoca-RPM-2.3.7-2.el5 ready
lhlo truco
250-localhost.localdomain
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-SIZE
250-AUTH EXTERNAL
250 IGNOREQUOTA
mail from:<>
250 2.1.0 ok
rcpt to:<abcd>
250 2.1.5 ok
data
354 go ahead
subject: test 1

testing
.
quit250 2.1.5 Ok
221 2.0.0 bye
Connection closed by foreign host.

A subfolder delivery fails:

$ telnet 192.168.44.70 24
Trying 192.168.44.70...
Connected to 192.168.44.70 (192.168.44.70).
Escape character is '^]'.
220 localhost.localdomain LMTP Cyrus v2.3.7-Invoca-RPM-2.3.7-2.el5 ready
lhlo truco
250-localhost.localdomain
250-8BITMIME
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-SIZE
250-AUTH EXTERNAL
250 IGNOREQUOTA
mail from:<>
250 2.1.0 ok
rcpt to:<abcd.test1>
250 2.1.5 ok
data
354 go ahead
subject: test 2

testing ...
.
554 5.0.0 Unexpected internal error
quit
221 2.0.0 bye
Connection closed by foreign host.

The log file contains this information:

File /var/log/maillog after a unsucessful delivery:

Apr 6 12:05:46 localhost master[21710]: about to exec
/usr/lib/cyrus-imapd/lmtpd
Apr 6 12:05:46 localhost lmtp[21710]: executed
Apr 6 12:05:46 localhost lmtp[21710]: accepted connection
Apr 6 12:05:46 localhost lmtp[21710]: connection from bugs.mail0.xxx
[192.168.44.11] preauth'd as postman
Apr 6 12:06:40 localhost lmtp[21710]: IOERROR: fstating sieve script
/var/lib/imap/sieve/a/abcd.test1/defaultbc: No such file or directory

The exact information about the version I am using is:

Server version information (version inside cyradm):

localhost.localdomain> version
name : Cyrus IMAPD
version : v2.3.7-Invoca-RPM-2.3.7-2.el5 2006/07/10 13:46:20
vendor : Project Cyrus
support-url: http://asg.web.cmu.edu/cyrus
os : Linux
os-version : 2.6.18-92.1.22.el5.centos.plus
environment: Built w/Cyrus SASL 2.1.22
Running w/Cyrus SASL 2.1.22
Built w/Sleepycat Software: Berkeley DB 4.3.29: (January 7,
2007)
Running w/Sleepycat Software: Berkeley DB 4.3.29: (January 7,
2007)
Built w/OpenSSL 0.9.8b 04 May 2006
Running w/OpenSSL 0.9.8b 04 May 2006
CMU Sieve 2.3
TCP Wrappers
NET-SNMP
mmap = shared
lock = fcntl
nonblock = fcntl
idle = idled

[brong]

From:

I can confirm this with cyrus imapd 2.4.8 package on Fedora 15 (Beta at this time).

My old procmail script tried to deliver the following

/usr/lib/cyrus-imapd/deliver -a myaccount -m user.myaccount.ebay

After debugging heavily i found that moving folders does not work anymore as before in the old cyrus version of fedora 14.

Bug description:

The old behaviour of cyrus was to exactly taking the -m argument and moving this into the folder. The behavour of 2.4.8 is that there is already an
existing prefix "user.myaccount" and this is joined with the -m command.
The result "user.myaccount.user.myaccount.ebay" is of course an invalid Folder.
I can understand the intention to avoid the unnecessary prefix. So i created
a patch which looks if the preifx is already there in the -m option. If yes, the whole content of the -m option is copied into the buffer.

• Additional benefit. I have added some debug messages while hunting for the
  problem, please apply, i think they will help all other users
  (see separate patch)

----------- Patch 1 Restore behaviour of old cyrus ----------

diff -ur cyrus-imapd-2.4.8/imap/lmtpd.c ../cyrus-imapd-2.4.8.modified/imap/lmtpd.c
--- cyrus-imapd-2.4.8/imap/lmtpd.c 2011-04-13 16:35:22.000000000 +0200
+++ ../cyrus-imapd-2.4.8.modified/imap/lmtpd.c 2011-04-23 15:10:53.244644764 +0200
@@ -503,6 +503,10 @@
struct appendstate as;
unsigned long uid;
const char *notifier;
+

• syslog(LOG_DEBUG, "deliver_mailbox: user %s", user);
• syslog(LOG_DEBUG, "deliver_mailbox: mailboxname %s", mailboxname);

 r = append_setup(&as, mailboxname,
                 authuser, authstate, acloverride ? 0 : ACL_POST,

@@ -701,14 +705,18 @@

    tail = namebuf + strlen(namebuf);
    if (mailboxname) {

•       strlcat(namebuf, ".", sizeof(namebuf));
  

•       strlcat(namebuf, mailboxname, sizeof(namebuf));
  

•       if ( strncmp(mailboxname,namebuf,strlen(namebuf)) == 0 ) {
  

•         strncpy(namebuf,mailboxname,strlen(mailboxname));
  

•        } else  {
  

•         strlcat(namebuf, ".", sizeof(namebuf));
  

•         strlcat(namebuf, mailboxname, sizeof(namebuf));
  

•        }
  

----- Patch 2: Helpful debug messages ---------------------------

diff -ur cyrus-imapd-2.4.8/imap/append.c cyrus-imapd-2.4.8.modified/imap/append.c
--- cyrus-imapd-2.4.8/imap/append.c 2011-04-13 16:35:22.000000000 +0200
+++ cyrus-imapd-2.4.8.modified/imap/append.c 2011-04-23 14:45:13.671147050 +0200
@@ -163,7 +163,10 @@

 as->mailbox = NULL;
 r = mailbox_open_iwl(name, &as->mailbox);

• if (r) return r;

• if (r) {

• syslog(LOG_DEBUG, "append_setup: open mailbox failed %s", name);
  

• return r;
  

• }

  as->myrights = cyrus_acl_myrights(auth_state, as->mailbox->acl);

@@ -171,6 +174,12 @@
r = (as->myrights & ACL_LOOKUP) ?
IMAP_PERMISSION_DENIED : IMAP_MAILBOX_NONEXISTENT;
mailbox_close(&as->mailbox);

•    if (r == IMAP_PERMISSION_DENIED ) {
  

•     syslog(LOG_DEBUG, "append_setup: aclcheck failed IMAP_PERMISSION_DENIED");
  

•    } else {
  

•     syslog(LOG_DEBUG, "append_setup: aclcheck failed IMAP_MAILBOX_NONEXISTENT");
  

•   }
  

•   return r;
  

  }

@@ -180,6 +189,7 @@
if (q.limit >= 0 && quotacheck >= 0 &&
q.used + quotacheck > ((uquota_t) q.limit * QUOTA_UNITS)) {
r = IMAP_QUOTA_EXCEEDED;

•        syslog(LOG_DEBUG, "append_setup: quota exceeded");
    }
  

  }
  else if (r == IMAP_QUOTAROOT_NONEXISTENT) r = 0;
  @@ -199,6 +209,7 @@
  r = mailbox_open_cache(as->mailbox);
  if (r) {
  mailbox_close(&as->mailbox);

•    syslog(LOG_DEBUG, "append_setup: could not open cachefile");
    return r;
  

  }

[brong]

Attachment-Id: 1380
From:
Type: text/plain
File: patch.bug

Restores old behaviour of cyrus

[brong]

Attachment-Id: 1381
From:
Type: text/plain
File: patch.release

Adds Debug Messages for finding problems in CYrus

[brong]

From: Jeroen van Meeuwen (Kolab Systems)

Updating the milestone to be 2.4.9, the upcoming 2.4-next release.

[brong]

From:

Guys, there was an additional bug, i wonder if anyone has ever test the old way of cyrus by using something like

/usr/lib/cyrus-imapd/deliver -a myusername -m user.myusername.ebay myusername

This doesn't work with the current code, one for the reason i posted above,
and the other reason is, that the deliver_local function is broken. Actually it is because of the "auth_newstate(username" being on the wrong line, the two calls to deliver_mailbox never suceed, because on the way down there is a check that auth_state->userid=anonymous and the identifier=myusername (deep down in auth_unix, is where it is checked).
Therefore these calls never succeed and cyrus always falls back into delivering into INBOX.

---> all attempts to deliver into a subfolder fail.

Therefore i cleaned up the code a little bit (i admit i couldn't understand what was oging on which is always a bad sign) and move the auth_newstate(username) to hopefully the right position.

Now the check for acls do not fails --> and the move into the mailbox succeeds even with the acl check on.

I tested it on my system (with and without -m pathnames) it seems to work now.

Please add this asap into your next release, as i cannot go without a working cyrus (and this functionality worked for 7 years now in my old setup)

Patch will be attached

[brong]

Attachment-Id: 1382
From:
Type: text/plain
File: patch2.diff

Restores old behaviour of cyrus (and fixes 2 bugs, one acl and one folderpath)

[brong]

Attachment-Id: 1383
From:
Type: text/plain
File: patch3.diff

Restores old behaviour V3.0 (fixes 2bugs)

[brong]

Attachment-Id: 1384
From:
Type: text/plain
File: patch3.1.diff

V3.1 of patch, further refinement, and fixed typos and missing %s

[brong]

From: Bron Gondwana

I'm trying to find where it's documented this should work.

My apologies for leaving this one so long - I really do want to integrate it into stable Cyrus, but I want to know that it's actually the correct interface and not some historical accident that was never supposed to work.

My reading is that the correct way to deliver to a subfolder is plus addressing...

[brong]

From:

Hi,
even if it was an initial bug, then this behaviour is now documented in various FAQs and emails

after a quick search with Google here an english mail from 2002 which describes the behaviour

http://lists.andrew.cmu.edu/pipermail/info-cyrus/2002-September/018660.html

From 2007 (sorry german, but there actually a lot of German FAQ documenting this behavior)
e.g. (search for DELIVER -e -a tiberian -m user.tiberian.Junk"
http://www.denkweite.de/2007-03-10/lokaler-mailserver-mit-fetchmail-procmail-und-cyrus/

so i believe this makes the case that it is today an expected behaviour - and people who start to switch from old cyrus 1.4* to 2.* releases will be very disappointed.

(Mit Bezug zu comment #9)
> I'm trying to find where it's documented this should work.
>
> My apologies for leaving this one so long - I really do want to integrate it
> into stable Cyrus, but I want to know that it's actually the correct interface
> and not some historical accident that was never supposed to work.
>
> My reading is that the correct way to deliver to a subfolder is plus
> addressing...

[brong]

From: Adam Tauno Williams

I believe the correct behavior is the use of plussed addressing. We've been using Cyrus for a decade or more and recall that as having always been the documented mechanism for subfolder delivery.

[brong]

From:

Please explain "plussed addressing"

But please provide also evidence for this. Here another bunch of URLs which document the behaviour i want to restore:
Any google search for procmail recipes and cyrus makes the case since 2001.

http://www.irbs.net/internet/info-cyrus/0504/0167.html
http://objectmix.com/imap/200854-re-cyrus-imap-sendmail-procmail-sieve-spamassassin-oh-my.html
http://www.issociate.de/board/post/405775/Another_procmail_configuration.html
http://www.spinics.net/lists/info-cyrus/msg11048.html
http://macosx.com/forums/mac-os-x-system-mac-software/49937-procmail-postfix.html

(Mit Bezug zu comment #11)
> I believe the correct behavior is the use of plussed addressing. We've been
> using Cyrus for a decade or more and recall that as having always been the
> documented mechanism for subfolder delivery.

[brong]

From: Bron Gondwana

(In reply to comment #12)
> Please explain "plussed addressing"

RCPT TO: <username+subfolder>

> But please provide also evidence for this. Here another bunch of URLs which
> document the behaviour i want to restore:
> Any google search for procmail recipes and cyrus makes the case since 2001.
>
>
> http://www.irbs.net/internet/info-cyrus/0504/0167.html
> http://objectmix.com/imap/200854-re-cyrus-imap-sendmail-procmail-sieve-spamassassin-oh-my.html
> http://www.issociate.de/board/post/405775/Another_procmail_configuration.html
> http://www.spinics.net/lists/info-cyrus/msg11048.html
> http://macosx.com/forums/mac-os-x-system-mac-software/49937-procmail-postfix.html

They all look like documentation of behaviour of the "deliver" command line tool,
which can be given a mailbox rather than a user. Hmm.

I will look and see what I can find on the matter. I guess we could go with an "if the user is pre-authenticated admin then allow a direct mailbox name", which is probably what it used to do.

By the way, the example here was "rcpt to abcd.test" - the examples in the links you have given seem to be "rcpt to user.abcd.test". I would be more willing to accept the second format than the first. The first is an unholy abomination.

Bron.

[brong]

From:

i am absolutly fine with "user.myusername.subfolder" - this is the way i want it. but the implementation (which didnt work ) currently internally prepares "user.myusername" already as an prefix and would - if it would have worked - only accept "subfolder".

My patch actually just does.
"check if user supplied subfolder has "user.myusername.subfolder", than use this.
and if the user hasn't supplied "user.myusername" it simply appends the given string internally to the provided string by the deliver system, which again results in "user.myusername.subfolder"

>
> By the way, the example here was "rcpt to abcd.test" - the examples in the
> links you have given seem to be "rcpt to user.abcd.test". I would be more
> willing to accept the second format than the first. The first is an unholy
> abomination.
>
> Bron.

[brong]

From:

i am not exactly sure what you mean by pre-authenticated admin.

The usual usage is to call "procmail" instead of "deliver" from sendmail or postfix (i used both mtas). Remember that sendmail and postfix run with the userid "mail" or "postfix" on a fedora system. With this user deliver gets called nowadays.

Btw the patch 3.1 runs now since one month witouth troubles on my system.
callpath is postfix->procmail->deliver

> "if the user is pre-authenticated admin then allow a direct mailbox name",
> which is probably what it used to do.
>
> By the way, the example here was "rcpt to abcd.test" - the examples in the
> links you have given seem to be "rcpt to user.abcd.test". I would be more
> willing to accept the second format than the first. The first is an unholy
> abomination.
>
> Bron.

[brong]

From: Bron Gondwana

Wow, just trying to read the patch again. Please, please, PLEASE if you need to do code cleanup first, make it two separate patches. It's really hard figuring out what actually changed in amongst a bunch of meaningless code layout changes.

Even better, get yourself a copy of the git repository and make a series of commits in on a branch somewhere that I can pull :)

[brong]

From:

;-)
sorry - but no git available here.
but to make it easier the only section that is "new" code is this in 3.1

• everything else is cleanup. Its an incredible messy part of code.
  (not that i claim that i did perfect, but imho, the structure is more
  clear)

• /* if the prefix is already existing in the mailbox name, then do not attach him */
• if ( strncmp(mailboxname,namebuf,strlen(namebuf)) == 0 ) {

•  strncpy(namebuf,mailboxname,strlen(mailboxname));
  

• } else {

•  strlcat(namebuf, ".", sizeof(namebuf));
  

•  strlcat(namebuf, mailboxname, sizeof(namebuf));
  

• }

(Mit Bezug zu comment #16)
> Wow, just trying to read the patch again. Please, please, PLEASE if you need
> to do code cleanup first, make it two separate patches. It's really hard
> figuring out what actually changed in amongst a bunch of meaningless code
> layout changes.
>
> Even better, get yourself a copy of the git repository and make a series of
> commits in on a branch somewhere that I can pull :)

[brong]

From: Bron Gondwana

No, you made a memory leak :(

I'm cleaning it up to match the coding standards.

The general concept of the cleanup was good, but large rewrites that don't fit with the rest of the codebase aren't going to win you confidence that it's a safe thing to apply to a stable release.

As for git - you can get it pretty easily, and install it as an ordinary user. It really is worth it.

[brong]

From: Bron Gondwana

Yeah, OK - this will go in 2.4.9. Thanks for following up on it.

[brong]

From: Bron Gondwana

The patch broke Sieve deliveries, so I have reverted it on both branches.

Here's what deliver actually does:

/* just deliver to mailbox 'mailbox' */
const char *BB = config_getstring(IMAPOPT_POSTUSER);
txn->rcpt[0].addr = (char ) xmalloc(ml + strlen(BB) + 2); / xxx leaks! */
sprintf(txn->rcpt[0].addr, "%s+%s", BB, mailbox);
txn->rcpt[0].ignorequota = ignorequota;

so in other words what gets send to the lmtpd is:

RCPT TO: <postuser+user.brong.SubFolder>

(where postuser is configurable: but most people just put 'anyone p' as an ACL or us a pre-authed admin instance of lmtpd, so it doesn't really matter)

NOTE: I'm not certain how all this interacts with altnamespace and unixheirarchysep - it could get pretty ugly. I'm more certain that your solution is not the correct one though.

[brong]

From: Bron Gondwana

Ok - I'm marking this whole kit and kaboodle invalid.

Maybe it used to work because of a bug in altnamespace and unixheirarchysep. It might be easier for you to turn those two off for your local lmtp daemon, lime this:

imapd.conf:
lmtplocal_altnamespace: 0
lmtplocal_unixhierarchysep: 0

cyrus.conf:
lmtplocal cmd="lmtpd -a" listen="/var/run/cyrus/socket/lmtp"

(or whatever)

Then deliver -m user.foo.subdir will work (I just tested it on master and stable).

You will need either '-a' in cyrus.conf, or "anyone p" ACL on the mailbox.

If you leave unixheirarchysep on, you may need to deliver to user/foo/subdir instead. I have also tested that this works.

Under the hood, if you don't set a postuser, the network traffic will be:

RCPT TO: <+user.foo.subdir>

Note the leading plus.

If you do set a postuser, it will be

RCPT TO: <postuser+user.foo.subdir>

Regards,

Bron.

[brong]

From:

sorry for causing a memory leak.

However i'd like to know if it only because of the memory leak it breaks (because this i can fix)

Why is the patch invalid?

Please provide a test case which proves failure with sieve.

I am not sure, but i spotted way too much bugs when i did go
to the delivery codepath, so my common sense tells me now:
"another sieve bug hiding"

I request to reopen is and wait for a memory leak fixed patch.

Or a test case which proves that it breaks sieve.

But please do not hide bugs by closing it.

[brong]

From: Bron Gondwana

It's invalid because you're wrong. It's not correct to deliver via LMTP to recipient user.foo.subdir, and it never was - even if somewhere on the net said so, and it perhaps used to work.

Which is why deliver sends to either:

RCPT TO: <+user.foo.subdir>

or if you specify a "postuser", to:

RCPT TO: <postuser+user.foo.subdir>

when you specify the '-m mailbox' option.

As for the sieve case, I can try to recreate a test case, but I can promise you that I rolled this code out on FastMail's production systems and had literally thousands of "could not find folder INBOX.name" errors, for mailboxes that clearly existed. When I checked the sieve scripts, they had rules of the form:

fileinto INBOX.name;

I rebuilt our packages with the one patch reverted, and the problem went away.

I do appreciate that you've put a lot of work into this, but I have spent a lot of time testing this, and it's not a bug. It's supposed to work like this. The target of a "rcpt to" is a username, not a mailbox spec.

It might contain a "detail" part, see rfc5233 for description of plus addressing and "detail" - and the detail is interpreted as being a mailbox name in the user's namespace if sieve doesn't do anything else with it.

But RCPT TO: <foo.subdir> is clearly wrong, as could be clearly seen in the
degenerate case of a user called 'user' or called 'shared'.

I have fixed a bunch of actual bugs relating to auth and delivery with the
'deliver' tool, which should mean that your LMTP deliveries can keep working,
and I have documented that you need to be careful what namespace your lmtpd is operating in on those other bugs, or you may need to use:

RCPT TO: <+user/foo/subdir@domain>

which is pretty strange looking!

Bron.

[brong]

From:

let me be clear - i do not want to defend my code endlessly - if the facts
are there i sincerly apologize for having overseen something.

but at this point in time, i only see you claiming that there is a faulty
behaviour and no fact that this due to this patch.

you are telling sieve rule doesn't work.
I have to counter argument
(1) at that point the code, the namespace has already been set by other code pathes. Thus the behaviour you are describing has nothing to do with this code snippet - it doesn't fiddle with namespace hierachies nor the separators.

(2) I also have used a sieve script and tried to reproduce your problem with filtering to inbox and "inbox.ebay". This worked correctly in my environment(*).

Therefore i fail to reproduce your problem. I have added now some more debug messages and would be happy to get a test case which reproduces your problem.

I do not argue that you don't have a problem. But it seems that we have not a common understanding of the real reason.

(*) my environment has 5% more debug messages, which i didn't publish :-)

[brong]

From: Bron Gondwana

(In reply to comment #24)
> but at this point in time, i only see you claiming that there is a faulty
> behaviour and no fact that this due to this patch.

Well, I reverted just this patch and the problem went away again. That's
usually a pretty good indicator...

> you are telling sieve rule doesn't work.
> I have to counter argument
> (1) at that point the code, the namespace has already been set by other code
> pathes. Thus the behaviour you are describing has nothing to do with this code
> snippet - it doesn't fiddle with namespace hierachies nor the separators.

It fiddles with the mailbox name inside "deliver_local", which path is used
by two things - one of which is sieve delivery.

> (2) I also have used a sieve script and tried to reproduce your problem with
> filtering to inbox and "inbox.ebay". This worked correctly in my
> environment(*).

Interesting. I'll try to make a complete testcase now, but I sure did
get a lot of issues in our environment.

Can you please give me the values of "altnamespace" and "unixheirarchysep"
that you use. For all that they don't get set at this point, they do have
influence over what gets passed to here.

> Therefore i fail to reproduce your problem. I have added now some more debug
> messages and would be happy to get a test case which reproduces your problem.

I'll see what I can do!

> I do not argue that you don't have a problem. But it seems that we have not a
> common understanding of the real reason.

Sure. I'd like to get a deeper understanding too. Will do.

> (*) my environment has 5% more debug messages, which i didn't publish :-)

[brong]

From:

:-) regarding your content you seem to believe that i have added the changing of the mailbox name. let me be clear. I have appended the original modification from 2.4.8 source code. I have added line numbers to the patch, i have made originally. "-" indicates removed code from 2.4.8 and + inidicates added code.

Important Info: From my debugging i know that in namebuf, already contains the string "user.myusername" when reaching this part of the code.

Line 20,30 are "original 2.4.8" code, they show up again in in line 70,80.

So the only modification i claim to be mine is line 40,50,60,90, two of the only contain brackets - so leaves me with 40,50.

i currently cannnot access my machiine it would be nice if you would addd on debugging statement at line 15, which would print out the content of namebuf and the content of mailboxname

Therefore i am puzzled how i would have to crash the statement in line 40,or 50.
which only says "if the mailboxname already contains the prefix, then use the whole mailboxaname".

10 if (mailboxname) {
20 - strlcat(namebuf, ".", sizeof(namebuf));
30 - strlcat(namebuf, mailboxname, sizeof(namebuf));
40 + if ( strncmp(mailboxname,namebuf,strlen(namebuf)) == 0 ) {
50 + strncpy(namebuf,mailboxname,strlen(mailboxname));
60 + } else {
70 + strlcat(namebuf, ".", sizeof(namebuf));
80 + strlcat(namebuf, mailboxname, sizeof(namebuf));
90 + }

[brong]

From: Bron Gondwana

Sorry - I can't recreate my crash, but I want to show you this:

'man deliver'

   -m mailbox
          Deliver  to  mailbox.   If any userids are specified, attempts to deliver to user.userid.mailbox for
          each userid.  If the ACL on any such mailbox does not grant the sender the "p" right or if -m is not
          specified, then delivers to the INBOX for the userid, regardless of the ACL on the INBOX.

          If  no  userids are specified, attempts to deliver to mailbox.  If the ACL on mailbox does not grant
          the sender the "p" right, the delivery fails.

See this: ff any userids are specified, attempts to deliver to user.userid.mailbox for each userid.

This documentation dates from the stoneage some time.

git blame man/deliver.8

ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 116) .BI -m " mailbox"
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 117) Deliver to
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 118) .IR mailbox .
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 119) If any
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 120) .IR userid s
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 121) are specified, attempts to deliver to
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 122) .RI user. userid . mailbox
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 123) for each
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 124) .IR userid .
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 125) If the ACL on any such mailbox does not grant the sende
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 126) or if
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 127) .B -m
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 128) is not specified,
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 129) then delivers to the INBOX for the
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 130) .IR userid ,
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 131) regardless of the ACL on the INBOX.
3c30c54 (John Gardiner Myers 1994-10-17 17:44:52 +0000 132) .IP
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 133) If no
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 134) .IR userid s
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 135) are specified, attempts to deliver to
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 136) .IR mailbox .
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 137) If the ACL on
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 138) .I mailbox
ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 139) does not grant the sender the "p" right, the delivery f

This is really, REALLY clear that the current behaviour is intentional. You have two clear correct ways to do this:

a) use a pre-authenticated connection without specifying userid and give a full mailbox path.

b) use a userid and give the bit after user.$username only.

If you're sending via LMTP then you can either specify:

a) RCPT TO: <username+subdir>
b) RCPT TO:<+user.username.subdir>

As per the RFC on subaddressing.

And I'm closing this as invalid again. If you old procmail worked, it was due to a bug, and this behaviour is correct as per the documentation.

[brong]

From:

so lets summarize the facts

(1) you did fail to recreate the behaviourr
i conclude, that your claim can not be further hold to be true
(2) you are bringing up the documentation argument
which was already was discussed in the history of this bug report
- counter arguments were also given, which should not be simply ignored

At this pointed i am disappointed - i will not fight ignorance, which is probably coming from the will to make a release.

I will keep a patched version, please proceed as you want.

(Mit Bezug zu comment #27)
> Sorry - I can't recreate my crash, but I want to show you this:
>
> 'man deliver'
>
> -m mailbox
> Deliver to mailbox. If any userids are specified, attempts to
> deliver to user.userid.mailbox for
> each userid. If the ACL on any such mailbox does not grant the
> sender the "p" right or if -m is not
> specified, then delivers to the INBOX for the userid, regardless
> of the ACL on the INBOX.
>
> If no userids are specified, attempts to deliver to mailbox.
> If the ACL on mailbox does not grant
> the sender the "p" right, the delivery fails.
>
>
>
>
> See this: ff any userids are specified, attempts to deliver to
> user.userid.mailbox for each userid.
>
> This documentation dates from the stoneage some time.
>
> git blame man/deliver.8
>
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 116) .BI -m "
> mailbox"
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 117) Deliver to
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 118) .IR mailbox .
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 119) If any
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 120) .IR userid s
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 121) are specified,
> attempts to deliver to
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 122) .RI user. userid
> . mailbox
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 123) for each
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 124) .IR userid .
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 125) If the ACL on any
> such mailbox does not grant the sende
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 126) or if
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 127) .B -m
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 128) is not specified,
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 129) then delivers to
> the INBOX for the
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 130) .IR userid ,
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 131) regardless of the
> ACL on the INBOX.
> 3c30c54 (John Gardiner Myers 1994-10-17 17:44:52 +0000 132) .IP
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 133) If no
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 134) .IR userid s
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 135) are specified,
> attempts to deliver to
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 136) .IR mailbox .
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 137) If the ACL on
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 138) .I mailbox
> ff90aff (John Gardiner Myers 1994-06-28 21:28:50 +0000 139) does not grant
> the sender the "p" right, the delivery f
>
>
> This is really, REALLY clear that the current behaviour is intentional. You
> have two clear correct ways to do this:
>
> a) use a pre-authenticated connection without specifying userid and give a full
> mailbox path.
>
> b) use a userid and give the bit after user.$username only.
>
> If you're sending via LMTP then you can either specify:
>
> a) RCPT TO: <username+subdir>
> b) RCPT TO:<+user.username.subdir>
>
> As per the RFC on subaddressing.
>
> And I'm closing this as invalid again. If you old procmail worked, it was due
> to a bug, and this behaviour is correct as per the documentation.

[brong]

From: Bron Gondwana

(In reply to comment #28)
> so lets summarize the facts
>
> (1) you did fail to recreate the behaviourr
> i conclude, that your claim can not be further hold to be true

I take back my claim that your patch is responsible for the crashes that we saw at FastMail, since I am unable to reproduce them with just your patch in a standalone environment. I strongly suspect it was actually my porting of those patches on top of master interacting badly with our environment.

> (2) you are bringing up the documentation argument
> which was already was discussed in the history of this bug report
> - counter arguments were also given, which should not be simply ignored

I have also been unable to find a single Cyrus release in which the behaviour was EVER what you want here. Cyrus 2.2 does not appear to have (in the released version anyway) done mailbox expansion at all when using an authenticated user, and Cyrus 2.3.x all did exactly what Cyrus 2.4.x does - append the mailbox name to the user's INBOX name if you pass a username to deliver. This is also what the documentation says it does.

> At this pointed i am disappointed - i will not fight ignorance, which is
> probably coming from the will to make a release.

No - I had already decided that this would be dropped from the release anyway, and I can happily release without it regardless. It's not ignorance, it's disagreeing with your assertion that this was ever a bug. Webpages published by a third party suggesting something that's not how it ever worked are not evidence to the contrary.

> I will keep a patched version, please proceed as you want.

You're quite welcome to apply whatever patches you like. We apply a small subset of patches at FastMail which provide functionality we want, but which are not appropriate for upstream. The reason I'm annoyed here is that this was pushed as a bugfix, when it's actually a new feature that clashes with the documented behaviour. And I spent a lot of time on it because I believed it was a legitimate bug, when actually it was just a failure to understand how mailboxes are addressed via LMTP with Cyrus.