BUG in separator handling for SIEVE scripts [fixed]

[nielsd]

Dear all,

on a fresh installed cyrus 3.0.5 under Linux i can't get SIEVE reacting to anything (in LMTP). It seems lmtpd avoid calling sieve/timsieve because it did not "find" any "defaultbc" script (which is there).

After tracing the process i found this - it seems the lmtp does not recognize the (existing) "defaultbc" sieve script:

--- schnipp ---
0.000019 stat("/var/imap/mailboxes.db", {st_mode=S_IFREG|0640, st_size=25920, ...}) = 0
0.000026 fcntl(5, F_SETLKW, {l_type=F_UNLCK, l_whence=SEEK_SET, l_start=0, l_len=0}) = 0
0.000030 readlink("/var/imap/sieve/n/niels^xxxxx^info/defaultbc", 0x7ffc30148f70, 4096) = -1 ENOENT (No such file or directory)
0.000025 stat("/var/imap/sieve/n/niels^xxxxx^info/defaultbc", 0x7ffc30149f30) = -1 ENOENT (No such file or directory)
0.000029 open("/etc/passwd", O_RDONLY|O_CLOEXEC) = 13
0.000024 fstat(13, {st_mode=S_IFREG|0644, st_size=1380, ...}) = 0
0.000020 read(13, "root:x:0:0:root:/root:/bin/bash\n"..., 4096) = 1380
0.000030 read(13, "", 4096) = 0
0.000019 close(13) = 0
0.000022 open("/etc/group", O_RDONLY|O_CLOEXEC) = 13
0.000022 lseek(13, 0, SEEK_CUR) = 0
0.000018 fstat(13, {st_mode=S_IFREG|0644, st_size=616, ...}) = 0
0.000021 read(13, "root:x:0:root\nbin:x:1:root,bin,d"..., 4096) = 616
0.000021 lseek(13, 0, SEEK_CUR) = 616
0.000018 lseek(13, 0, SEEK_CUR) = 616
0.000018 lseek(13, 0, SEEK_CUR) = 616
--- schnapp ---

means:
0.000030 readlink("/var/imap/sieve/n/niels^xxxxx^info/defaultbc", 0x7ffc30148f70, 4096) = -1 ENOENT (No such file or directory)
0.000025 stat("/var/imap/sieve/n/niels^xxx^info/defaultbc", 0x7ffc30149f30) = -1 ENOENT (No such file or directory)

If i try to ls the files:

not found:
ls -alh /var/imap/sieve/n/niels^xxxxx^info/defaultbc
ls: Zugriff auf '/var/imap/sieve/n/niels^xxxxx^info/defaultbc' not found

found:
ls -alh /var/imap/sieve/n/niels.xxxxx.info/defaultbc
lrwxrwxrwx 1 cyrus mail 6 22. Mär 12:04 /var/imap/sieve/n/niels.xxxxx.info/defaultbc -> neu.bc

So it seems the unixsep / path translation does not work here to find the "defaultbc", while other things seem to work:

i.e.:
openat(AT_FDCWD, "/var/spool/imap/n/user/niels^xxxxx^info/cyrus.index", O_RDONLY) = 13

some settings from imapd.conf
--- schnipp ---
configdirectory: /var/imap

duplicate_db_path: /run/cyrus/deliver.db
ptscache_db_path: /run/cyrus/ptscache.db
statuscache_db_path: /run/cyrus/statuscache.db
tls_sessions_db_path: /run/cyrus/tls_sessions.db

defaultpartition: default
partition-default: /var/spool/imap

sievedir: /var/imap/sieve
sievenotifier: notifyd
sieveusehomedir: 0
sieve_maxscriptsize: 256
sieve_maxscripts: 64

lmtpsocket: /var/imap/socket/lmtp
idlesocket: /var/imap/socket/idle
notifysocket: /var/imap/socket/notify

hashimapspool: yes
virtdomains: yes
unixhierarchysep: yes
lmtp_downcase_rcpt: yes
idlemethod: idled
lmtp_admins: mail cyrus exim
imap_admins: cyrus
sieve_admins: cyrus
altnamespace: yes
umask: 027
--- schnapp ---

any ideas / solutins / hints?

Or id i hit a bug in the cyrus internal "path translation" (i.e. conversion between "." and "^" here? Or any internal caching regading this?

add:
On the mailing list i've got pointed to:
https://lists.andrew.cmu.edu/pipermail/info-cyrus/2018-March/040017.html

which seem similiar, even if that user comes frrom a upgrade (my is a fresh / "out of the box" install).

many thanks for your time and any kind of help,

Niels.

[nielsd]

...should be fixed as got informed by mailing list in 3.0.6.

sorry for the noise...ß)

[hagedose]

Fixed in 73af8e1

[slimlv]

How user "." (Single dot) will be handled with no conversation on fs where dots have special meaning?

[elliefm]

Off the top of my head, I'm not sure. But the fix for this issue doesn't change it.

[elliefm]

User . (and .., and all user/mailbox names with a leading .) are forbidden by mboxname_policycheck() in imap/mboxname.c: https://github.com/cyrusimap/cyrus-imapd/blob/master/imap/mboxname.c#L1552-L1571

[slimlv]

The commit is very major change. It affects users who will migrate existing account base from previous Cyrus versions where were no such restrictions.
I bet that thousands of existing account will be blocked around the world with this commit

[elliefm]

I'm not sure which commit you refer to? Leading dots in user/mailbox names have been forbidden in Cyrus since at least 1993: 5ed0a30

The lmtpd bug being reported in this issue was introduced in 3.0.0-beta2. The fix restores the behaviour of lmtpd to match earlier versions.

[slimlv]

I'm wondering how mailbox's name is checking against badmboxpatterns in the code.
Will newly created account "inbox" (cyrus mailbox "user.inbox") be matched ?

I have cyrus-imapd-2.5.10 - not latest nor 1993 made

0 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE URLAUTH URLAUTH=BINARY X-NETSCAPE LOGINDISABLED COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User logged in SESSIONID=<cyrus-15161-1522832270-1-9856250718854119771>
0 create user/inbox
0 OK Completed
0 create user/.test
0 OK Completed

then I got in mbailboxes.db the record
user.^test%(A %(.test lrswipkxtecdan) I fc7adfae-be93-4579-bf87-faf217e3395b P default V 1522832294 M 1522832536)
and folder "/var/spool/imap/q/user/^test" on file system.

So with commit 73af8e1 and upgrading to cyrus 3.0 I'll get folder name "/var/spool/imap/q/user/.test", right ?

and now I can do

0 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE URLAUTH URLAUTH=BINARY X-NETSCAPE LOGINDISABLED COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User logged in SESSIONID=<cyrus-16091-1522832861-1-4156766652862691837>
0 create user/..
0 OK Completed

May be other version is not affected ?

* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE STARTTLS AUTH=PLAIN AUTH=LOGIN AUTH=CRAM-MD5 SASL-IR] buba Cyrus IMAP git2.4.17+0-Debian-2.4.17+nocaldav-0+deb8u2 server ready
0 login cyrus passwordhere
0 OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxte QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SORT SORT=MODSEQ SORT=DISPLAY THREAD=ORDEREDSUBJECT THREAD=REFERENCES ANNOTATEMORE LIST-EXTENDED WITHIN QRESYNC SCAN XLIST URLAUTH URLAUTH=BINARY LOGINDISABLED COMPRESS=DEFLATE IDLE] User logged in SESSIONID=<cyrus-9486-1522833194-1>
0 create user/..
0 OK Completed

How you think, may be something is broken here ?

[elliefm]

That's weird. I'm not sure what's going on yet, but I've separated it into a new issue: #2308

[elliefm]

So with commit 73af8e1 and upgrading to cyrus 3.0 I'll get folder name "/var/spool/imap/q/user/.test", right ?

I need to reiterate that that commit does not change where files are written to. I think you've found a bug (hence opening the new issue), but if so it's a bug that already exists. 73af8e1 doesn't change things for this case.

[elliefm]

I should also clarify that it applies only to the paths where sieve scripts are read from. Other mailbox paths have the usual . => ^ escaping, so there will not be a dot on disk for these.

[dilyanpalauzov]

Does a1a9aad help? I counted five different places where the algorithm for the sieve path is implemented (so five implementations of the same algorithm) which I addressed in #2694 .

[dilyanpalauzov]

As everybody agreed that this is fixed, can the case be closed?

[nielsd]

Am Mittwoch, 13. Januar 2021, 17:24:47 CET schrieb Дилян Палаузов:

As everybody agreed that this is fixed, can the case be closed?

for me, it seems working flawless now so far. many thanks to you all! niels. -- --- Niels Dettenbach Syndicat IT & Internet https://www.syndicat.com PGP: https://syndicat.com/pub_key.asc ---