Mailboxes with children are reported as "(\HasNoChildren)" if mailbox with similar name

[guimard]

Issue reported to Debian BTS, reference: 939264 by Gernot Schilling

---

Dear Maintainer,

I installed a fresh buster for reproduction.
I tested with cyrus-imapd version "3.0.8" from buster and "3.0.11" from stable-backports.

Create user "imapuser" with password "password"
User created 3 folders:

• INBOX/foo
• INBOX/foo/1
• INBOX/foo-1

imap command '. list "" * ' marks folder "foo" as HasNoChildren.

root@debian-buster:~# telnet localhost 143
Trying ::1...
Connected to localhost.
Escape character is '^]'.
* OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE AUTH=DIGEST-MD5 AUTH=CRAM-MD5 AUTH=NTLM AUTH=LOGIN AUTH=PLAIN SASL-IR] debian-buster Cyrus IMAP 3.0.11-Debian-3.0.11-1~bpo10+1 server ready
. login testuser password
. OK [CAPABILITY IMAP4rev1 LITERAL+ ID ENABLE ACL RIGHTS=kxten QUOTA MAILBOX-REFERRALS NAMESPACE UIDPLUS NO_ATOMIC_RENAME UNSELECT CHILDREN MULTIAPPEND BINARY CATENATE CONDSTORE ESEARCH SEARCH=FUZZY SORT SORT=MODSEQ SORT=DISPLAY SORT=UID THREAD=ORDEREDSUBJECT THREAD=REFERENCES THREAD=REFS ANNOTATEMORE ANNOTATE-EXPERIMENT-1 METADATA LIST-EXTENDED LIST-STATUS LIST-MYRIGHTS LIST-METADATA WITHIN QRESYNC SCAN XLIST XMOVE MOVE SPECIAL-USE CREATE-SPECIAL-USE DIGEST=SHA1 X-REPLICATION URLAUTH URLAUTH=BINARY LOGINDISABLED COMPRESS=DEFLATE X-QUOTA=STORAGE X-QUOTA=MESSAGE X-QUOTA=X-ANNOTATION-STORAGE X-QUOTA=X-NUM-FOLDERS IDLE] User logged in SESSIONID=<cyrus-6552-1567434229-1-698549360105201265>
. create INBOX.foo
. OK Completed
. create INBOX.foo.1
. OK Completed
. create INBOX.foo-1
. OK Completed
. list "" *
* LIST (\HasChildren) "." INBOX
* LIST (\HasNoChildren) "." INBOX.foo
* LIST (\HasNoChildren) "." INBOX.foo-1
* LIST (\HasNoChildren) "." INBOX.foo.1
. OK Completed (0.000 secs 4 calls)
. list "" INBOX.%
* LIST (\HasNoChildren) "." INBOX.foo
* LIST (\HasNoChildren) "." INBOX.foo-1
* LIST (\Noselect \HasChildren) "." INBOX.foo
. OK Completed (0.000 secs 3 calls)
. list "" INBOX.foo
* LIST (\HasChildren) "." INBOX.foo
. OK Completed (0.000 secs 2 calls)

Because of the "\HasNoChildren" some clients fail to select subfolder INBOX.foo.1

The code seems to depend on the order of mailboxes and sets "HasChildren" on the previous Entry if the current entry is a child.

[elliefm]

The problem is that the internal mailbox separator is ASCII '.', which sorts after valid mailbox characters like '-' and space. It's fixed by enabling the following setting in imapd.conf:

      improved_mboxlist_sort: 0
          If  enabled,  a special comparator will be used which will correctly sort
          mailbox names that contain characters such as ' ' and '-'.

          Note that this option SHOULD NOT be changed on a live system.  The  mail‐
          boxes  database  should  be  dumped  (ctl_mboxlist)  before the option is
          changed, removed, and then undumped after changing the option.  When  not
          using  flat files for the subscriptions databases the same has to be done
          (cyr_dbtool)      for      each      subscription      database       See
          improved_mboxlist_sort.html.

Ideally this setting would default to enabled, or indeed, be the only behaviour (with no option at all). But it has ramifications for existing deployments, so we can't just change it. I assume that it's not simply enabled in the imapd.conf that the Debian package ships for much the same reason.

The "improved_mboxlist_sort.html" referenced in the man page seems to be https://www.cyrusimap.org/imap/developer/thoughts/improved_mboxlist_sort.html

The problem they're seeing (as well as other similar issues between similarly-named folders that differ by a character that sorts before ASCII '.') are unfixable in stable releases without enabling this option.

I don't understand all the ramifications of an admin changing the improved_mboxlist_sort option. It might be as "simple" as the documentation suggests (dumping and undumping mailboxes and subscription databases) or it might be more complicated these days. I'd suggest they join the info-cyrus list and see if anyone has been through this process recently and has guidance to offer.

[guimard]

@elliefm: I forwarded your response, thanks !