Re-enable FTS when it proves stable

[bilyboy785]

Hello,

I've got some problem with my Mailu. All is ok, and then roundcube respond 504 Gateway Timeout, email not comes. I've got theses errors in docker-compose logs :

imap         | Dec 15 19:46:54 indexer: Error: Indexer worker disconnected, discarding 1 requests for contact@{domain.tld}
imap         | Dec 15 19:46:54 indexer-worker(contact@{domain.tld})<273></nlTsxR9DMesEgAJ:MLmnFL1YFVwRAQAAJV66Xw>: Fatal: master: service(indexer-worker): child 273 killed with signal 11 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)
imap         | Dec 15 19:50:11 Error: imap(contact@{mydomain})<392><593fyBR95sisEgAJ>: Timeout (180s) while waiting for lock for transaction log file /mail/contact@{mydomain}/dovecot.index.log (WRITE lock held by pid 236)
imap         | Dec 15 19:50:11 imap(contact@{mydomain})<392><593fyBR95sisEgAJ>: Info: Connection closed (UID store finished 180.001 secs ago) in=442 out=153559 deleted=0 expunged=0 trashed=0 hdr_count=22 hdr_bytes=9557 body_count=0 body_bytes=0

i'm on Master.

If i docker-compose restart, it's OK but each message is received in quadruple

[ofthesun9]

Hello, where do you store your mailu data. Locally ? on a remote share ? (I had similar issues with an NFS share)

[bilyboy785]

I use local storage on Scaleway Instance.
I rollback to 1.5 and now all is OK... strange, i’ve got another mailu on master, with 8 domains : NO PROBLEMS.

[muhlemmer]

It seems that the Log file is locked by another process or a lock that remained in place from a crash or improper shutdown? It would be interesting to know what PID 236 and if it is still running?

[muhlemmer]

@bilyboy785:

• Scaleway as in https://www.scaleway.com/?
• This would mean a Virtual machine on NVME storage?
• Do you have any override configuration on Dovecot? Please share if so.

Tomorrow I'll try to create core dumps and trace backs to submit a report upstream. Would be helpful to make sure if it deployment related or agnostic.

[muhlemmer]

I can confirm on my test server. Fresh install, empty mailbox and no exotic filesystem. When sending the first mail to the server it crashed the worker with:

Dec 19 14:52:46 lmtp(admin@usrpro.io)<479><tB6ZJD5bGlzfAQAAaHleug>: Info: sieve: msgid=<adbc2da7-0ef6-f2ad-d4f5-439e32cb1379@usrpro.io>: stored mail into mailbox 'INBOX'
imap_1       | Dec 19 14:52:46 lmtp(479): Info: Disconnect from 192.168.203.10: Client has quit the connection (state = READY)
imap_1       | Dec 19 14:52:46 indexer-worker(admin@usrpro.io)<485><FtJIJ2F9nKPAqMsG:ewjsLj5bGlzlAQAAaHleug>: Fatal: master: service(indexer-worker): child 485 killed with signal 11 (core not dumped - https://dovecot.org/bugreport.html#coredumps - set /proc/sys/fs/suid_dumpable to 2)
imap_1       | Dec 19 14:52:46 indexer: Error: Indexer worker disconnected, discarding 2 requests for admin@usrpro.io
imap_1       | Dec 19 14:52:46 pop3-login: Info: Aborted login (no auth attempts in 0 secs): user=<>, rip=127.0.0.1, lip=127.0.0.1, secured, session=<9G/LKmF9u6x/AAAB>
imap_1       | Dec 19 14:52:46 imap(admin@usrpro.io)<466><FtJIJ2F9nKPAqMsG>: Info: Connection closed (IDLE running for 0.001 + waiting input for 0.001 secs, 2 B in + 10 B out, state=wait-input) in=2483 out=1904 deleted=0 expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0

[muhlemmer]

https://bugs.alpinelinux.org/issues/9779

[muhlemmer]

See #775. Furthermore I would like add some info (as discussed on matrix earlier this week).

On systems that are affected by this bug, the crashing indexer-worker sometimes leaves index files corrupted. This slowly kills the imap server to the point where login is no longer possible. To fix this condition you need to delete the index files and restart the imap container to force client re-login.

sudo find /mailu/mail -name "dovecot.index*" -exec rm -v {} +
docker-compose restart imap

The index files will be re-generated by Dovecot.

[Nebukadneza]

Since this issue is part of the milestone 1.6, and @HorayNarea just removed FTS to increase stability of the upcoming 1.6 release … shall we take this issue out of 1.6, or close it in favor of one that describes getting clucene working and back in?

[frzme]

A potential option might be to go for fts-solr, according to the docs it's more stable than fts-lucense (see https://wiki.dovecot.org/Plugins/FTS/Lucene )

[Nebukadneza]

In #1320 , we added a FTS again. Thus, closing this as finished.