Really big mailbox are delayed to show

[chrisdent]

For really big mailboxes there is a delay to show messages. (This one is 15.6GB)

Some times the Loading appears for hours then the list appears, then Empty List.

[the-djmaze]

SnappyMail relies on the speed of the IMAP server.
This is mostly noticeable on big mailboxes.
Cleanup the box by removing messages and/or categorize them in folders.

[chrisdent]

does SnappyMail reads the index file in order to load the list of messages and folders?

Or does it reads all the mailbox.

[the-djmaze]

SnappyMail connects with IMAP server and asks all UID's because of threads and sorting options.
If you have 100000's of UID's this is slow.

Like i said: organize

1. Create folder
2. Move some messages to that folder
3. Create folder
4. Move some messages to that folder
5. etc.

That should speed things up.

[chrisdent]

A user that has 70GB mailbox, can't see the emails.

User waited for hours to see the mailbox. Then everything was working fine.

Suddenly like the process repeated as for the first login, the user is unable to see emails. Is the process repeating? Maybe the cache of the UID was blown?

Difficult to tell users how to use their mailbox. Is there a way to optimize the load? I can sponsor.

[the-djmaze]

The problem is several things.

1. An UID list is created of all threads
2. An UID list is created of each SORT routine
3. An UID list is created of each SEARCH
4. Folder STATUS is run to find unseen messages.

The UID lists are important to find and match messages with pagination.

There are options in the config to disable sort, thread and list-status which speed things up.
But also the speed of the IMAP server is important and if it uses good indexes for such large mailboxes.

Best is always to debug and find where you can gain speed improvements.

For each there are options or new options to prevent overloading.

Hack, there was even a forced date filter that i remove in next release.
It limited mail to the last N months and that also speeds up, but the side effect is that you can't read, search, find messages older then that.

[the-djmaze]

Also see

snappymail/snappymail/v/0.0.0/app/libraries/MailSo/Mail/MailClient.php

Line 775 in d0db7f5

public function MessageList(MessageListParams $oParams) : MessageCollection

And the messagelist limits there

[the-djmaze]

I've modified the code so that you can play around with the settings per domain (in next release).
In previous versions it was only global through application.ini

Most important is the "messages limit" (application.ini message_list_count_limit_trigger)

[chrisdent]

you're awesome!

[MI-KY]

Hi @the-djmaze - for the documentation I would like to be sure on one thing: what side effects has limiting the message count (I already saw that folder count was removed again) ?
Will the user only see this number of mails or does this only have an effect on searching / filtering the messages as the search will only work on this number of mails?

[the-djmaze]

When messages > N the system will not cache UID or use SORT for messages.
Instead it will just fetch a range of messages based on index.
This is faster but disables features like: threads, sort by date/size/subject

RainLoop only had a global setting for all domains.
Now in SnappyMail you can control it per domain and modify in extensions/plugins

[chrisdent]

Been thinking.

What if?

If mailbox is huge in the first load, get last N messages for example 100. So user can visualize email, while in the background the process is running?

[the-djmaze]

So user can visualize email, while in the background the process is running?

Yes, but that all depends on HTTP daemon capabilities and PHP timeouts.

But if server does not support SORT/THREAD it could do this by default.

[chrisdent]

That´s part of the awesome experience! =)

[chrisdent]

How does this process runs?

• Is there a possibility to show % while running this process?

• Is it possible to run it from command line?

• Can it show the messages then in background run the process?

• If Inbox and Sent are big, changing folder cancels the previous process?

• Using dovecot director with different backend, when changed backend the process runs again.

• If mailboxes are 30GB + it takes hours to show.

[the-djmaze]

* Is there a possibility to show % while running this process?

No

* Is it possible to run it from command line?

No

* Can it show the messages then in background run the process?

Currently not

* If Inbox and Sent are big, changing folder cancels the previous process?

Yes and no, depends on the current active PHP code

* If mailboxes are 30GB + it takes hours to show.

Why are boxes so huge?
Do you ever read 30G of messages?

[chrisdent]

Users don't delete there messages as they use it as their archive. Some users have 20+ years of emails, thew aren't allow to delete them.

Can this be a feature request? -> Can it show the messages then in background run the process?