New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize POP3 fetcher UID compare algorithm #93

Closed
japi opened this Issue Mar 16, 2015 · 1 comment

Comments

Projects
None yet
2 participants
@japi
Copy link

japi commented Mar 16, 2015

Due to high CPU usage i have just come across an interesting behaviour:
If an external pop3 account has many mails (in this case 17000+), the server seems to be busy for ~2 minutes just for comparing the UIDs, with one CPU core at maximum load (on an Xeon 3060 Dualcore). Since the user had the fetch interval at 7 minutes, the load was clearly noticeable.
I guess the code at https://github.com/hmailserver/hmailserver/blob/master/hmailserver/source/Server/ExternalFetcher/FetchAccountUIDList.cpp is doing this check.

Server Log:
"POP3C" 2916 7039 "2015-03-16 22:04:55.432" "IPXXX" "RECEIVED: +OK password required for user "XXXXXXXXXX""
"POP3C" 2916 7039 "2015-03-16 22:04:55.432" "IPXXX" "SENT: (password)"
"POP3C" 2924 7039 "2015-03-16 22:04:55.556" "IPXXX" "RECEIVED: +OK mailbox "XXXXXXXXXX" has 17402 messages (1007106996 octets) H XXXXXXXXXX"
"POP3C" 2924 7039 "2015-03-16 22:04:55.556" "IPXXX" "SENT: UIDL"
[..]
"POP3C" 2948 7039 "2015-03-16 22:06:34.258" "IPXXX" "RECEIVED: +OK[nl]1 [...]
what follows is a line of ~512kb length with UIDs.

Windows Performance Counters look like this during the fetch:
image
Im not sure why the page faults rise after some seconds into the fetch. The Server has lots of free Ram.

Of course this was an extreme case, but optimizing this algorithm could maybe improve the performance, especially if this feature is heavily used by more than one user :-)
I increased the fetch interval and asked the user to think about enabling the automatic deletion, but in some scenarios this may not be an option.

@martinknafve

This comment has been minimized.

Copy link
Collaborator

martinknafve commented Apr 24, 2016

Fixed in 5.7.

RvdHout added a commit to RvdHout/hmailserver that referenced this issue Dec 12, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment