IMAP logins intermittently slow. Sometimes fast, sometimes +4sec, or +8sec

[saaby]

I am having a weird issue with my docker-mailserver deployment. IMAP logins are sometimes "fast" (~150ms), sometimes they are "slow" (~4.150sec, or ~8.150sec). I cannot find anything related reported in mail.log, mail.warn or mail.err.

Context:

• imap connection is always fast, it is the logins which are sometimes slow
• relatively few active users; <10 on average. server has no significant load.
• running under kubernetes, IMAP configured as a "LoadBalancer" service.
• login times are always ~150ms + a multiple of 4secs, lending itself to some timeout, but I have no idea where.

This is my testcase:

time echo -e "A login user@domain.com PASSWORD\nA LOGOUT" | openssl s_client --connect mail.domain.com:993 -crlf -quiet

This is the good result:

real	0m0.155s
user	0m0.039s
sys	0m0.018s

These are the bad results:

real	0m4.143s
user	0m0.040s
sys	0m0.009s

or

real	0m8.136s
user	0m0.021s
sys	0m0.006s

[glime-ai[bot]]

Please note that this response was created using generative AI, leveraging previously available information from the repository. This information may be helpful. However, it is our strong recommendation that you carefully consider the information before blindly applying it.

This response is generated only once - you will not be able to interact with the glime-ai.

---

It seems like you are experiencing intermittent delays in IMAP logins, which could be related to timeouts or retries in the authentication process. Here are a few steps and considerations to help diagnose and potentially resolve the issue:

1. Check DNS Resolution: Ensure that DNS resolution is consistent and fast. Delays in DNS resolution can cause timeouts. You can test DNS resolution times using tools like dig or nslookup.

2. Review Dovecot Configuration: Ensure that your Dovecot configuration is optimized and does not have any settings that could introduce delays. Specifically, check the authentication mechanisms and any timeout settings.

3. Network Latency: Since you are running under Kubernetes with a LoadBalancer service, there might be network latency or issues with the LoadBalancer itself. Check the network performance and ensure that there are no intermittent network issues.

4. Fail2Ban: Ensure that Fail2Ban is not mistakenly banning or throttling IP addresses. You can check and unban IPs using:

   docker exec <CONTAINER NAME> setup fail2ban
   docker exec <CONTAINER NAME> setup fail2ban unban <YOUR IP>

5. SSL/TLS Handshake: Since you are using SSL/TLS, ensure that the SSL certificates are valid and that there are no issues with the SSL handshake. You can test the SSL connection using:

   docker exec mailserver openssl s_client -connect mail.domain.com:993 -CApath /etc/ssl/certs/

6. Logs and Debugging: Enable detailed logging for Dovecot and review the logs for any anomalies or patterns that could indicate the source of the delay. You can also use the setup debug command to debug fetchmail if relevant:

   docker exec -it dms-container-name setup debug fetchmail

7. Kubernetes Configuration: Ensure that your Kubernetes configuration, including the LoadBalancer and any network policies, is correctly set up and not causing intermittent delays.

For more detailed steps and troubleshooting, you can refer to the following documentation:

• Debugging DMS
• SSL/TLS Configuration
• Fetchmail Configuration

This comment was generated by Glime.

[saaby]

Found the source of the delay.

A combination of the following three:
1: Dovecot failed auth penalties: https://doc.dovecot.org/configuration_manual/authentication/auth_penalty/
2: An ongoing bruteforce imap login attack from various IPs
3: My kubernetes service definition somehow not forwarding real remote IPs of clients, rendering all login attempts originating from the same (internal cluster) IP.

Fixed, by adding:

"externalTrafficPolicy: Local"

To my k8s Service definition (type: LoadBalancer)

[polarathene]

My kubernetes service definition somehow not forwarding real remote IPs of clients, rendering all login attempts originating from the same (internal cluster) IP.

Fixed, by adding:

"externalTrafficPolicy: Local"

To my k8s Service definition (type: LoadBalancer)

FWIW this information is covered in our docs:

---

A combination of the following three:
1: Dovecot failed auth penalties: https://doc.dovecot.org/configuration_manual/authentication/auth_penalty/
2: An ongoing bruteforce imap login attack from various IPs

You would need something equivalent to Fail2Ban for this if your k8s deployment is across multiple nodes. IIRC one of our maintainers uses a script that Fail2Ban triggers, which has a remote node that manages the network security ban the IP.

If it all runs on the same host, then Fail2Ban should work at preventing malicious users causes problems AFAIK.

[saaby]

My bad, that completely eluded me, was instead staring at the "Exposing your Mail Server to the Outside World" section.

As this is a single-server deployment, I have already enabled the fail2ban function, which works wonders.

Thanks!