We are using keter as reverse-proxy. Recently we upgraded keter from 0.3.4.2 to 0.3.5.4 and after that we started getting frequent Out of memory error.
In one instance, we found keter using more that 95% memory before it kills & restarts itself.
******* Syslog *****
Mar 26 10:54:02 ip-10-160-58-234 kernel: [262376.828223] Out of memory: Kill process 10797 (keter) score 919 or sacrifice child
Mar 26 10:54:02 ip-10-160-58-234 kernel: [262376.828245] Killed process 10797 (keter) total-vm:2883048kB, anon-rss:1618140kB, file-rss:0kB
Mar 26 10:54:03 ip-10-160-58-234 kernel: [262377.767756] init: keter main process (10797) killed by KILL signal
Mar 26 10:54:03 ip-10-160-58-234 kernel: [262377.778114] init: keter main process ended, respawning
I dont think we experience such an outage while using 0.3.4.2.
Is there any chances of memory leaks in the latest version?
How long does it take before you get this memory error and are you only hosting site(s) built with yesod?
Immediately after we browsed we pages, it is starting to increase and reaches above 95% with no further load on keter.
Yes, we have only one application deployed under keter which uses yesod 220.127.116.11 and it's has no memory issues.
I'm not ignoring this issue, I just haven't had a chance to dig into it yet. It would be incredibly helpful if you could rebase this down to the commit that begins to demonstrate the problem.
I'm unable to reproduce this problem locally. Can you provide the config file you're using? And can you test if this problem occurs with a plain, scaffolded Yesod app?
- from: www.example.com
- host: example.com/static
root: ../static # relative to config file, just like the executable
Sure I will try with plain app under keter.
Also, we have a another server running fine under old keter(0.3.4.2). I will upgrade the keter there to see if I'm getting same problem.
Bingo, that was the important missing piece: SSL. I can now reproduce it.
Fix memory leak in TLS closed connections (snoyberg/keter#13)
TLS bump for #13
OK, I believe I found the bug in network-conduit-tls. I've released a new version of that package, as well as keter 0.3.6.1 which requires it. Can you try the new version and see if it solves the problem?
Thanks Michael. Will try out now and let you know the result.
I believe the bug has been fixed. It's working fine till now. Shall we close this issue?
Yes, thanks for confirming.