New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
corsware beta regression #43962
Comments
Hey! So this happens due to ""Too many open files". The corsware test suite opens sockets. And since cargo runs all tests in parallel, the number of open sockets while the tests are running are not negligible. How are the tests ran? Can you increase ulimit? |
I can't reproduce this locally, and there's a ton of I/O happening here which is a candidate for spurious failures. While I don't believe that the error is related to open files, without a reproduction locally, I'm going to close assuming spurious. |
I'll investigate a bit. I can reproduce if I lower nofile, and there seems to be a leak somewhere. I'll write back here if I find anything of value. |
So, the reason for having so many files open is that iron and hyper leaks connections. If you look at https://docs.rs/iron/0.5.1/iron/struct.Listening.html#method.close, it is documented that closing the iron server doesn't work and leaks connections and threads due to a known bug in hyper 0.10. Additionally, iron and hyper by default use many threads and large connection pools. I've tuned those down for the tests. But I couldn't prevent the leak completely. Despite disabling KeepAlive in the server, connections remain idle. The client doesn't seem to close the connection properly with KeepAlive=false. But even though I couldn't prevent the leak completely, tuning made the number of open files when running the http tests to go down from around 200 to 14. So I hope that this won't be an issue again. I released corsware 0.1.5 with these changes. |
@atorstling many thanks! |
corsware version 0.1.4 test-regressed from stable to beta (full log):
cc @atorstling
The text was updated successfully, but these errors were encountered: