-
Notifications
You must be signed in to change notification settings - Fork 935
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase MAX_WORKER_THREADS so it can be used in high traffic websites #50
Comments
MAX_WORKER_THREADS is set to 1024 in case it has not been set before. Windows 32 has a limit of about 2000 threads Since this is a limit due to memory architecture, Linux may have similar limits (that seem configurable: http://stackoverflow.com/questions/344203/maximum-number-of-threads-per-process-in-linux) Increasing the number of threads to increase the number of connections seems to be a huge waste of memory. The alternative is a redesign: However, I would be interested to find out how the server behaves by just increasing the number of threads. Did you test it with 20480? |
I just checked (with cat /proc/sys/kernel/threads-max) my Mageia linux, it is 62108, on CentOS 7, 6187514. |
By "interface 10G limit" you mean, your test already transfers 10 gigabit per second, so the number of connection does not increase anymore. You could try it with slow clients, or even just standard telnet clients that don't do anything after connecting. If you do have good experience with a some-10-k limit, there is not reason not to increase the default maximum define, at least for 64 bit. Yes, there would be much smarter ways to handle that much connections, but they take more time to implement than to just change one line - and in the meantime it does not hurt to change this number. I would change it to a number where you would hit the port limit (64k), after doing some tests. |
May I ask you to tell me a little bit about your experience with using |
Hi, I limited the client speed in my test, so I hit the 20480 connections limit I put, all working fine, there is no single error message, thank you. Regards |
Hi, You are using a 64 bit system - how much physical memory does it have? Regards, On Thu, Nov 20, 2014 at 8:54 AM, Mustafa Muhammad notifications@github.com
|
Hi, Regards |
Please tell me when you found time to test what happens with the 102400 thread limit. |
Hi, I did compile with 100k, I am hitting a limit at about 32k, it is not number of open files nor process count, but this is where I stopped. Regards |
In the IP protocol, there is a 16 bit port number, so the number of ports is limited to 64k. |
The 64k limit is actually per IP address, so if your server has 2 ip addresses, it could have 128k connections - but this is rather theoretical, I don't think a limit above 64k would make any sense in a practical application. |
The limit is now 64k. It is still interesting to know what number you can actually reach at maximum. |
I don't think this is related to the ports, and no, not the same machine, this is about the running threads, the maximum number of threads I got is 32K, before testing the connections, I checked the number of threads (ps aux -L | grep civetweb | wc -l). Regards |
Civetweb now prints out how much worker threads have been created (if it were less than configured). I tried to reach >30000 threads, but Linux is, for whatever reason, full of limits to basically everything. |
There are a couple of limits, I had to adjust for Debian/Ubuntu-Linux. /etc/sysctl.conf: /etc/security/limits.conf:
The stack limit is already set in the Makefile. Then I can reach 65536 worker threads, and the server runs stable with it. When running the clients at the same machine as the server, I reach about On Sat, Nov 29, 2014 at 8:08 AM, Mustafa Muhammad notifications@github.com
|
This will ensure that we use lower case UUIDs. Undefining the macro won't work and we likely don't want to change the default behavior. This closes #50. Signed-off-by: Aldrin Piri <aldrin@apache.org>
…when-mg_start2-f EDGE-2116-resource-leak-when-mg_start2-fails and !defined(ALTERNATIVE…
Hi,
MAX_WORKER_THREADS is set to 1024 in civetweb.c, [1] why not set it to something like 20K or 30K, this is useful and has no bad effect since the number is set in the config file, I needed to increase this to 20480 becuase I need a very large number of long connctions.
[1] https://github.com/bel2125/civetweb/blob/master/src/civetweb.c
The text was updated successfully, but these errors were encountered: