-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Autotuning of the number of async I/O threads #151
Comments
So is default value like |
@binarin that sounds good but I'd add "but no fewer than 64". |
Looks like detecting number of CPUs is not as simple as it seems on some OSes. Lev Walkin posted the following snippet in erlang-russian ML:
|
@binarin combined with the "no fewer than 64" condition above, it sounds good to me. It's unlikely that there will be significantly more schedulers configured than there are logical cores. We might want to cap the maximum number, say, at 512 or 1024, too. |
Proper version of this calculation is implemented upstream - rabbitmq/rabbitmq-server#151 And removed version of the code is actually harmful, as it's using physicalprocessorcount (i.e. number of CPU sockets) for calculations. So on a 2 CPU/48 thread system there it was 30 threads instead of 768 calculated upstream. I've decided that it's easier to completely remove this code instead of duplicating current formula in upstream and providing a way to override this value through hiera - just not worth a hassle. Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115 Closes-Bug: 1573696
Proper version of this calculation is implemented upstream - rabbitmq/rabbitmq-server#151 And removed version of the code is actually harmful, as it's using physicalprocessorcount (i.e. number of CPU sockets) for calculations. So on a 2 CPU/48 thread system there it was 30 threads instead of 768 calculated upstream. I've decided that it's easier to completely remove this code instead of duplicating current formula in upstream and providing a way to override this value through hiera - just not worth a hassle. Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115 Closes-Bug: 1573696 (cherry picked from commit 0c16cbc)
Proper version of this calculation is implemented upstream - rabbitmq/rabbitmq-server#151 And removed version of the code is actually harmful, as it's using physicalprocessorcount (i.e. number of CPU sockets) for calculations. So on a 2 CPU/48 thread system there it was 30 threads instead of 768 calculated upstream. I've decided that it's easier to completely remove this code instead of duplicating current formula in upstream and providing a way to override this value through hiera - just not worth a hassle. Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115 Closes-Bug: 1573696
Proper version of this calculation is implemented upstream - rabbitmq/rabbitmq-server#151 And removed version of the code is actually harmful, as it's using physicalprocessorcount (i.e. number of CPU sockets) for calculations. So on a 2 CPU/48 thread system there it was 30 threads instead of 768 calculated upstream. I've decided that it's easier to completely remove this code instead of duplicating current formula in upstream and providing a way to override this value through hiera - just not worth a hassle. Change-Id: I415d446af0a822d2a5ce3478fd9db1dd0f13e115 Closes-Bug: 1573696
See this mailing list thread.
I wonder if we can do autotuning that will work reasonably well on all supported platforms.
The text was updated successfully, but these errors were encountered: