New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase workerThreads and completionPortThreads defaults for Helios #94

Closed
PinpointTownes opened this Issue Jul 1, 2014 · 12 comments

Comments

Projects
None yet
7 participants
@PinpointTownes
Copy link
Contributor

PinpointTownes commented Jul 1, 2014

The default values for the thread pool seem really low under Helios. I personally get workerThreads = completionPortThreads = 16 on a Core i7 machine with 4 cores and hyper-threading enabled (= 8 cores).

An ASP.NET Web Forms app running IIS Express on the same machine comes with far higher default values: workerThreads = 4096 and completionPortThreads = 1000.

You should consider increasing these values to get rid of these weird exceptions:

InvalidOperationException: There were not enough free threads in the ThreadPool to complete the operation.

In the meantime, manually calling ThreadPool.SetMaxThreads with higher values will do the trick.

@Eilon

This comment has been minimized.

Copy link
Member

Eilon commented Jul 1, 2014

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jul 1, 2014

I'd guess this is not Helios specific, but regarding KRE and how it sets (or doesn't set) defaults when it spins up the CLR.

@PinpointTownes

This comment has been minimized.

Copy link
Contributor

PinpointTownes commented Jul 1, 2014

@pksorensen on JabbR had similar issues with the "Katana version" of Helios and that's why I immediately thought it was Helios and not KRE. I'll ask him to check his TP values.

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jul 1, 2014

The OWIN/Katana Helios starts the CLR itself, so yes it would be responsible for setting these kinds of defaults.

@PinpointTownes PinpointTownes changed the title Increase workerThreads and completionPortThreads defaults for Helios Increase workerThreads and completionPortThreads defaults Jul 1, 2014

@davidfowl

This comment has been minimized.

Copy link
Member

davidfowl commented Jul 9, 2014

@GrabYourPitchforks This is an issue with Helios specifically. WebEngine4 sets the limits to 2 * cpuCount and normally System.Web sets the limits to 1000 and 4096 (basically the autoConfig settings in web.config).

@PinpointTownes PinpointTownes changed the title Increase workerThreads and completionPortThreads defaults Increase workerThreads and completionPortThreads defaults in Helios Jul 9, 2014

@PinpointTownes PinpointTownes changed the title Increase workerThreads and completionPortThreads defaults in Helios Increase workerThreads and completionPortThreads defaults for Helios Jul 9, 2014

@Tratcher

This comment has been minimized.

Copy link
Member

Tratcher commented Jul 9, 2014

@davidfowl Wouldn't we want to do the same thing in Hosting?

@cdeutsch

This comment has been minimized.

Copy link

cdeutsch commented Sep 9, 2014

I've been running into the "There were not enough free threads in the ThreadPool to complete the operation" error a lot on a project at work.

This is likely the culprit: workerThreads = completionPortThreads = 4

Is there a recommendation for how to calculate those two values until a fix lands?

Thanks

@PinpointTownes

This comment has been minimized.

Copy link
Contributor

PinpointTownes commented Sep 9, 2014

@crdeutsch you can simply add ThreadPool.SetMaxThreads(4096, 1000) at the top of your Startup.Configure method to fix this issue (these are the default values used by System.Web, as said earlier by @davidfowl).

If you find these values a bit high, you can also limit the worker threads to 100 * cpu-count (that's how ProcessModelSection.DefaultMaxWorkerThreadsForAutoConfig computes the default value)

@pranavkm

This comment has been minimized.

Copy link
Member

pranavkm commented Sep 10, 2014

@davidfowl @yishaigalatzer says they also saw this being set in the machine.config. Presumably this is equivalent to setting this in root web.config.

@danroth27

This comment has been minimized.

Copy link
Member

danroth27 commented Mar 9, 2015

See aspnet/Helios@c9be2ee for the new defaults.

@danroth27 danroth27 closed this Mar 9, 2015

@PinpointTownes

This comment has been minimized.

Copy link
Contributor

PinpointTownes commented Mar 9, 2015

@danroth27 this repository is not public yet. That would be cool if you could directly share some of the details here.

@danroth27

This comment has been minimized.

Copy link
Member

danroth27 commented Mar 9, 2015

// When webengine4.dll first starts, it sets the max thread pool size to an artificially low number, and it depends 
// on System.Web.dll to set it back. Since we're replacing System.Web.dll, we need to perform this fixup manually. 
// For now we'll use 100 * numCPUs. 
int newLimits = 100 * Environment.ProcessorCount; // this is actually # cores (including hyperthreaded cores) 
int existingMaxWorkerThreads; 
int existingMaxIocpThreads; 
ThreadPool.GetMaxThreads(out existingMaxWorkerThreads, out existingMaxIocpThreads); 
ThreadPool.SetMaxThreads(Math.Max(newLimits, existingMaxWorkerThreads), Math.Max(newLimits, existingMaxIocpThreads)); 

natemcmaster added a commit that referenced this issue Nov 30, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment