Need Threadpool minThreads config setting

[benaadams]

ThreadPool.SetMinThreads doesn't exist as an api in coreclr and apparently its available and the responsibility of the hosting model https://github.com/dotnet/coreclr/issues/1606#issuecomment-144126757

So this should be exposed via a config to the dotnet cli

[TheRealPiotrP]

@benaadams is this blocking Kestrel?

@schellap, can you take a look?

[benaadams]

@piotrpMSFT depends, might block how users use Kestrel - if that's not too enigmatic 😁

@halter73 is probably better to comment on it.

[davidfowl]

Not kestrel, it's server scenarios in general.

[schellap]

Can you try specifying this as a workaround?

export ComPlus_ThreadPool_ForceMinWorkerThreads=2
set ComPlus_ThreadPool_ForceMinWorkerThreads=2

See ThreadPool_* here:
https://github.com/dotnet/coreclr/blob/master/Documentation/project-docs/clr-configuration-knobs.md

@gkhanna79, the configuration knobs for CoreCLR need to be exposed by the corhost interface or through the PropertyKeys/PropertyValues to coreclr_initialize. Is someone working on this? Or is the host or the user supposed to set this as an env variable for own process?

Cc @ericeil

[benaadams]

@schellap yep that works used

set ComPlus_ThreadPool_ForceMinWorkerThreads=1000

For 1024 of synchronous blocking type connections and it worked happily

[gkhanna79]

@schellap COMPlus* configuration settings are intended to come from environment variables. However, the question about exposing ThreadPool configuration (either by adding a setting in runtime.config.json or FX API) is something @sergiy-k should comment on.

[benaadams]

@gkhanna79 a config or api approach would be better than environment as its more likely to travel well with the component that has the constraint - whereas environment var feels like it needs to rely on good documentation/process and it being followed well.

If I had to choose only one, config would probably be best; as it can be retro-actively be applied to a deployed app - without recompilation to include the increased threadpool count. (in a something's gone wrong in production panic moment)

[sergiy-k]

If exposing this setting via runtime config file works for ASP.NET then I would prefer to go with this approach.

@kouvel, could you please take a closer look at this and see if there are any problems with exposing this option in the hosting APIs?
Also please see: https://github.com/dotnet/coreclr/issues/1606

[ericeil]

In general, the trouble with SetMinThreads is that it depends on the underlying thread pool having a notion of "min threads" in the first place, which not all do.

Which thread pool are we using in CoreRT?

Maybe this would be fine as a config option as long as it was clear that it was configuring the CLR thread pool, and has no effect if a different thread pool is being used (i.e., if the runtime is not CoreCLR).

[schellap]

For CoreRT we would have a separate set of nativeOptions and for CoreCLR runtimeOptions in the runtime.config.json.

[schellap]

@kouvel, hopefully we can pass on other CLRConfigValues too (ints and strings) than the current startup flags approach for bool values ex: server_gc.

[kouvel]

Yea it should be fine to pass ints and strings as well. I'll take a look.

[benaadams]

is that it depends on the underlying thread pool having a notion of "min threads" in the first place, which not all do.

Are the characteristics of these threadpools known? Will they be subject to jamming issues when too many interdependent things are blocked and their unblockers queued?

[kouvel]

CC @ericeil

I just want to bring this up again. Why not add the APIs SetMinThreads and SetMaxThreads?

My thoughts:

• It seems like the scenario requires this configuration, regardless of which thread pool is being used. The .NET Native thread pool may not support it yet, but if it will be used for server scenarios in the future, wouldn't it be necessary for the .NET Native thread to support it as well? Until then, perhaps it could just throw NotSupportedException.
• The user would know best how to configure this, based on machine configuration, load distribution, etc. Whether it is through host config or server app config, I guess both would work, but it seems the app would be the natural thing to configure this based on its user-provided configuration.
• While the thread pool is ideally used for short-running CPU-bound work items, I'm sure that is not the case in practice and can't be expected to be true. The TPL uses the thread pool, and async tasks can include disk or network IO. The app doesn't necessarily know whether a work item is going to be long-running to flag the task as such (depends on network conditions, size of database query results, etc.), and always flagging network tasks as long-running may not be feasible.

So, it seems to me like a fair piece of configuration to support for any thread pool.

[kouvel]

Also, not sure if this is done already, but perhaps the thread pool could be improved to track which of its scheduled threads are in a wait state and schedule new threads more quickly, to keep the CPU as busy as possible. When threads go out of the wait state, there may be more threads than is ideal, which are all CPU-bound, but as a heuristic it may work better than scheduling one thread per second. If all scheduled threads are not in a wait state and are long-running, I guess it would be a choice whether to start new threads to release waiting work items (perhaps one per second, or some other heuristic) or to block pending work items until a thread becomes available.

[kouvel]

MinThreads and MaxThreads are now exposed in the app config, see https://github.com/dotnet/cli/blob/rel/1.0.0/Documentation/specs/runtime-configuration-file.md and https://github.com/dotnet/coreclr/blob/master/Documentation/project-docs/clr-configuration-knobs.md.