Elastic threadpool RFC

[bnoordhuis]

Something I worked on over the weekend and would like to get some feedback on. The premise is that threads are started on demand and scale up to some TBD limit. I'll comment inline on places that still need improvement.

One benefit is that it makes it possible to use libuv in an environment that restricts threading.

[bnoordhuis]

Arbitrary timeout. Maybe this should be configurable with an option for 'never'.

[saghul]

I'll have a look!

[bnoordhuis]

@saghul WDYT?

[saghul]

Sorry, got distracted :-S I'll review tomorrow for sure.

[saghul]

Should uv_thread_self work if called from a worker thread? We use a TLS key for that, which wouldn't be setup in this case.

Maybe we can use uv_thread_create and fiddle with the internals here?

[bnoordhuis]

I didn't realize the TLS was used for that. There is one reason (well, two) why I implemented it like this:

1. uv_thread_create() on Windows creates the thread in suspended mode, then calls ResumeThread(). I don't quite understand why it does that so I decided not to mess with it.
2. uv_thread_create() calls malloc() and free() from different threads, which might be slow. For example, glibc has to acquire a process-global lock when the free happens on a different thread.

We can just try it and see how well it works.

[saghul]

on Windows creates the thread in suspended mode, then calls ResumeThread(). I don't quite understand why it does that so I decided not to mess with it.

It does it so it can store the thread handle as the current thread in TLS.

We can just try it and see how well it works.

Sounds reasonable.

[saghul]

Left some comments, but the idea SGTM. Some pseudo-random thoughts on the matter:

• should we have a min_threads concept as well? That is, we can scale elastically, but always keep a set of threads available, to avoid the overhead of always having to create one if operations in the threadpool are sporadic.
• since the threadpool would now be elastic, why have a max threads clamp?
• how about some uv_threadpool_resize API to change the min and max threads at runtime?

[bnoordhuis]

should we have a min_threads concept as well? That is, we can scale elastically, but always keep a set of threads
how about some uv_threadpool_resize API to change the min and max threads at runtime?

That sounds reasonable.

since the threadpool would now be elastic, why have a max threads clamp?

To stop runaway thread creation? Most operating systems can only handle a limited number of threads before performance and throughput (both in-process and system-wide) starts to seriously degrade.

An unlimited number of threads also isn't that useful if the individual threads are mostly CPU-bound (which they normally aren't with internal work but can be for work that comes in through uv_queue_work().)

[saghul]

To stop runaway thread creation? Most operating systems can only handle a limited number of threads before performance and throughput (both in-process and system-wide) starts to seriously degrade.

Just to clarify, I mean this clamp:

#define MAX_THREADPOOL_SIZE 128

If the user wants to set the maximum to 256 and bring their systems to a halt it's all good by me :-) I'd go with a note in the documentation.

[saghul]

Closing this in favor of #267. @bnoordhuis will be writing a new LEP.