Possibly use rayon for the worker threads

[nicokoch]

I am developing a web app based on rocket and ws-rs (websockets).

It would be helpful if rocket provided the worker threads based on rayon, so that the websockets and web framework can share the same threadpool.

Rayon has the big advantage that any (dependency) crate will share the same thread pool, avoiding that each crate spawns X new threads that possibly end up doing close to nothing.

Maybe it's possible to create a feature for this?

[SergioBenitez]

I'm not sure I understand; why does this matter? Rocket isn't continuously spawning threads; it simply starts n threads (= to the workers config parameter) at launch time and sits in an accept loop in each of them. If there are no requests to serve, the threads do nothing, but at worst this results in a low amount of memory being consumed. Are you concerned about having a bit of memory allocated without being used?

[lnicola]

A fixed thread pool only works when the requests are CPU-bound. If you throw IO-bound requests in the mix they will tie the pool and starve other requests. This is made worse by not having async support.

I'm not sure rayon is quite there yet, but .NET's thread pool does a pretty cool thing where it continuously spawns and shuts threads down while monitoring the throughput: https://www.researchgate.net/publication/228977836_Optimizing_concurrency_levels_in_the_net_threadpool_A_case_study_of_controller_design_and_implementation.

[SergioBenitez]

A fixed thread pool only works when the requests are CPU-bound. If you throw IO-bound requests in the mix they will tie the pool and starve other requests. This is made worse by not having async support.

When you make a blocking I/O call, the OS deschedules the responsible thread and marks it as waiting. None of the other threads in the pool are blocked as a result of this. As long as there are threads available to handle requests, requests will be handled. If you're concerned about every thread being descheduled by the OS as a result of an I/O call, then this is a valid concern, but this has nothing to do with sharing or not sharing a thread-pool.

[lnicola]

If you're concerned about every thread being descheduled by the OS as a result of an I/O call

This. If the application is doing I/O, it's probably doing I/O on most of the requests it gets, while the thread pool is fixed and pretty small.

[SergioBenitez]

What does that have to do with sharing or not sharing a threadpool? This is simply an artifact of synchronous I/O, and it will continue to occur regardless of which threadpool we use.

[lnicola]

"Sufficiently smart" thread pools will ramp up the number of threads (up to a limit, of course) to absorb slower tasks. Of course, there is the workaround of starting Rocket with 200 workers or something like that, but it would be better to have something that can scale up when needed.

[SergioBenitez]

That's not how Rocket uses thread-pools, nor how thread-pools are usually used outside of the context of a task-based system, like Rayon. Rocket (Hyper, really) starts up N threads and sits in an accept() loop in each thread. That's it.

If we did have a "smart" thread-pool, we could change the way Rocket and Hyper work by "spawning" a new thread every time we want to try to accept() a new connection and call accept() there. We'd ideally be reusing an existing thread in the common case. The thread-pool would then limit how many real threads can be waiting on accept() (by limiting the number of successful spawn()s), and it can play fun tricks to maximize throughput and reduce latency.

That would be a fine route to take if we were dead-set on sticking to a synchronous server. But we'll definitely be migrating Rocket to an asynchronous solution (see #17), so I don't see any benefit in trying to be overly clever to solve a problem that will inevitably be solved later.

[lnicola]

I don't see any benefit in trying to be overly clever to solve a problem that will inevitably be solved later.

I agree that it's not worth in if there are plans to migrate to async soon. A better thread pool might still be useful afterwards, but that's futures-cpupool territory, not Rocket.

[nicokoch]

This is not about ressources.

I guess what I'm actually asking for is access to the thread pool in order to share the threads between ws and rocket.
On the one hand, this feels "cleaner".
The other advantage is I have a single rocket.toml configuration file, and do not need a second one to configure the second thread pool.

Rayon felt like the natural choice (because it has the ability to share a thread pool between all crates), but I actually don't really care about the underlying technology.

[lnicola]

@nicokoch depending on the application and the thread pool implementation, you might not want to share the same one for different libraries.

In the case of Rocket, like @SergioBenitez mentioned, the thread pool comes almost for free and might change when switching to async Hyper, so it's probably not worth the hassle at this point.

[SergioBenitez]

Ah! Perhaps you want Rocket to handle websockets directly, @nicokoch? If so, it will! But not yet; #90 is tracking that. In terms of other kinds of thread-pool sharing, it's not something I see Rocket doing. The benefit simply isn't there. For that reason, I'm closing this issue as declined. Thank you for opening this up, however! It's appreciated. :)

[nicokoch]

Having rocket handling websockets directly would indeed be even better.
I just wasn't aware that this was planned.

Thanks for pointing me to the issue :)