Find appropriate number of workers for the worker pool

[ivan-aksamentov]

Currently workers don't use all available resources.

First of all, the size of the worker pool is hard limited to 4:
https://github.com/neherlab/webclades/blob/45b9930b819e3ffec60b0f01be53a844944a1e1c/packages/web/src/workers/createWorkerPools.ts#L8

Secondly, heavy rendering on main thread delays workers from getting work items, which causes unnecessary idling in worker processes.

The reason is that the task queues which feed the workers is in the memory of the main process, along with rendering. If main process is busy (with rendering SVGs in sequence views for example), free workers cannot pop tasks from the queue.

Ultimately, there is no currently a solution for that in JS, because there is no shared memory (where the queue would be stored and accessible to all threads/processes at all times, like in native environments)

The problem can be mitigated by oversubscribing the available resources. For example, you may create 8 workers, even if there are only 4 physical processors. This way workers will compete for resources more aggressively, and if 4 of the workers are done with their tasks and are unable to get new work items, there are still 4 others that haven't finished and they will keep all 4 CPUs busy. This basically abuses the OS scheduler, using it as an additional task queuing mechanism (where items are temporarily stored in preempted worker's memory until it is scheduled for execution again)

The disadvantage of course is that too much oversubscription may lead to too much context switching overhead. That is, workers will be scheduled and unscheduled periodically to run on cores, so that each can make progress. This context switching takes time and wastes CPU cycles. And there is no way for us to set any scheduling priorities.

So there is a balance we should find when choosing the number of workers. I think that the navigator.hardwareConcurrency + 2 is a safe bet. We may also expose this setting to users.

There is an experimental work on SharedArrayBuffer happening in Mozilla for a few years
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/SharedArrayBuffer/Planned_changes
It does not seem production ready and AFAIK there were some serious security concerns in the past. Neither Chrome, nor our library (Threads.js) supports it currently. So we probably don't want to go there quite yet.