More general options like `basesize`

[oxinabox]

Several functions take a basesize option.

One main use of this is when threading to avoid the cost of @spawn dominating over the cost of the actual work.
Set it too low and @spawn cost dominates.
Set it too high, and if the work is uneven then some threads will be sitting around with nothing to do.

basesize makes it easy to specify if you know roughly how long each item should take to process.
Rule of thumb is something like set basesize such that processing that many takes about 1ms.

If one the other hand you don't really have good idea how long something is, but know how even it is. then something else is desired.
If it is expected to be exactly even then optimal is basesize = div(length(work), nthreads()).
If one wants to soften that because less confidant how ev en then perhaps:
basesize = div(length(work), 10nthreads()).

I am not sure the best way to expost this.
One option might be to have say basesize=0 or basesize=:even
todo the even splits.
I suspect even splits isn't a great option for default in the equal case anyway though,
since one might have a thread get taken by another process running (outside of dedicated machines).
Another might be basesize=:auto to do say basesize = div(length(work), 10nthreads()), which is probably a better bet than even.

Perhaps a fuller API would be useful.
say sizing taking a number of possible options like sizing = Basesize(1), or sizing = Even() or sizing = TimeEstimate(mean=0.1, std=0.5)

[tkf]

Perhaps a fuller API would be useful.
say sizing taking a number of possible options like sizing = Basesize(1), or sizing = Even() or sizing = TimeEstimate(mean=0.1, std=0.5)

I'm thinking of a similar API, too. In addition to these, sizing = OversubscribeBy(factor) to mean basesize = div(length(work), factor * nthreads()) could be useful. (Though not sure "oversubscribe" is the best word to use here.)

Another might be basesize=:auto to do say basesize = div(length(work), 10nthreads()), which is probably a better bet than even.

Yeah, I've been thinking to switch to use 5nthreads() or 10nthreads() or something like that by default.

[denius]

Is it possible to set a custom function/Iterator for basesize?
Thus, it will be possible to determine the exact granularity of work for each thread. For example, when working with matrices whose size is not a factor of nthreads, we can set something

threadrange(N) = UnitRange(1+div((threadid()-1)*N,nthreads()), div(threadid()*N,nthreads()))

and it will be similar to TiledIteration.jl.

[tkf]

Transducers.jl mainly uses the divide-and-conquer approach so the container size doesn't have to be the factor of nthreads. SplittablesBase.jl already supports N-dim AbstractArray which includes CartesianIndices. But yeah, it's nice to add more flexible options.