Change threads
field behaviour
#399
Labels
enhancement
processors
Any tasks or issues relating specifically to processors
v4
An issue proposed for v4 of Benthos
Currently the
threads
field ofpipeline
is used to create an explicit number of processing threads. This can be useful as sometimes it's desired to limit the workload of a pipeline to one single logical thread (conditional sleeping), and sometimes it's useful to spawn a much larger number of threads than CPU cores if the workload is mostly IO bound (these are green threads anyway).However, I suspect that for the vast majority of users the best behaviour for them is to have as many processing threads as there are CPU cores. In this case our defaults are failing as one pipeline is almost never going to be the ideal number.
I'd like to keep this value as a configured field because that allows you to have different thread counts for isolated streams running in streams mode. Therefore I propose changing to default to
0
and documenting that a value of 0 means Benthos matches the number of logical CPU threads on the running machine.Since this would be a breaking change for some hypothetical configs I'll wait for Benthos v4 to implement this.
The text was updated successfully, but these errors were encountered: