Sane defaults / output worker setup for Logstash 2.2 ng_pipeline

[andrewvc]

Now that the ng_pipeline branch has been merged into logstash 2.x, which is destined for our 2.2 release now is a good time to reconsider some default behavior that could be improved for the ng_pipeline.

1. We should reconsider our default worker policy of 50% of cores. I propose that we raise this to 100% of cores. Remember that now the workers spend some time blocking on IO when performing an output operation (esp. over a network). Many users will see optimal numbers with more workers than CPU cores in fact! Benchmark results can be seen on this issue.
2. We should consider defaulting the number of worker-safe plugins to be == the number of workers. In most cases this will give the optimal result with the new pipeline. We could, alternatively, consider creating a 'threadsafe' worker option, where a worker could declare itself threadsafe and share itself across all worker threads. This is 'cleaner', but probably higher effort, and maybe could be done as a next step for 3.0.

[suyograo]

Fixed in #4391