Skip to content

Commit

Permalink
Revert to use only a single worker by default on outputs
Browse files Browse the repository at this point in the history
The move to auto-scale output workers was great in theory, but a lot of outputs
just weren't built to support it well, they often used too many resources or had
logical errors.
  • Loading branch information
andrewvc committed Mar 28, 2016
1 parent 08fdb0b commit 2248aae
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 6 deletions.
8 changes: 3 additions & 5 deletions logstash-core/lib/logstash/pipeline.rb
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ module LogStash; class Pipeline
:config_str,
:original_settings

DEFAULT_OUTPUT_WORKERS = 1

DEFAULT_SETTINGS = {
:default_pipeline_workers => LogStash::Config::CpuCoreStrategy.maximum,
:pipeline_batch_size => 125,
Expand Down Expand Up @@ -460,18 +462,14 @@ def plugin(plugin_type, name, *args)
klass = LogStash::Plugin.lookup(plugin_type, name)

if plugin_type == "output"
LogStash::OutputDelegator.new(@logger, klass, default_output_workers, pipeline_scoped_metric.namespace(:outputs), *args)
LogStash::OutputDelegator.new(@logger, klass, DEFAULT_OUTPUT_WORKERS, pipeline_scoped_metric.namespace(:outputs), *args)
elsif plugin_type == "filter"
LogStash::FilterDelegator.new(@logger, klass, pipeline_scoped_metric.namespace(:filters), *args)
else
klass.new(*args)
end
end

def default_output_workers
@settings[:pipeline_workers] || @settings[:default_pipeline_workers]
end

# for backward compatibility in devutils for the rspec helpers, this method is not used
# in the pipeline anymore.
def filter(event, &block)
Expand Down
2 changes: 1 addition & 1 deletion logstash-core/spec/logstash/pipeline_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,7 @@ class TestPipeline < LogStash::Pipeline
pipeline.run

expect(pipeline.outputs.size ).to eq(1)
expect(pipeline.outputs.first.workers.size ).to eq(pipeline.default_output_workers)
expect(pipeline.outputs.first.workers.size ).to eq(::LogStash::Pipeline::DEFAULT_OUTPUT_WORKERS)
expect(pipeline.outputs.first.workers.first.num_closes ).to eq(1)
end

Expand Down

0 comments on commit 2248aae

Please sign in to comment.