Logstash should exit with non-zero exit code if an exception is thrown by a plugin in register method

[breml]

If a plugin throws an exception in the register method, an "Pipeline aborted due to error" occurs and Logstash is stopped. Unfortunately, the exit code for Logstash in this case is still 0.

• Version: 5.2.1, 2.4.1
• Operating System: Linux
• Config File (if you have sensitive info, please remove it):

input { stdin { } } 
filter { grok { match => { "message" => [ "%{NOT_DEFINED:alt_message}" ] } } } 
output { stdout {} }

• Sample Data: n/a
• Steps to Reproduce:

Run the following command:

logstash-5.2.1/bin/logstash -e 'input { stdin { } } filter { grok { match => { "message" => [ "%{NOT_DEFINED:alt_message}" ] } } } output { stdout {} }'; echo $?

or

logstash-2.4.1/bin/logstash -e 'input { stdin { } } filter { grok { match => { "message" => [ "%{NOT_DEFINED:alt_message}" ] } } } output { stdout {} }'; echo $?

In the above example, the register method of the logstash-filter-grok fails, because the pattern NOT_DEFINED does not exist, which leads to a compile error of the grok pattern.

Current output:

Sending Logstash's logs to /home/lubr/code/logstash/logstash-5.2.1/logs which is now configured via log4j2.properties
The stdin plugin is now waiting for input:
[2017-02-15T09:29:15,152][ERROR][logstash.agent           ] Pipeline aborted due to error {:exception=>#<Grok::PatternError: pattern %{NOT_DEFINED:alt_message} not defined>, :backtrace=>["/home/lubr/code/logstash/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:123:in `compile'", "org/jruby/RubyKernel.java:1479:in `loop'", "/home/lubr/code/logstash/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/jls-grok-0.11.4/lib/grok-pure.rb:93:in `compile'", "/home/lubr/code/logstash/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:274:in `register'", "org/jruby/RubyArray.java:1613:in `each'", "/home/lubr/code/logstash/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:269:in `register'", "org/jruby/RubyHash.java:1342:in `each'", "/home/lubr/code/logstash/logstash-5.2.1/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.3.1/lib/logstash/filters/grok.rb:264:in `register'", "/home/lubr/code/logstash/logstash-5.2.1/logstash-core/lib/logstash/pipeline.rb:235:in `start_workers'", "org/jruby/RubyArray.java:1613:in `each'", "/home/lubr/code/logstash/logstash-5.2.1/logstash-core/lib/logstash/pipeline.rb:235:in `start_workers'", "/home/lubr/code/logstash/logstash-5.2.1/logstash-core/lib/logstash/pipeline.rb:188:in `run'", "/home/lubr/code/logstash/logstash-5.2.1/logstash-core/lib/logstash/agent.rb:302:in `start_pipeline'"]}
[2017-02-15T09:29:15,235][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-02-15T09:29:18,189][WARN ][logstash.agent           ] stopping pipeline {:id=>"main"}
0

Expected output:
Logstash should return a non-zero exit code.

[ph]

We effectively should return the right error code at that moment, I will check with my agent refactor in #6632

It might not be easy to detect since this error is happening outside of the main thread.

[breml]

@ph As I can see, you merged #6632 in 645fcec. Did you include the fix for this issue as well?

[jsvd]

this is fixed in the upcoming 6:

 /tmp/logstash-6.0.0-beta1 % bin/logstash -e 'input { stdin { } } filter { grok { match => { "message" => [ "%{NOT_DEFINED:alt_message}" ] } } } output { stdout {} }'; echo $?


/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:108: warning: already initialized constant DEFAULT_MAX_POOL_SIZE
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:110: warning: already initialized constant DEFAULT_REQUEST_TIMEOUT
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:111: warning: already initialized constant DEFAULT_SOCKET_TIMEOUT
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:112: warning: already initialized constant DEFAULT_CONNECT_TIMEOUT
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:113: warning: already initialized constant DEFAULT_MAX_REDIRECTS
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:114: warning: already initialized constant DEFAULT_EXPECT_CONTINUE
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:115: warning: already initialized constant DEFAULT_STALE_CHECK
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:590: warning: already initialized constant ISO_8859_1
/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/manticore-0.6.1-java/lib/manticore/client.rb:641: warning: already initialized constant KEY_EXTRACTION_REGEXP
Sending Logstash's logs to /tmp/logstash-6.0.0-beta1/logs which is now configured via log4j2.properties
[2017-08-04T16:42:35,204][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.queue", :path=>"/tmp/logstash-6.0.0-beta1/data/queue"}
[2017-08-04T16:42:35,215][INFO ][logstash.setting.writabledirectory] Creating directory {:setting=>"path.dead_letter_queue", :path=>"/tmp/logstash-6.0.0-beta1/data/dead_letter_queue"}
[2017-08-04T16:42:35,260][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
[2017-08-04T16:42:35,307][INFO ][logstash.agent           ] No persistent UUID file found. Generating new UUID {:uuid=>"1283f185-eaec-4731-bb01-6f4245e7c9e9", :path=>"/tmp/logstash-6.0.0-beta1/data/uuid"}
[2017-08-04T16:42:35,789][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
[2017-08-04T16:42:37,610][ERROR][logstash.pipeline        ] Error registering plugin {:pipeline_id=>"main", :plugin=>"#<LogStash::FilterDelegator:0x65940edb @id=\"bc761fff241ebf41977aab9155a49cec806acde0-2\", @klass=LogStash::Filters::Grok, @metric_events=#<LogStash::Instrument::NamespacedMetric:0x4f3dfd5b @metric=#<LogStash::Instrument::Metric:0x6d8fd6c @collector=#<LogStash::Instrument::Collector:0x64cf833 @agent=nil, @metric_store=#<LogStash::Instrument::MetricStore:0x436bccf7 @store=#<Concurrent::Map:0x00000000000fb0 entries=3 default_proc=nil>, @structured_lookup_mutex=#<Mutex:0x639c26e4>, @fast_lookup=#<Concurrent::Map:0x00000000000fb4 entries=54 default_proc=nil>>>>, @namespace_name=[:stats, :pipelines, :main, :plugins, :filters, :\"bc761fff241ebf41977aab9155a49cec806acde0-2\", :events]>, @logger=#<LogStash::Logging::Logger:0x43026d68 @logger=#<Java::OrgApacheLoggingLog4jCore::Logger:0x5e930d8b>>, @filter=<LogStash::Filters::Grok match=>{\"message\"=>[\"%{NOT_DEFINED:alt_message}\"]}, id=>\"bc761fff241ebf41977aab9155a49cec806acde0-2\", enable_metric=>true, periodic_flush=>false, patterns_files_glob=>\"*\", break_on_match=>true, named_captures_only=>true, keep_empty_captures=>false, tag_on_failure=>[\"_grokparsefailure\"], timeout_millis=>30000, tag_on_timeout=>\"_groktimeout\">>", :error=>"pattern %{NOT_DEFINED:alt_message} not defined", :thread=>"#<Thread:0xaa149fe@/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:278 run>"}
[2017-08-04T16:42:37,619][ERROR][logstash.pipeline        ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<Grok::PatternError: pattern %{NOT_DEFINED:alt_message} not defined>, :backtrace=>["/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.4/lib/grok-pure.rb:123:in `block in compile'", "org/jruby/RubyKernel.java:1298:in `loop'", "/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/jls-grok-0.11.4/lib/grok-pure.rb:93:in `compile'", "/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-3.4.2/lib/logstash/filters/grok.rb:286:in `block in register'", "org/jruby/RubyArray.java:1734:in `each'", "/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-3.4.2/lib/logstash/filters/grok.rb:280:in `block in register'", "org/jruby/RubyHash.java:1343:in `each'", "/tmp/logstash-6.0.0-beta1/vendor/bundle/jruby/2.3.0/gems/logstash-filter-grok-3.4.2/lib/logstash/filters/grok.rb:275:in `register'", "/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:376:in `register_plugin'", "/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:387:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:387:in `register_plugins'", "/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:397:in `start_workers'", "/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:321:in `run'", "/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:281:in `block in start'"], :thread=>"#<Thread:0xaa149fe@/tmp/logstash-6.0.0-beta1/logstash-core/lib/logstash/pipeline.rb:278 run>"}
[2017-08-04T16:42:37,637][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: LogStash::PipelineAction::Create/pipeline_id:main, action_result: false"}
bin/logstash -e   76.66s user 3.83s system 139% cpu 57.607 total
1