Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5.0.0-rc1 Cannot load JDBC Plugin #6019

Closed
imw opened this issue Oct 10, 2016 · 2 comments
Closed

5.0.0-rc1 Cannot load JDBC Plugin #6019

imw opened this issue Oct 10, 2016 · 2 comments

Comments

@imw
Copy link

imw commented Oct 10, 2016

Having done
dpkg -i logstash-5.0.0-rc1.deb
and
/usr/share/logstash/bin/logstash-plugin install logstash-input-jdbc

when I attempt to run jdbc input with the following:

cat /etc/logstash/logstash.yml | sed '/^#/ d'

node.name: esclient0
path.data: /var/lib/logstash
path.config: /etc/logstash/conf.d
config.test_and_exit: false
log.level: debug
path.logs: /var/log/logstash

cat /etc/logstash/conf.d/jdbc.conf

input {
jdbc {
jdbc_driver_library => "/usr/share/java/mysql-connector-java-5.1.38.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://XXXXXXXXXXX:3306/XXXXXXXXX"
jdbc_user => "XXXXXXXXX"
jdbc_password => "XXXXXXXXX"
jdbc_paging_enabled => true
jdbc_page_size => 10000
statement => "SELECT id, publish_ts, source, title, UNCOMPRESS(body), link FROM data_archive"
}
}
output {
stdout { codec => json_lines }
}

I get:
[2016-10-10T20:34:21,479][ERROR][logstash.agent ] fetched an invalid config {:config=>"input {\n jdbc {\n jdbc_driver_library => "/usr/share/java/mysql-connector-java-5.1.38.jar"\n jdbc_driver_class => "com.mysql.jdbc.Driver"\n jdbc_connection_string => "jdbc:mysql://XXXXX:3306/XXXXXX"\n jdbc_user => "XXXXXXX"\n\tjdbc_password => "XXXXXXXXX"\n\tjdbc_paging_enabled => true\n\tjdbc_page_size => 10000\n statement => "SELECT id, publish_ts, source, title, UNCOMPRESS(body), link FROM data_archive"\n }\n}\noutput {\n stdout { codec => json_lines }\n}\n\n", :reason=>"Couldn't find any input plugin named 'jdbc'. Are you sure this is correct? Trying to load the jdbc input plugin resulted in this error: Problems loading the requested plugin named jdbc of type input."}

I believe I have followed the published instructions. Any advice on how to proceed with debugging or identifying user error would be appreciated.

Please post all product and debugging questions on our forum. Your questions will reach our wider community members there, and if we confirm that there is a bug, then we can open a new issue here.

Logstash Plugins are located in a different organization: https://github.com/logstash-plugins. For bugs on specific Logstash plugins, for example, if Redis Output has a defect, please open it in the respective Redis Output repository.

For all general issues, please provide the following details for fast resolution:

  • Version: 5.0.0=rc1
  • Operating System: 4.4.0-38-generic ElasticSearch and AMQP enhancements #57-Ubuntu SMP Tue Sep 6 15:42:33 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  • Config File (if you have sensitive info, please remove it):
  • Sample Data:
  • Steps to Reproduce:
@suyograo
Copy link
Contributor

@imw this is a bug, and thanks for reporting it! 4.1.2 of JDBC input has been published. You can update to it manually using bin/logstash-plugin update logstash-input-jdbc. Otherwise, the next RC will include this fix. Thanks

andrewvc added a commit to andrewvc/logstash that referenced this issue Oct 10, 2016
This problem is seen here:
elastic#6019

This change will log extra information that will be helpful to the
debugging process.

Testing this out with jdbc-input 4.1.1 (cited in elastic#6019 above) we now see
as output:

```
~/p/l/logstash-alt (better_plugin_error_messages) $ bin/logstash -e "input { jdbc {} } output { stdout {} }"
Sending Logstash logs to /Users/andrewvc/projects/lsp/logstash-alt/logs which is now configured via log4j2.properties.
[2016-10-10T16:38:50,723][WARN ][logstash.registry        ] Problems loading a plugin with {:type=>"input", :name=>#<LogStash::Registry::Plugin:0x50ec4e78 @type="input", @name="jdbc">, :path=>"logstash/inputs/jdbc", :error_message=>"uninitialized constant LogStash::PluginMixins::Jdbc::Cabin", :error_class=>NameError, :error_backtrace=>["org/jruby/RubyModule.java:2719:in `const_missing'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/plugin_mixins/jdbc.rb:11:in `Jdbc'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/plugin_mixins/jdbc.rb:9:in `(root)'", "org/jruby/RubyKernel.java:1040:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/inputs/jdbc.rb:1:in `(root)'", "org/jruby/RubyKernel.java:1040:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/inputs/jdbc.rb:4:in `(root)'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugins/registry.rb:1:in `(root)'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugins/registry.rb:59:in `lookup'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugin.rb:121:in `lookup'", "org/jruby/RubyKernel.java:1079:in `eval'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/pipeline.rb:418:in `plugin'", "(eval):8:in `initialize'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/agent.rb:195:in `create_pipeline'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/agent.rb:87:in `register_pipeline'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/runner.rb:256:in `execute'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/Users/andrewvc/projects/lsp/logstash-alt/lib/bootstrap/environment.rb:68:in `(root)'"]}
[2016-10-10T16:38:50,735][ERROR][logstash.agent           ] fetched an invalid config {:config=>"input { jdbc {} } output { stdout {} }", :reason=>"Couldn't find any input plugin named 'jdbc'. Are you sure this is correct? Trying to load the jdbc input plugin resulted in this error: Problems loading the requested plugin named jdbc of type input. Error: NameError uninitialized constant LogStash::PluginMixins::Jdbc::Cabin"}
```
suyograo added a commit to suyograo/logstash that referenced this issue Oct 10, 2016
suyograo pushed a commit that referenced this issue Oct 10, 2016
elasticsearch-bot pushed a commit that referenced this issue Oct 11, 2016
This problem is seen here:
#6019

This change will log extra information that will be helpful to the
debugging process.

Testing this out with jdbc-input 4.1.1 (cited in #6019 above) we now see
as output:

```
~/p/l/logstash-alt (better_plugin_error_messages) $ bin/logstash -e "input { jdbc {} } output { stdout {} }"
Sending Logstash logs to /Users/andrewvc/projects/lsp/logstash-alt/logs which is now configured via log4j2.properties.
[2016-10-10T16:38:50,723][WARN ][logstash.registry        ] Problems loading a plugin with {:type=>"input", :name=>#<LogStash::Registry::Plugin:0x50ec4e78 @type="input", @name="jdbc">, :path=>"logstash/inputs/jdbc", :error_message=>"uninitialized constant LogStash::PluginMixins::Jdbc::Cabin", :error_class=>NameError, :error_backtrace=>["org/jruby/RubyModule.java:2719:in `const_missing'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/plugin_mixins/jdbc.rb:11:in `Jdbc'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/plugin_mixins/jdbc.rb:9:in `(root)'", "org/jruby/RubyKernel.java:1040:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/inputs/jdbc.rb:1:in `(root)'", "org/jruby/RubyKernel.java:1040:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/inputs/jdbc.rb:4:in `(root)'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugins/registry.rb:1:in `(root)'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugins/registry.rb:59:in `lookup'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugin.rb:121:in `lookup'", "org/jruby/RubyKernel.java:1079:in `eval'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/pipeline.rb:418:in `plugin'", "(eval):8:in `initialize'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/agent.rb:195:in `create_pipeline'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/agent.rb:87:in `register_pipeline'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/runner.rb:256:in `execute'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/Users/andrewvc/projects/lsp/logstash-alt/lib/bootstrap/environment.rb:68:in `(root)'"]}
[2016-10-10T16:38:50,735][ERROR][logstash.agent           ] fetched an invalid config {:config=>"input { jdbc {} } output { stdout {} }", :reason=>"Couldn't find any input plugin named 'jdbc'. Are you sure this is correct? Trying to load the jdbc input plugin resulted in this error: Problems loading the requested plugin named jdbc of type input. Error: NameError uninitialized constant LogStash::PluginMixins::Jdbc::Cabin"}
```

Fixes #6020
elasticsearch-bot pushed a commit that referenced this issue Oct 11, 2016
This problem is seen here:
#6019

This change will log extra information that will be helpful to the
debugging process.

Testing this out with jdbc-input 4.1.1 (cited in #6019 above) we now see
as output:

```
~/p/l/logstash-alt (better_plugin_error_messages) $ bin/logstash -e "input { jdbc {} } output { stdout {} }"
Sending Logstash logs to /Users/andrewvc/projects/lsp/logstash-alt/logs which is now configured via log4j2.properties.
[2016-10-10T16:38:50,723][WARN ][logstash.registry        ] Problems loading a plugin with {:type=>"input", :name=>#<LogStash::Registry::Plugin:0x50ec4e78 @type="input", @name="jdbc">, :path=>"logstash/inputs/jdbc", :error_message=>"uninitialized constant LogStash::PluginMixins::Jdbc::Cabin", :error_class=>NameError, :error_backtrace=>["org/jruby/RubyModule.java:2719:in `const_missing'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/plugin_mixins/jdbc.rb:11:in `Jdbc'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/plugin_mixins/jdbc.rb:9:in `(root)'", "org/jruby/RubyKernel.java:1040:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/inputs/jdbc.rb:1:in `(root)'", "org/jruby/RubyKernel.java:1040:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/polyglot-0.3.5/lib/polyglot.rb:65:in `require'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-4.1.1/lib/logstash/inputs/jdbc.rb:4:in `(root)'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugins/registry.rb:1:in `(root)'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugins/registry.rb:59:in `lookup'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/plugin.rb:121:in `lookup'", "org/jruby/RubyKernel.java:1079:in `eval'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/pipeline.rb:418:in `plugin'", "(eval):8:in `initialize'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/pipeline.rb:90:in `initialize'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/agent.rb:195:in `create_pipeline'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/agent.rb:87:in `register_pipeline'", "/Users/andrewvc/projects/lsp/logstash-alt/logstash-core/lib/logstash/runner.rb:256:in `execute'", "/Users/andrewvc/projects/lsp/logstash-alt/vendor/bundle/jruby/1.9/gems/clamp-0.6.5/lib/clamp/command.rb:67:in `run'", "/Users/andrewvc/projects/lsp/logstash-alt/lib/bootstrap/environment.rb:68:in `(root)'"]}
[2016-10-10T16:38:50,735][ERROR][logstash.agent           ] fetched an invalid config {:config=>"input { jdbc {} } output { stdout {} }", :reason=>"Couldn't find any input plugin named 'jdbc'. Are you sure this is correct? Trying to load the jdbc input plugin resulted in this error: Problems loading the requested plugin named jdbc of type input. Error: NameError uninitialized constant LogStash::PluginMixins::Jdbc::Cabin"}
```

Fixes #6020
@suyograo
Copy link
Contributor

Fixed in #6021

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants