Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

JRuby hbase plugins #41

Merged
merged 1 commit into from

4 participants

@abhijith
  • check hbase status
  • check number of regions on a regionserver

There's a bit of code common to both the plugins. Need to figure out a way to do factor it out. Ideas?

@portertech
Owner

@abhijith Nice work! Is there a reason you didn't use the sensu-plugin gem? It would provide cli arg parsing (w/ a bit of self documenting), and cut down the boilerplate code in both plugins. Would it be possible to provide some sample output in your environment? This one is kinda hard for me to test :) Thanks!

@abhijith

@portertech Thanks!

I have written very little JRuby code. So I am not sure how to go about using gems with it. Is it possible to use the sensu-plugin gem with JRuby? If so, how do I go about it?

@abhijith

Nevermind the previous comment. Figured it out.

However, using sensu-plugin is not straightforward in this case.

Following are the complications:

  • chd3u3 ships with jruby-1.6.0 which has a bug which shows up during gem installation (Strangely, it does not affect all gems). JRuby 1.6.0 jar will have to be replaced with 1.6.2 which contains the fix for the bug.

http://jira.codehaus.org/browse/JRUBY-5581

Steps to be followed to get his working:

a) wget http://jruby.org.s3.amazonaws.com/downloads/1.6.2/jruby-complete-1.6.2.jar

b) mv jruby-complete-1.6.2.jar /usr/lib/hbase/lib/jruby-complete-1.x.x.jar

  • The plugin needs to explicitly add the gem path for mixlib-cli and sensu-plugin like so:

$:.push Dir.glob("/var/lib/gems/1.8/gems/mixlib-cli*/lib")
$:.push Dir.glob("/var/lib/gems/1.8/gems/sensu-plugin*/lib")
$:.flatten!

This makes assumptions about both the platform and the gem directory path.

Although using sensu-plugin removes the redundant code, it makes the plugins harder to use by introducing manual steps. Let me know if there is a better way to solve the above problems.

Link to the version using sensu-plugin: https://gist.github.com/2375847

@decklin

Hi - I'd definitely like sensu-plugin to be usable with JRuby. If think opening an issue in its repo would be appropriate, please go ahead.

I'm not sure I understand the 1.6.0 bug -- is adding the /var/lib/gems/1.8/gems path to $: just a workaround for this? A quick googling brings up http://kenai.com/projects/jruby/pages/GettingStarted#Installing_and_Using_Ruby_Gems which seems to imply to me that JRuby would install gems somewhere that it controls, not to /var/lib/gems/1.8 (looks like the typical Debian/Ubuntu path). Is that right?

@abhijith

@decklin: The JRuby bug shows up when we try to install gems. I don't think this has got anything to do with sensu-plugin as such. This seems to be affecting lot of other gems too.
The problem is in https://github.com/olabini/yecht which has been fixed and bundled with newer version of JRuby which explains why senu-plugin works with JRuby 1.6.2.

Adding /var/lib/gems/1.8/gems to path is not a workaround for the bug. This is a separate step that is required if we have to use gems which are not bundled with JRuby. This would hold true even if the JRuby bug hadn't existed.
Note that, we are exec'ing the plugin using "#!/usr/bin/hbase org.jruby.Main". This will look inside the jar for gems which have been "require'd" and won't find it. Hence, we need to explicitly add /var/lib/gems/1.8... to $:

And yes, JRuby would install gems to a different path.

NOTE: I am very new to JRuby and still trying to find my way around. Please feel free to comment if you think I am missing something obvious.

@portertech
Owner

This plugin can potentially be an executable Java JAR, using warbler (w/ bundler support) to vendor everything.

@nstielau
Collaborator

Let's get this merged or closed. I vote that if it works for you @abhijith we should merge it. If there is any weirdness with JRuby still, let's either add an EXPERIMENTAL note in the header, or more explicitly move it to a plugins/experimental/hbase/ directory.

@portertech
Owner

JRuby 1.7 is now available as well, the plugin looks pretty clean, :+1:

@abhijith

@nstielau: works fine. we've been using it in production for about a year now. I guess its good enough to be merged in with a 'experimental' tag like time you've mentioned.

@abhijith abhijith closed this
@abhijith abhijith reopened this
@abhijith

Oops clicked the close button by mistake

@portertech portertech merged commit 4e7ccb4 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 9, 2012
This page is out of date. Refresh to see the latest.
View
114 plugins/hbase/check-hbase-regions.rb
@@ -0,0 +1,114 @@
+#!/usr/bin/hbase org.jruby.Main
+#
+# HBase regions plugin
+# ===
+#
+# This plugin checks the number of regions on a regionserver
+#
+# Copyright 2011 Runa Inc
+#
+# Released under the same terms as Sensu (the MIT license); see LICENSE
+# for details.
+
+require 'java'
+require 'pp'
+
+include Java
+include_class('java.lang.Integer') { |package,name| "J#{name}" }
+include_class('java.lang.Long') { |package,name| "J#{name}" }
+include_class('java.lang.Boolean') { |package,name| "J#{name}" }
+
+import org.apache.hadoop.hbase.client.HBaseAdmin
+import org.apache.hadoop.hbase.client.HTable
+import org.apache.hadoop.hbase.HBaseConfiguration
+import org.apache.hadoop.hbase.util.Bytes
+import org.apache.log4j.Logger
+
+
+packages = ["org.apache.zookeeper", "org.apache.hadoop", "org.apache.hadoop.hbase"]
+
+packages.each do |package|
+ logger = org.apache.log4j.Logger.getLogger(package)
+ logger.setLevel(org.apache.log4j.Level::ERROR);
+end
+
+module SensuUtils
+ # Copied from sensu-plugin
+
+ EXIT_CODES = {
+ 'OK' => 0,
+ 'WARNING' => 1,
+ 'CRITICAL' => 2,
+ 'UNKNOWN' => 3,
+ }
+
+ def output(fn, *args)
+ puts "#{fn.upcase}: #{args}"
+ end
+
+ EXIT_CODES.each do |status, code|
+ define_method(status.downcase) do |*args|
+ output(status, *args)
+ exit(code)
+ end
+ end
+
+end
+
+include SensuUtils
+
+def regionserver_info
+ conf = HBaseConfiguration.new
+ admin = HBaseAdmin.new(conf)
+
+ status = admin.getClusterStatus()
+ status.getServerInfo().map { |server|
+ { :hostname => server.getServerAddress().getHostname(),
+ :regions => server.getLoad().getNumberOfRegions()
+ }
+ }
+end
+
+def check_threshold(info)
+ case info[:regions]
+ when config[:ok]..config[:warning]
+ { :status => :ok, :msg => "Regions: #{info.inspect}" }
+ when config[:warning]..config[:critical]
+ { :status => :warning, :msg => "Regions: #{info.inspect}" }
+ else
+ { :status => :critical, :msg => "Regions: #{info.inspect}" }
+ end
+end
+
+$config = { :ok => 0, :warning => 900, :critical => 1000 }
+
+class Array
+ def second
+ self[1]
+ end
+end
+
+def config
+ $config
+end
+
+def run
+ status = regionserver_info.map { |x| check_threshold(x) }
+
+ msg = "\n" + status.map { |x| x[:msg] }.join("\n")
+
+ if status.any? { |x| x[:status] == :critical }
+ critical msg
+ elsif status.any? { |x| x[:status] == :warning }
+ warning msg
+ else
+ ok msg
+ end
+
+end
+
+
+$config[:warning] = ARGV.first.to_i if ARGV.first
+$config[:critical] = ARGV.second.to_i if ARGV.second
+
+run
View
80 plugins/hbase/check-hbase-status.rb
@@ -0,0 +1,80 @@
+#!/usr/bin/hbase org.jruby.Main
+#
+# HBase status plugin
+# ===
+#
+# This plugin checks if any of the regionservers are down
+#
+# Copyright 2011 Runa Inc
+#
+# Released under the same terms as Sensu (the MIT license); see LICENSE
+# for details.
+
+require 'java'
+require 'pp'
+
+include Java
+include_class('java.lang.Integer') { |package,name| "J#{name}" }
+include_class('java.lang.Long') { |package,name| "J#{name}" }
+include_class('java.lang.Boolean') { |package,name| "J#{name}" }
+
+import org.apache.hadoop.hbase.client.HBaseAdmin
+import org.apache.hadoop.hbase.client.HTable
+import org.apache.hadoop.hbase.HBaseConfiguration
+import org.apache.hadoop.hbase.util.Bytes
+import org.apache.log4j.Logger
+
+
+packages = ["org.apache.zookeeper", "org.apache.hadoop", "org.apache.hadoop.hbase"]
+
+packages.each do |package|
+ logger = org.apache.log4j.Logger.getLogger(package)
+ logger.setLevel(org.apache.log4j.Level::ERROR);
+end
+
+
+module SensuUtils
+ # Copied from sensu-plugin
+
+ EXIT_CODES = {
+ 'OK' => 0,
+ 'WARNING' => 1,
+ 'CRITICAL' => 2,
+ 'UNKNOWN' => 3,
+ }
+
+ def output(fn, *args)
+ puts "#{fn.upcase}: #{args}"
+ end
+
+ EXIT_CODES.each do |status, code|
+ define_method(status.downcase) do |*args|
+ output(status, *args)
+ exit(code)
+ end
+ end
+
+end
+
+
+include SensuUtils
+
+def check_hbase_status
+ conf = HBaseConfiguration.new
+ admin = HBaseAdmin.new(conf)
+
+ status = admin.getClusterStatus()
+ dead_servers = status.getDeadServerNames()
+
+ count = dead_servers.length
+
+ if count == 0
+ ok "Alive"
+ else
+ critical "Dead: #{dead_servers.join(" ")}"
+ end
+
+ unknown "No output from plugin"
+end
+
+check_hbase_status
Something went wrong with that request. Please try again.