Add hbase jars to Oozie

[bijugs]

Currently Oozie doesn't get shipped with HBase jars. This I believe is to keep check on the Oozie war size. This enhancement is to add HBase jars to Oozie.

Following are the steps to add HBase jars to Oozie: The version used here is HDP 2.2

Prepare Oozie war

• cp /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war /tmp/
• mkdir /tmp/oozie-hbase-prep
• cd /tmp/oozie-hbase-prep
• jar -xf ../oozie.war
• cd /usr/hdp/2.2.0.0-2041/hbase/lib/
• cp hbase-common-0.98.4.2.2.0.0-2041-hadoop2.jar hbase-client-0.98.4.2.2.0.0-2041-hadoop2.jar - hbase-server-0.98.4.2.2.0.0-2041-hadoop2.jar hbase-protocol-0.98.4.2.2.0.0-2041-hadoop2.jar hbase-hadoop2-compat-0.98.4.2.2.0.0-2041-hadoop2.jar htrace-core-*.jar netty-3.6.6.Final.jar /tmp/oozie-hbase-prep/WEB-INF/lib/.
• cd /tmp/oozie-hbase-prep
• jar -cMf ../oozie.war *

Deploying updated war to Oozie

• service oozie stop
• Edit /etc/oozie/conf/oozie-site.xml and edit the property oozie.credentials.credentialclasses to include hbase credential handler class "hbase=org.apache.oozie.action.hadoop.HbaseCredentials"
• cp /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war /tmp/oozie.war1
• rm -fr /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war
• cp /tmp/oozie.war /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war
• rm -rf /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie
• service oozie start

[bijugs]

This is a sample recipe. This doesn't include the step to modify oozie-site.xml. Obviously it needs bit more enhancement wrt this to be idempotent. Need to be run on the node where oozie server recipe runs.

#
# Recipe to include hbase jar files into Oozie
#
%w{
hbase
}.each do |p|
  package p do
    action :install
  end
end

file "/tmp/oozie.war" do
  owner 'root'
  group 'root'
  mode 0744
  content ::File.open("/usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war").read
  action :create
end

directory '/tmp/oozie-hbase-prep' do
  owner 'root'
  group 'root'
  mode '0755'
  action :create
end

bash 'extract-oozie-war' do
  user "root"
  group "root"
  code <<-EOH
    cd /tmp/oozie-hbase-prep
    jar -xf ../oozie.war
  EOH
  action :run
end

bash 'copy-hbase-jars' do
  user "root"
  group "root"
  code <<-EOH
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/hbase-common-0.98.4.2.2.0.0-2041-hadoop2.jar /tmp/oozie-hbase-prep/WEB-INF/lib/hbase-common-0.98.4.2.2.0.0-2041-hadoop2.jar
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/hbase-client-0.98.4.2.2.0.0-2041-hadoop2.jar /tmp/oozie-hbase-prep/WEB-INF/lib/hbase-client-0.98.4.2.2.0.0-2041-hadoop2.jar
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/hbase-server-0.98.4.2.2.0.0-2041-hadoop2.jar /tmp/oozie-hbase-prep/WEB-INF/lib/hbase-server-0.98.4.2.2.0.0-2041-hadoop2.jar
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/hbase-protocol-0.98.4.2.2.0.0-2041-hadoop2.jar /tmp/oozie-hbase-prep/WEB-INF/lib/hbase-protocol-0.98.4.2.2.0.0-2041-hadoop2.jar
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/hbase-hadoop2-compat-0.98.4.2.2.0.0-2041-hadoop2.jar /tmp/oozie-hbase-prep/WEB-INF/lib/hbase-hadoop2-compat-0.98.4.2.2.0.0-2041-hadoop2.jar
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/htrace-core-*.jar /tmp/oozie-hbase-prep/WEB-INF/lib/htrace-core-*.jar
    cp /usr/hdp/2.2.0.0-2041/hbase/lib/netty-3.6.6.Final.jar /tmp/oozie-hbase-prep/WEB-INF/lib/netty-3.6.6.Final.jar
  EOH
  action :run
end

bash 'create-new-oozie-war' do
  user "root"
  group "root"
  code <<-EOH
    cd /tmp/oozie-hbase-prep
    jar -cMf ../oozie.war *
  EOH
  action :run
end

service "stop-oozie" do
  action :stop
  service_name "oozie"
  supports :status => true, :restart => true, :reload => false
end

bash 'copy-hbase-jars' do
  user "root"
  group "root"
  code <<-EOH
    cp /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war /tmp/oozie.war1
    rm -fr /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war
    cp /tmp/oozie.war /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie.war
    rm -rf /usr/hdp/2.2.0.0-2041/oozie/oozie-server/webapps/oozie
  EOH
  action :run
end

service "start-oozie" do
  action :start
  service_name "oozie"
  supports :status => true, :restart => true, :reload => false
end

[bijugs]

PR #369 raised for the enhancement.

[bijugs]

Addressed in #681.