This cookbook extends the ambari-chef cookbook to install Hortonworks Data Platform and provision Hadoop clusters using Blueprints.
HDP requires Ambari, Java, and Python as prerequisites for installation. These prerequisites can be fulfilled using recipes found in the ambari-chef cookbook or by any other means independently. You can include and customize these prerequisites by modifying the following.
# -- INCLUDE COOKBOOKS --
# ambari-server
include 'ambari-chef::ambari_server_prerequisites'
include 'ambari-chef::ambari_server'
# ambari-agent
include `ambari-chef::ambari_agent_prerequisites`
include 'ambari-chef::ambari_agent'# -- INCLUDE ATTRIBUTES --
# python 2
override['python'] = {
'python2' => {
'packages' => ['python'],
},
}
# java 8
override['java'] = {
'install_from' => 'amazon_source',
'install_version' => 'jdk-8u232-linux-x64',
}By default this cookbook installs HDP version 3.1.4. A different version of HDP can be specified for installation by overriding the version attribute.
# hdp
override['hw']['hdp'] =
'version' => '3.1.4',
}Blueprints allow hdp clusters to be programmatically provisioned. A sample hdp_demo blueprint has been included with this cookbook contains core Hadoop applications.
You can create your own blueprints and modify the following to push them out with this cookbook
# -- INCLUDE ATTRIBUTES --
# hortonworks hdp cluster
override['hw']['hdp'] = {
'version' => '3.1.4',
'cluster' => {
'name' => 'hdp_demo',
'blueprint_name' => 'hdp_demo_3.1.4_blueprint',
'blueprint_file' => 'hdp_demo_3.1.4_blueprint.json'
'hostmapping_file' => 'hdp_demo_3.1.4_hostmapping.json'
},
}To install HDP, include the hdp recipe in your run list. Unless independently handling the perquisites yourself, include the recipes provided by the ambari-chef cookbook as well.
# include prerequisites
include 'ambari-chef::ambari_server_prerequisites'
include 'ambari-chef::ambari_server'
# include hdp
include 'hdp-chef::hdp'
include 'ambari-chef::ambari_cluster'Once installed, log into Ambari at http://127.0.0.1:8080 using admin for both the username and password. The hdp_demo cluster will automatically provision using the included blueprint.
The following resources may be helpful to better understand HDP: