A command line tool for managing Hadoop Cluster via Cloudera Manager API using Shell Script
This tool is developed for ease of managing CDH cluster via Cloudera Manager API. Currently it supports:
- Generate SSL/TLS Certificates
- Enable SSL/TLS for Cloudera Manager
- Enable SSL/TLS for below CDH components (other services will be added in the near future):
How to use the tool
This tool needs to be run on the Cloudera Manager server host, running on Agent hosts has not been tested.
It uses Unix's "curl" command to communicate with Cloudera Manager API, so it must be installed.
It uses SSH to run commands remotely to generate certificates and update agent configuration files using "root" user, because we need to update and restart services that requires "root" privilege, so to avoid being asked to enter passwords, it is advised to have passwordless setup from the host you want to run the tool to all other hosts.
I have provided a simple setup script to do this for you:
bash setup.sh CM_HOST TLS_ENABLED CM_HOST: Cloudera Manager Host URL, without port number TLS_ENABLED: Whether the current Cloudera Manager already has TLS enabled or not, 1 or 0
Above script will help you to generate the public/private key pair on the CM host and then copy the IDs to the remote server to setup.
To enable SSL/TLS for the cluster, there are three steps:
Generate certificates on each host:
bash ssl/cert-install.sh CM_HOST TLS_ENABLED TYPE CM_HOST: Cloudera Manager Host URL, without port number TLS_ENABLED: Whether the current Cloudera Manager already has TLS enabled or not, 1 or 0 TYPE: Either "ca" for CA Signed Certificate or "self" for Self-Signed Certificate
If you choose to generate Self-Signed certificates, this script will generate Java KeyStore, Private Key and Public Key files automatically and save them to /opt/cloudera/security/jks directory.
If you choose to generate Internal CA Signed Certificates, it requires that you already have Root CA (rootca.pem) and Intermediate CA (intca-x.pem) certificates under the "ssl/ca" directory at the root of project directory.
It will display you with the Public Key output and you will need to generate the Certificate file from your Internal CA system, copy and paste the result into the script output when prompted. This process needs to be done per host, so please be patient.
Update Cloudera Manager Server and Agent Configuration to enable SSL/TLS
bash ssl/cm/enable.sh CM_HOST TLS_ENABLED CM_HOST: Cloudera Manager Host URL, without port number TLS_ENABLED: Whether the current Cloudera Manager already has TLS enabled or not, 1 or 0
This script will update CM server and agent configurations to enable SSL/TLS, restart all CM agents, server and management services processes automatically.
It assumes that the SSL Certificates are generated from first step.
Update CDH configurations in Cloudera Manager to enable SSL for each services
bash ssl/cdh/enable.sh CM_HOST TLS_ENABLED CM_HOST: Cloudera Manager Host URL, without port number TLS_ENABLED: Whether the current Cloudera Manager already has TLS enabled or not, 1 or 0
This script will automatically detect the number of clusters and services managed by Cloudera Manager and will ask you to choose which cluster and services you want to enable SSL for.
After configurations are updated, it will go ahead to restart the cluster at the end.
It also assumes that the SSL Certificates are generated from first step.
Known Issues & Limitations
- Relies on /usr/bin/bigtop-detect-javahome to detect JAVA_HOME, and assumes that it is the same on all hosts
- Need to be run on Cloudera Manager host
- Currently only supports Cloudera Management Services running on the same host as the Cloudera Manager Server if using self-signed
- Manual step required for Hue to connect to Impala & HiveServer2, please refer to below links:
- Support disablement of SSL/TLS for any service
- Support SSL/TLS for other CDH components
- Support enabling HA services for various CDH components
- Support enabling Kerberos for CDH
- Support enabling Sentry for various CDH components
- Since it will clear /opt/cloudera/security/jks directory, need to have warning before going ahead
- More to come...