From 2c0264272c05c23a327b2ef97e812a098eba551c Mon Sep 17 00:00:00 2001 From: Lisa Owen Date: Wed, 11 Jan 2017 14:44:43 -0800 Subject: [PATCH] HAWQ-1263 - clarifications to ambari (inc subnav), hawq op env (closes #84) --- .../source/subnavs/apache-hawq-nav-210.erb | 10 ++-- markdown/admin/ambari-admin.html.md.erb | 2 + markdown/admin/ambari-rest-api.html.md.erb | 50 ++++++++++--------- markdown/admin/setuphawqopenv.html.md.erb | 7 +-- 4 files changed, 38 insertions(+), 31 deletions(-) diff --git a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb index 50ee5bc..1c7449f 100644 --- a/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb +++ b/book/master_middleman/source/subnavs/apache-hawq-nav-210.erb @@ -79,11 +79,13 @@
  • Introducing the HAWQ Operating Environment
    • -
  • +
  • Managing HAWQ Using Ambari -
    • -
  • - Using the Ambari REST API +
  • Starting and Stopping HAWQ diff --git a/markdown/admin/ambari-admin.html.md.erb b/markdown/admin/ambari-admin.html.md.erb index a5b2169..d080b2e 100644 --- a/markdown/admin/ambari-admin.html.md.erb +++ b/markdown/admin/ambari-admin.html.md.erb @@ -437,3 +437,5 @@ To set the threshold for the unresponsive segments that will trigger an alert: 3. Click **Save** when done. Alerts for **WARN**, and **CRITICAL** will be displayed when the number of unresponsive HAWQ segments in the cluster is greater than the specified percentage. +## Using the Ambari REST API +You can perform certain HAWQ cluster management activities from the command line using Ambari REST APIs. Refer to [Using the Ambari REST API](ambari-rest-api.html) for additional information. diff --git a/markdown/admin/ambari-rest-api.html.md.erb b/markdown/admin/ambari-rest-api.html.md.erb index 2cc79e4..f029acb 100644 --- a/markdown/admin/ambari-rest-api.html.md.erb +++ b/markdown/admin/ambari-rest-api.html.md.erb @@ -30,7 +30,9 @@ The Ambari REST API supports standard HTTP request methods including: - `GET` - read resource properties, metrics - `POST` - create new resource - `PUT` - update resource -- `DELETE` - delete resource +- `DELETE` - delete resource (use with care) + +**Note**: Be careful when using `DELETE` or `PUT` requests; typos or other incorrect usage may leave your cluster in an inoperable state. URIs for Ambari REST API resources have the following structure: @@ -88,16 +90,16 @@ Example: Testing Authentication 1. Set up some environment variables; replace the values with those appropriate for your operating environment. For example: ``` shell - $ export AMBUSER=admin - $ export AMBPASSWD=admin - $ export AMBHOST= - $ export AMBPORT=8080 + $ export AMBARI_USER=admin + $ export AMBARI_PASSWD=admin + $ export AMBARI_HOST= + $ export AMBARI_PORT=8080 ``` 2. Submit a `curl` request to the Ambari server: ``` shell - $ curl -u $AMBUSER:$AMBPASSWD http://$AMBHOST:$AMBPORT + $ curl -u $AMBARI_USER:$AMBARI_PASSWD http://$AMBARI_HOST:$AMBARI_PORT ``` If authentication succeeds, Apache license information is displayed. @@ -111,19 +113,19 @@ Example: Testing Authentication 1. Set up an additional environment variables: ``` shell - $ export AMBCREDS="$AMBUSER:$AMBPASSWD" - $ export AMBURLBASE="http://${AMBHOST}:${AMBPORT}/api/v1/clusters" + $ export AMBARI_CREDS="$AMBARI_USER:$AMBARI_PASSWD" + $ export AMBARI_URLBASE="http://${AMBARI_HOST}:${AMBARI_PORT}/api/v1/clusters" ``` You will use these variables in upcoming examples to simplify `curl` calls. -2. Use the Ambari REST API to determine the name of your HAWQ cluster; also set `$AMBURLBASE` to include the cluster name: +2. Use the Ambari REST API to determine the name of your HAWQ cluster; also set `$AMBARI_URLBASE` to include the cluster name: ``` shell - $ export CLUSTER_NAME="$(curl -u ${AMBCREDS} -i -H 'X-Requested-By:ambari' $AMBURLBASE | sed -n 's/.*"cluster_name" : "\([^\"]*\)".*/\1/p')" + $ export CLUSTER_NAME="$(curl -u ${AMBARI_CREDS} -i -H 'X-Requested-By:ambari' $AMBARI_URLBASE | sed -n 's/.*"cluster_name" : "\([^\"]*\)".*/\1/p')" $ echo $CLUSTER_NAME TestCluster - $ export AMBURLBASE=$AMBURLBASE/$CLUSTER_NAME + $ export AMBARI_URLBASE=$AMBARI_URLBASE/$CLUSTER_NAME ``` ### Examples: Managing the HAWQ and PXF Services @@ -137,27 +139,27 @@ Refer to [API usage scenarios, troubleshooting, and other FAQs](https://cwiki.ap | Task |Command | |----------------------|------------------------| -| View HAWQ service information. | `curl -u $AMBCREDS -X GET -H 'X-Requested-By:ambari' $AMBURLBASE/services/HAWQ` | -| List all HAWQ components. | `curl -u $AMBCREDS -X GET -H 'X-Requested-By:ambari' $AMBURLBASE/services/HAWQ/components` | -| View information about the HAWQ master. | `curl -u $AMBCREDS -X GET -H 'X-Requested-By:ambari' $AMBURLBASE/services/HAWQ/components/HAWQMASTER` | -| View the `hawq-site` configuration settings. | `curl -u $AMBCREDS -X GET -H 'X-Requested-By:ambari' "$AMBURLBASE/configurations?type=hawq-site&tag=TOPOLOGY_RESOLVED"` | -| View the initial `core-site` configuration settings. | `curl -u $AMBCREDS -X GET -H 'X-Requested-By:ambari' "$AMBURLBASE/configurations?type=core-site&tag=INITIAL"` | -| View the `pxf-profiles` configuration file. | `curl -u $AMBCREDS -X GET -H 'X-Requested-By:ambari' "$AMBURLBASE/configurations?type=pxf-profiles&tag=INITIAL"` | -| View all components on node. | `curl -u $AMBCREDS -i -X GET -H 'X-Requested-B:ambari' $AMBURLBASE/hosts/` | +| View HAWQ service information. | `curl -u $AMBARI_CREDS -X GET -H 'X-Requested-By:ambari' $AMBARI_URLBASE/services/HAWQ` | +| List all HAWQ components. | `curl -u $AMBARI_CREDS -X GET -H 'X-Requested-By:ambari' $AMBARI_URLBASE/services/HAWQ/components` | +| View information about the HAWQ master. | `curl -u $AMBARI_CREDS -X GET -H 'X-Requested-By:ambari' $AMBARI_URLBASE/services/HAWQ/components/HAWQMASTER` | +| View the `hawq-site` configuration settings. | `curl -u $AMBARI_CREDS -X GET -H 'X-Requested-By:ambari' "$AMBARI_URLBASE/configurations?type=hawq-site&tag=TOPOLOGY_RESOLVED"` | +| View the initial `core-site` configuration settings. | `curl -u $AMBARI_CREDS -X GET -H 'X-Requested-By:ambari' "$AMBARI_URLBASE/configurations?type=core-site&tag=INITIAL"` | +| View the `pxf-profiles` configuration file. | `curl -u $AMBARI_CREDS -X GET -H 'X-Requested-By:ambari' "$AMBARI_URLBASE/configurations?type=pxf-profiles&tag=INITIAL"` | +| View all components on node. | `curl -u $AMBARI_CREDS -i -X GET -H 'X-Requested-B:ambari' $AMBARI_URLBASE/hosts/` | #### Starting/Stopping HAWQ and PXF Services | Task |Command | |----------------------|------------------------| -| Start the HAWQ service. | `curl -u $AMBCREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Start HAWQ via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' $AMBURLBASE/services/HAWQ` | -| Stop the HAWQ service. | `curl -u $AMBCREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Stop HAWQ via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' $AMBURLBASE/services/HAWQ` | -| Start the PXF service. | `curl -u $AMBCREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Start PXF via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' $AMBURLBASE//services/PXF` | -| Stop the PXF service. | `curl -u $AMBCREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Stop PXF via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' $AMBURLBASE/services/PXF` | +| Start the HAWQ service. | `curl -u $AMBARI_CREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Start HAWQ via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' $AMBARI_URLBASE/services/HAWQ` | +| Stop the HAWQ service. | `curl -u $AMBARI_CREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Stop HAWQ via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' $AMBARI_URLBASE/services/HAWQ` | +| Start the PXF service. | `curl -u $AMBARI_CREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Start PXF via REST"}, "Body": {"ServiceInfo": {"state": "STARTED"}}}' $AMBARI_URLBASE//services/PXF` | +| Stop the PXF service. | `curl -u $AMBARI_CREDS -X PUT -H 'X-Requested-By:ambari' -d '{"RequestInfo": {"context" :"Stop PXF via REST"}, "Body": {"ServiceInfo": {"state": "INSTALLED"}}}' $AMBARI_URLBASE/services/PXF` | #### Invoking HAWQ and PXF Service Actions | Task |Command | |----------------------|------------------------| -| Run a HAWQ service check. | `curl -u $AMBCREDS -X POST -H 'X-Requested-By:ambari' -d '{"RequestInfo":{"context":"HAWQ Service Check","command":"HAWQ_SERVICE_CHECK"}, "Requests/resource_filters":[{ "service_name":"HAWQ"}]}' $AMBURLBASE/requests` | -| Run a PXF service check. | `curl -u $AMBCREDS -X POST -H 'X-Requested-By:ambari' -d '{"RequestInfo":{"context":"PXF Service Check","command":"PXF_SERVICE_CHECK"}, "Requests/resource_filters":[{ "service_name":"PXF"}]}' $AMBURLBASE/requests` | +| Run a HAWQ service check. | `curl -u $AMBARI_CREDS -X POST -H 'X-Requested-By:ambari' -d '{"RequestInfo":{"context":"HAWQ Service Check","command":"HAWQ_SERVICE_CHECK"}, "Requests/resource_filters":[{ "service_name":"HAWQ"}]}' $AMBARI_URLBASE/requests` | +| Run a PXF service check. | `curl -u $AMBARI_CREDS -X POST -H 'X-Requested-By:ambari' -d '{"RequestInfo":{"context":"PXF Service Check","command":"PXF_SERVICE_CHECK"}, "Requests/resource_filters":[{ "service_name":"PXF"}]}' $AMBARI_URLBASE/requests` | diff --git a/markdown/admin/setuphawqopenv.html.md.erb b/markdown/admin/setuphawqopenv.html.md.erb index 9d9b731..01e62ce 100644 --- a/markdown/admin/setuphawqopenv.html.md.erb +++ b/markdown/admin/setuphawqopenv.html.md.erb @@ -25,13 +25,13 @@ Perform the following steps to set up your HAWQ operating environment: hawq-user@master$ ``` -2. Set up your HAWQ operating environment by sourcing the `greenplum_path.sh` file: +2. Set up your HAWQ operating environment by sourcing the `greenplum_path.sh` file. If you built HAWQ from source or downloaded the tarball, substitute the path to the installed or extracted `greenplum_path.sh` file \(for example `/opt/hawq-2.1.0.0/greenplum_path.sh`\): ``` shell hawq-node$ source /usr/local/hawq/greenplum_path.sh ``` - - If you built HAWQ from source or downloaded the tarball, substitute the path to the installed or extracted `greenplum_path.sh` file \(for example `/opt/hawq-2.1.0.0/greenplum_path.sh`\). + + `source`ing `greenplum_path.sh` sets `PATH` to include the HAWQ `$GPHOME/bin/` directory. It also sets `LD_LIBRARY_PATH` to include the HAWQ libraries in `$GPHOME/lib/`. 3. Edit your `.bash_profile` or other shell initialization file to source `greenplum_path.sh` on login. For example, add: @@ -78,4 +78,5 @@ The following table identifies some files and directories of interest in a defau | /data/hawq/[master|segment]/pg_log/ | Default location of HAWQ master and segment log file directories | | /etc/pxf/conf/ | PXF service and configuration files | | /usr/lib/pxf/ | PXF service and plug-in shared libraries | +| /var/log/pxf/ | PXF log file directory | | /usr/hdp/current/ | HDP runtime and configuration files |