Separate routemgmt.yml out from openwhisk.yml playbook (apache#3760)

* Seperate route management package for API gateway from openwhisk.yml.

Update documentation for deployment.
Add API gateway to redo.

* Install routemgmt package when running API gateway playbook.

The routemgmt playbook remains its own independent playbook so that it may be installed seperately when necessary.

* Revert "Install routemgmt package when running API gateway playbook."

This reverts commit ae278f6.

* Fix Ruby example to use puts, adding a new line.

ansible/README.md

@@ -150,27 +150,31 @@ cd ansible
 ansible-playbook -i environments/<environment> couchdb.yml
 ansible-playbook -i environments/<environment> initdb.yml
 ansible-playbook -i environments/<environment> wipe.yml
-ansible-playbook -i environments/<environment> apigateway.yml
 ansible-playbook -i environments/<environment> openwhisk.yml
+
+# installs a catalog of public packages and actions
 ansible-playbook -i environments/<environment> postdeploy.yml
+
+# to use the API gateway
+ansible-playbook -i environments/<environment> apigateway.yml
+ansible-playbook -i environments/<environment> routemgmt.yml
 ```
 
-You need to run `initdb.yml` **every time** you do a fresh deploy CouchDB to initialize the subjects database.
-The playbooks `wipe.yml` and `postdeploy.yml` should be run on a fresh deployment only, otherwise all transient
-data that include actions and activations are lost.
+- You need to run `initdb.yml` **every time** you do a fresh deploy CouchDB to initialize the subjects database.
+- The `wipe.yml` playbook should be run on a fresh deployment only, otherwise actions and activations will be lost.
+- Run `postdeploy.yml` after deployment to install a catalog of useful packages.
+- To use the API Gateway, you'll need to run `apigateway.yml` and `routemgmt.yml`.
+- Use `ansible-playbook -i environments/<environment> openwhisk.yml` to avoid wiping the data store. This is useful to start OpenWhisk after restarting your Operating System.
 
 #### Limitation
 
-You can not run multiple CouchDB nodes on a single machine.
-This limitation comes from Erlang EPMD.
-When CouchDB forms a cluster, it counts on EPMD to find other nodes.
-If we want to run multiple nodes on a single machine, we must differentiate EPMD port(`4369`) for each nodes. But if this port is different on each nodes, they cannot find each other.
-So if you want to deploy multiple CouchDB nodes, all nodes should be placed on different machines respectively.
+You cannot run multiple CouchDB nodes on a single machine. This limitation comes from Erlang EPMD which CouchDB relies on to find other nodes.
+To deploy multiple CouchDB nodes, they should be placed on different machines respectively otherwise their ports will clash.
 
 
 ### Deploying Using Cloudant
--   Make sure your `db_local.ini` file is set up for Cloudant. See [Setup](#setup)
--   Then execute
+-   Make sure your `db_local.ini` file is set up for Cloudant. See [Setup](#setup).
+-   Then execute:
 
 ```
 cd <openwhisk_home>
@@ -180,16 +184,21 @@ ansible-playbook -i environments/<environment> initdb.yml
 ansible-playbook -i environments/<environment> wipe.yml
 ansible-playbook -i environments/<environment> apigateway.yml
 ansible-playbook -i environments/<environment> openwhisk.yml
-ansible-playbook -i environments/<environment> postdeploy.yml
-```
 
-You need to run `initdb` on Cloudant **only once** per Cloudant database to initialize the subjects database.
-The `initdb.yml` playbook will only initialize your database if it is not initialized already, else it will skip initialization steps.
+# installs a catalog of public packages and actions
+ansible-playbook -i environments/<environment> postdeploy.yml
 
-The playbooks `wipe.yml` and `postdeploy.yml` should be run on a fresh deployment only, otherwise all transient
-data that include actions and activations are lost.
+# to use the API gateway
+ansible-playbook -i environments/<environment> apigateway.yml
+ansible-playbook -i environments/<environment> routemgmt.yml
+```
 
-Use `ansible-playbook -i environments/<environment> openwhisk.yml` to avoid wiping the data store. This is useful to start OpenWhisk after restarting your Operating System.
+- You need to run `initdb` on Cloudant **only once** per Cloudant database to initialize the subjects database.
+- The `initdb.yml` playbook will only initialize your database if it is not initialized already, else it will skip initialization steps.
+- The `wipe.yml` playbook should be run on a fresh deployment only, otherwise actions and activations will be lost.
+- Run `postdeploy.yml` after deployment to install a catalog of useful packages.
+- To use the API Gateway, you'll need to run `apigateway.yml` and `routemgmt.yml`.
+- Use `ansible-playbook -i environments/<environment> openwhisk.yml` to avoid wiping the data store. This is useful to start OpenWhisk after restarting your Operating System.
 
 ### Configuring the installation of `wsk` CLI
 There are two installation modes to install `wsk` CLI: remote and local.

ansible/README_DISTRIBUTED.md

@@ -85,7 +85,13 @@ ansible-playbook -i environments/<environment> couchdb.yml
 ansible-playbook -i environments/<environment> initdb.yml
 ansible-playbook -i environments/<environment> wipe.yml
 ansible-playbook -i environments/<environment> openwhisk.yml
+
+# installs a catalog of public packages and actions
 ansible-playbook -i environments/<environment> postdeploy.yml
+
+# to use the API gateway
+ansible-playbook -i environments/<environment> apigateway.yml
+ansible-playbook -i environments/<environment> routemgmt.yml
 ```
 
 Setup your CLI and verify that OpenWhisk is working.

ansible/downloadcli.yml

@@ -0,0 +1,8 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more contributor
+# license agreements; and to You under the Apache License, Version 2.0.
+---
+# This playbook downloads the Openwhisk cli aka wsk from the API host.
+
+- hosts: ansible
+  roles:
+  - cli-install

ansible/openwhisk.yml

@@ -2,8 +2,9 @@
 # license agreements; and to You under the Apache License, Version 2.0.
 ---
 # This playbook deploys an Openwhisk stack.
-# It assumes you have already set up your database with the respective db provider playbook (currently cloudant.yml or couchdb.yml)
-# It assumes that wipe.yml have being deployed at least once
+# It assumes you have already set up your database with the respective db provider
+# playbook (currently cloudant.yml or couchdb.yml).
+# It assumes that wipe.yml have being deployed at least once.
 
 - import_playbook: kafka.yml
 
@@ -13,4 +14,4 @@
 
 - import_playbook: edge.yml
 
-- import_playbook: routemgmt.yml
+- import_playbook: downloadcli.yml

ansible/roles/cli-install/tasks/clean.yml

@@ -2,7 +2,7 @@
 # license agreements; and to You under the Apache License, Version 2.0.
 ---
 
-- name: remove cli nginx directory
+- name: remove wsk binary
   file:
     path: "{{ openwhisk_home }}/bin/{{ wsk }}"
     state: absent

ansible/roles/cli-install/tasks/deploy.yml

@@ -1,9 +1,9 @@
 # Licensed to the Apache Software Foundation (ASF) under one or more contributor
 # license agreements; and to You under the Apache License, Version 2.0.
 ---
-# Install the appropriate CLI into the ansible host for the routemgmt tasks
+# Install the appropriate CLI into the ansible host
 
-- name: "Grab the local CLI from the binaries unarchived into nginx"
+- name: grab the local CLI from the binaries unarchived into nginx
   get_url:
     url: "https://{{host}}/cli/go/download/{{os}}/{{arch}}/{{wsk}}"
     dest: "{{ openwhisk_home }}/bin"

ansible/roles/couchdb/tasks/deploy.yml

@@ -3,7 +3,7 @@
 ---
 # This role will run a CouchDB server on the db group
 
-- name: "Set the coordinator to the first node"
+- name: set the coordinator to the first node
   set_fact:
     coordinator: "{{ groups['db'][0] }}"
 

ansible/roles/routemgmt/files/installRouteMgmt.sh

@@ -13,7 +13,6 @@
 # WSK_CLI="$OPENWHISK_HOME/bin/wsk"
 
 set -e
-set -x
 
 if [ $# -eq 0 ]
 then
@@ -42,6 +41,11 @@ if [ -f "$AUTH" ]; then
     AUTH=`cat $AUTH`
 fi
 
+if [ ! -f $WSK_CLI ]; then
+    echo $WSK_CLI is missing
+    exit 1
+fi
+
 export WSK_CONFIG_FILE= # override local property file to avoid namespace clashes
 
 echo Installing apimgmt package

ansible/roles/routemgmt/files/uninstallRouteMgmt.sh

@@ -8,12 +8,11 @@
 # automatically
 #
 # To run this command
-# ./installRouteMgmt.sh  <AUTH> <APIHOST> <NAMESPACE> <WSK_CLI>
+# ./uninstallRouteMgmt.sh  <AUTH> <APIHOST> <NAMESPACE> <WSK_CLI>
 # AUTH, APIHOST and NAMESPACE are found in $HOME/.wskprops
 # WSK_CLI="$OPENWHISK_HOME/bin/wsk"
 
 set -e
-set -x
 
 if [ $# -eq 0 ]
 then
@@ -31,6 +30,11 @@ if [ -f "$AUTH" ]; then
     AUTH=`cat $AUTH`
 fi
 
+if [ ! -f $WSK_CLI ]; then
+    echo $WSK_CLI is missing
+    exit 1
+fi
+
 export WSK_CONFIG_FILE= # override local property file to avoid namespace clashes
 
 function deleteAction

ansible/routemgmt.yml

@@ -3,9 +3,6 @@
 ---
 # This playbook deploys the Openwhisk API Gateway route management actions.
 
-- import_playbook: properties.yml
-
 - hosts: ansible
   roles:
-  - cli-install
   - routemgmt

tools/build/redo

@@ -269,10 +269,21 @@ Components = [
     makeComponent('edge',
                   'deploy edge'),
 
+    makeComponent('cli',
+                  'download cli from api host',
+                  modes = 'clean',
+                  yaml = 'downloadcli.yml'),
+
     makeComponent('catalog',
                   'install catalog',
                   yaml = 'postdeploy.yml'),
 
+    makeComponent('apigw',
+                  'deploy api gateway',
+                  gradle = False,
+                  modes = 'clean',
+                  yaml = 'routemgmt.yml apigateway.yml'),
+
     # the following (re)build images via gradle
     makeComponent('runtime:([\w.-]+)',
                   'build a runtime action container, matching name using the regex; NOTE: must use --dir for path to runtime directory',

tools/travis/setupSystem.sh

@@ -25,7 +25,8 @@ ROOTDIR="$SCRIPTDIR/../.."
 
 cd $ROOTDIR/ansible
 
-$ANSIBLE_CMD apigateway.yml
 $ANSIBLE_CMD openwhisk.yml
+$ANSIBLE_CMD apigateway.yml
+$ANSIBLE_CMD routemgmt.yml
 
 echo "Time taken for ${0##*/} is $SECONDS secs"

tools/vagrant/Vagrantfile

@@ -149,9 +149,10 @@ Vagrant.configure('2') do |config|
     echo "`date`: deploy-start" >> /tmp/vagrant-times.txt
     cd ${ANSIBLE_HOME}
     su vagrant -c 'ansible-playbook -i environments/vagrant wipe.yml'
-    su vagrant -c 'ansible-playbook -i environments/vagrant apigateway.yml'
     su vagrant -c 'ansible-playbook -i environments/vagrant openwhisk.yml -e invoker_use_runc=False'
     su vagrant -c 'ansible-playbook -i environments/vagrant postdeploy.yml'
+    su vagrant -c 'ansible-playbook -i environments/vagrant apigateway.yml'
+    su vagrant -c 'ansible-playbook -i environments/vagrant routemgmt.yml'
 
     # Setup OpenWhisk CLI
     su vagrant -c 'mkdir ${HOME}/bin'