Separate routemgmt.yml out from openwhisk.yml playbook

[rabbah]

API gateway is missing from redo aka wskdev. Add it and also separate the wsk download into its own playbook. Fix bug in script which failed silently if binary did not exist.

Description

Related issue and scope

• I opened an issue to propose and discuss this change (routemgmt.yml playbook will fail silently if the wsk binary is missing #3759)

My changes affect the following components

• API
• Controller
• Message Bus (e.g., Kafka)
• Loadbalancer
• Invoker
• Intrinsic actions (e.g., sequences, conductors)
• Data stores (e.g., CouchDB)
• Tests
• Deployment
• CLI
• General tooling
• Documentation

Types of changes

• Bug fix (generally a non-breaking change which closes an issue).
• Enhancement or new feature (adds new functionality).
• Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

• I signed an Apache CLA.
• I reviewed the style guides and followed the recommendations (Travis CI will check :).
• I added tests to cover my changes.
• My changes require further changes to the documentation.
• I updated the documentation where necessary.

[codecov-io]

Codecov Report

Merging #3760 into master will decrease coverage by 4.7%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #3760      +/-   ##
=========================================
- Coverage      76%   71.3%   -4.71%     
=========================================
  Files         145     145              
  Lines        7039    7039              
  Branches      424     424              
=========================================
- Hits         5350    5019     -331     
- Misses       1689    2020     +331

Impacted Files	Coverage Δ
...core/database/cosmosdb/RxObservableImplicits.scala	0% <0%> (-100%)	⬇️
...core/database/cosmosdb/CosmosDBArtifactStore.scala	0% <0%> (-95.1%)	⬇️
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala	0% <0%> (-92.6%)	⬇️
...whisk/core/database/cosmosdb/CosmosDBSupport.scala	0% <0%> (-81.82%)	⬇️
...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala	0% <0%> (-58.83%)	⬇️
...la/whisk/core/database/cosmosdb/CosmosDBUtil.scala	92% <0%> (-4%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 35cf728...7acfb71. Read the comment docs.

[rabbah]

@mdeuser can you take a look at this PR - I separated the routemgmt package installation as discussed on Slack.

[mdeuser]

maybe even move after postdeploy.yml?

[mdeuser]

"Run postdeploy.yml after deployment to install a catalog of useful packages"

[mdeuser]

@rabbah - i think you have it all nicely covered. the only concern is the breaking change for folk who expect the api web actions to be installed without the extra step. just a communication thing though..

[rabbah]

Right they'll break if they're using the gateway - The cli repo, and api gw repos may thus require tweaks once this PR is merged.

An alternative is to have the apigateway playbook install the routemgmt package - which makes sense to me since you can't use the gateway without the package. For deployment that want to use a different gateway, they then can execute the independent routemgmt playbook separately.

[rabbah]

@mdeuser I pushed a commit to merge the routemgmt playbook installation with managing the API gateway itself. This should backward compatible. If someone wasn't using the apigw playbook then they will need to add the routemgmt to their deployment.

[csantanapr]

@mdeuser can you take this reviewed it, this might have impacts to our downstream CI for the ordering of gateway.yml playbook now needs to be done after openwhisk.yml
Also apigateway repo would need to be adjusted here https://github.com/apache/incubator-openwhisk-apigateway/blob/master/tools/travis/build.sh#L51

[rabbah]

@csantanapr @mdeuser I created this PR apache/openwhisk-apigateway#314 which should be OK to merge now (Travis will verify) and will work after this PR is merged.

[rabbah]

@mdeuser anything for me to address on this PR?

[rabbah]

@mdeuser is there a repo I need to fix downstream you’re concerned about?

[mdeuser]

@rabbah - my apologies for the delay. i've been in interrupt mode recently.. would you be ok with leaving the routemgmt.yml outside of the apigateway.yml.

[rabbah]

does it make sense to do that - can you explain the rationale? My thinking is:

• if someone deploys the gateway, they need the route management package.
• if someone doesn't deploy the gateway, they have the option of not running the playbook (which avoids installing the route management package as well).
• if someone is using a third party gateway, they just run the playbook to install the route management package.

[mdeuser]

i guess i was seeing if we can avoid, as much as possible, changes that require some folks to tweak their deployment procedures. currently, the apigateway.yml deployment has been agnostic as to whether or not openwhisk has been deployed; by including routemgmt in that .yml, it now requires openwhisk to have been deployed. also, any ci/cd pipelines that use a local api gw deployment in some environments, but use an already deployed production api gw in other environments will need new conditional logic when deploying openwhisk to avoid deploying routemgmt twice - a second deployment won't cause a failure, just would take some extra time.

[rabbah]

By pulling out routemgmt.yml from the openwhisk playbook there's already a need to adjust a deployment: if you need the package, you have to add it to the deployment explicitly. So it's move the apigw playbook down, or leave it where it is and then add a second playbook. I'm not really seeing a big benefit here since either way - there's a change.

Also none of our playbooks can be deployed again without tear down - what this PR does is fix the bug in the playbook so you can properly uninstall and reinstall the packages.

[rabbah]

As in this PR, this is not a big change:
https://github.com/apache/incubator-openwhisk-apigateway/pull/314/files#diff-183ae0496c3df4528d793134f32a6955R50

Maybe there are other scenarios I'm missing.

[mdeuser]

By pulling out routemgmt.yml from the openwhisk playbook there's already a need to adjust a deployment

true. but imho only needing to make the routemgmt.yml change is a bit simpler in a dynamic pipeline than handling the combined apigw/routemgmt. the routemgmt is always needed - no need for conditional deployment checks based on the target environment.

[rabbah]

I don't follow - are you saying you have a pipeline where you deploy the apigateway once but the routemgmt package many times?

[rabbah]

Noting result of discussion on slack: reverted the last commit. The rationale is that the apigw playbook is deploying a component (like couchdb) and routemgmt is related to configuration (like initdb/wipedb) and consistent with how we handle the lifecycle for those component, the two playbooks herein should be separate.

Thanks @mdeuser for the debate.

[mdeuser]

pg4/1965

[mdeuser]

changes look good to me

[rabbah]

@mdeuser status?

[csantanapr]

@mdeuser Can this be merge?