Create serverless functions that interacts with on-premise resources

The application demonstrates IBM Cloud Functions (based on Apache OpenWhisk) that interacts with on-premise resources by using Secure Gateway. The use case demonstrates how you can allow IBM Cloud Functions access to your on-premise resources such as databases and web applications. This code pattern uses a local CouchDB deployment that will be accessed by the serverless functions. This code pattern will also use a local Minio, an open source object storage server, and will be served through a local Node.js web application. The local resources will be exposed through a Secure Gateway tunnel.

The serverless functions for CouchDB will utilize the existing OpenWhisk's Cloudant package. This is possible since Cloudant is based on CouchDB and has almost similar APIs. An action will be triggered by changes in the database. In this case, the action will output the document ID. Other actions such as creating and reading a document can be invoked through the wsk or ibmcloud fn command. Another scenario is executing actions via REST APIs. The actions will interact with the local Node.js web application that's using an object storage. The actions will create a bucket and an object from a URL. All the actions will utilize the Secure Gateway tunnel.

When the reader has completed this Code Pattern, they will understand how to:

• Allow Cloud Functions to access on-premise resources
• Create a Secure Gateway tunnel that will be used by serverless functions

Flow

1. The user inserts a document on the on-premise database
2. A trigger is fired once the document was inserted in the database
3. An action connected to the trigger is invoked. The action simply prints the document ID of the inserted document
4. The user can also interact with the on-prem database via the actions available in the cloudant package
5. The user can also interact with serverless functions via REST APIs by using API Gateway that is easily integrated with IBM Cloud Functions.
6. The actions are doing http requests to the available cloud host that is exposed by the Secure Gateway service.
7. All the interaction with the on-prem database goes through a tunnel via the Secure Gateway service that is also installed in the on-prem environment.

Prerequisites

• IBM Cloud Functions CLI to create cloud functions from the terminal. Make sure you do the command ibmcloud fn api list so that your ~/.wskprops is pointing to the right account.

• Whisk Deploy (wskdeploy) is a utility to help you describe and deploy any part of the OpenWhisk programming model using a Manifest file written in YAML. You'll use it to deploy all the Cloud Function resources using a single command. You can download it from the releases page and select the appropriate file for your system.

You can also use ibmcloud fn deploy. This has the same functionality integrated with the ibmcloud CLI.

Steps

1. Clone the repo
2. Create Secure Gateway service with IBM Cloud
3. Run CouchDB and Minio with Docker
4. Deploy IBM Cloud Functions
5. Test with database events
6. Test with REST API to access on-prem webapp

1. Clone the repo

Clone this repo locally and cd into the repo's directory. In a terminal, run:

$ git clone https://github.com/IBM/ibm-cloud-functions-with-on-prem-resources

2. Create Secure Gateway service with IBM Cloud

Create a Secure Gateway instance:

• Secure Gateway

Then follow the instructions here to properly setup the connection to your on-premise resources:

• Setting up Secure Gateway connection

3. Run CouchDB and Minio with Docker

Run couchdb and minio in your host machine. Docker is one way of easily deploying it.

$ docker run -p 5984:5984 -e COUCHDB_USER=admin -e COUCHDB_PASSWORD=password --name=couchdb-test -d couchdb

$ docker run -p 9000:9000 --name minio-test \
  -e "MINIO_ACCESS_KEY=1234" \
  -e "MINIO_SECRET_KEY=12345678" \
  -v /tmp/minio-data:/data \
  -v /tmp/minio-config:/root/.minio \
  minio/minio server /data

Create a database named testdb in couchdb. Access the dashboard on http://localhost:5984/_utils/ and enter admin and password as the username and password.

4. Deploy IBM Cloud Functions

First, export the Cloud Host in an environment variable that you have from your Secure Gateway destination. It would look something like: cap-us-east-prd-sg-bm-03.integration.ibmcloud.com:15006

$ export CLOUD_HOST="cap-us-east-prd-sg-bm-03.integration.ibmcloud.com:15006"

Then deploy the set of IBM Cloud Function resources with ibmcloud fn deploy or wskdeploy. The manifest-couchdb.yaml and manifest-webapp-minio.yaml describes all the Actions, Packages, Triggers and Rules and deploys them with one command.

$ ibmcloud fn deploy -m manifest-couchdb.yaml
$ ibmcloud fn deploy -m manifest-webapp-minio.yaml

You may want to undeploy them later with ibmcloud fn undeploy

Want to learn more about wskdeploy? Check out the repository

5. Test with database events

Insert a document in couchdb using the dashboard on http://localhost:5984/_utils/. Insert a simple document

{
  "_id": "sample-document-1"
}

Verify that an action is triggered by the event of inserting a document in the dashboard https://cloud.ibm.com/openwhisk/dashboard

You can also invoke available actions from the Cloudant package. Try invoking the create-document action that will create a document in your local couchdb deployment.

$ ibmcloud fn action invoke couchdb-binding/create-document --param dbname testdb --param doc '{"_id":"sample-document-1"}' --result

{
    "id": "sample-document-1",
    "ok": true,
    "rev": "1-967a00dff5e02add41819138abb3284d"
}

Read your recently created document with read-document action.

$ ibmcloud fn action invoke couchdb-binding/read-document --param dbname testdb --param docid sample-document-1 --result

{
    "_id": "sample-document-1",
    "_rev": "1-967a00dff5e02add41819138abb3284d"
}

6. Test with REST API to access on-prem webapp

Cloud Functions can also be executed via REST APIs. The action can do http requests through the Secure Gateway's cloud host destination that would allow interaction to an on-prem web application.

The web application is a simple API server that is interfaced with a local object storage deployment. Deploy the web app using nodejs.

$ cd containers/webapp-minio
$ npm install
$ node app.js

Edit the destination for your Secure Gateway to the port 8080 which is the web application's.

Get the list of the current APIs' URL created for your actions.

$ ibmcloud fn api list sample-api-gateway-create

# Result #
Action                                      Verb                   API Name  URL
/interact-with-on-pre                       post  sample-api-gateway-create  https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/create/object
/interact-with-on-pre                       post  sample-api-gateway-create  https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/create/bucket

Create a bucket for your on-prem object storage deployment using the Cloud Function's exposed REST APIs. Use the .../create/bucket endpoint.

$ export CF_API='https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/create/bucket'
$ curl -X POST ''"$CF_API"'?bucketName=sample-bucket'

{"status":"Successfully created bucket"}

Create an object using the .../create/object endpoint. This accepts an image url that will be downloaded to your bucket.

$ export CF_API='https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/create/object'
$ curl -X POST -H 'Content-type: application/json' -d '{"imageUrl":"https://newsroom.ibm.com/image/IBM+logo.jpg","bucketName":"sample-bucket","objectName":"sample-image.png"}' $CF_API

{"etag":"c9fdfac834f00bc32958384e259c4d50"}

Get another list of APIs.

$ ibmcloud fn api list sample-api-gateway-get

# Result #
Action                                      Verb                   API Name  URL
/interact-with-on-pre                       get  sample-api-gateway-create  https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/get/object
/interact-with-on-pre                       get  sample-api-gateway-create  https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/get/ojbects

Get the list of objects in your created bucket using the .../get/objects endpoint.

$ export CF_API='https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/get/objects'
$ curl ''"$CF_API"'?bucketName=sample-bucket'

[{"name":"sample-image.png","lastModified":"2018-12-06T23:15:24.000Z","etag":"c9fdfac834f00bc32958384e259c4d50","size":19292}]

Download the object using the ../get/object endpoint.

$ export CF_API='https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<>/get/object'
$ curl ''"$CF_API"'?bucketName=sample-bucket&objectName=sample-image.png' > sample-image.png

## You can open the sample-image.png image in your file explorer.

Clean up

You can now clean up the resources you created.

$ docker stop couchdb-test
$ docker stop minio-test
$ ibmcloud fn undeploy -m manifest-couchdb.yaml
$ ibmcloud fn undeploy -m manifest-webapp-minio.yaml

Alternative Deployment Methods

Deploy manually with the ibmcloud fn command line tool

This approach shows you how to deploy individual the packages, actions, triggers, and rules with CLI commands. It helps you understand and control the underlying deployment artifacts.

• Export the Cloud Host in an environment variable that you have from your Secure Gateway destination. It would look something like: cap-us-east-prd-sg-bm-03.integration.ibmcloud.com:15006

$ export CLOUD_HOST="cap-us-east-prd-sg-bm-03.integration.ibmcloud.com:15006"

• Bind the Cloudant package

$ ibmcloud fn package bind /whisk.system/cloudant couchdb-binding \
-p username admin \
-p password password \
-p host ${CLOUD_HOST}

• Create a Trigger to listen to database events

This will listen to changes on the testdb database

$ ibmcloud fn trigger create couchdb-changes-trigger --feed couchdb-binding/changes \
--param dbname testdb

• Create a Rule

The rule connects the trigger to an action.

$ ibmcloud fn rule create couchdb-trigger-rule couchdb-changes-trigger /whisk.system/utils/echo

This connects the trigger to a simple echo action that just outputs the input which would be the database event.

• Create the actions

Create a package to organize the actions that will be created for this repo.

$ ibmcloud fn package create interact-with-on-prem
$ ibmcloud fn package create interact-with-on-prem-2

Create actions that interact with the web app

$ ibmcloud fn action create interact-with-on-prem-2/get-object-request actions/get-object.js \
--kind nodejs:8 \
--param CLOUD_HOST $CLOUD_HOST

$ ibmcloud fn action create interact-with-on-prem-2/get-bucket-request actions/get-bucket-objects.js \
--kind nodejs:8 \
--param CLOUD_HOST $CLOUD_HOST

$ ibmcloud fn action create interact-with-on-prem-2/create-bucket-request actions/create-bucket.js \
--kind nodejs:8 \
--param CLOUD_HOST $CLOUD_HOST

$ ibmcloud fn action create interact-with-on-prem-2/create-object-request actions/create-object.js \
--kind nodejs:8 \
--param CLOUD_HOST $CLOUD_HOST

• Create REST APIs

$ ibmcloud fn api create /get /object get interact-with-on-prem-2/get-object-request -n sample-api-gateway-get --response-type http
$ ibmcloud fn api create /get /objects get interact-with-on-prem-2/get-bucket-request -n sample-api-gateway-get --response-type http
$ ibmcloud fn api create /create /bucket post interact-with-on-prem-2/create-bucket-request -n sample-api-gateway-create --response-type http
$ ibmcloud fn api create /create /object post interact-with-on-prem-2/create-object-request -n sample-api-gateway-create --response-type http

You can now proceed to the next step.

• To delete them later:

$ ibmcloud fn trigger delete couchdb-changes-trigger
$ ibmcloud fn rule delete couchdb-trigger-rule
$ ibmcloud fn package delete couchdb-binding
$ ibmcloud fn package delete interact-with-on-prem
$ ibmcloud fn action delete interact-with-on-prem-2/get-object-request
$ ibmcloud fn action delete interact-with-on-prem-2/get-bucket-request
$ ibmcloud fn action delete interact-with-on-prem-2/create-object-request
$ ibmcloud fn action delete interact-with-on-prem-2/create-bucket-request
$ ibmcloud fn api delete sample-api-gateway-get
$ ibmcloud fn api delete sample-api-gateway-create
$ ibmcloud fn package delete interact-with-on-prem-2

License

This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.

Apache License FAQ