From 88a3d0903e49baa4f22ad3ef5790bd23cdb69a1f Mon Sep 17 00:00:00 2001 From: Olivier Tardieu Date: Tue, 27 Mar 2018 10:57:06 -0400 Subject: [PATCH 1/3] Add documentation and examples --- README.md | 6 +- docs/COMBINATORS.md | 220 +++++++++++++++++++++++++ docs/COMPOSE.md | 20 ++- docs/COMPOSER.md | 84 ++++++++++ docs/README.md | 373 ++----------------------------------------- samples/demo.js | 6 +- samples/demo.json | 6 +- samples/node-demo.js | 31 ++++ 8 files changed, 370 insertions(+), 376 deletions(-) create mode 100644 docs/COMBINATORS.md create mode 100644 docs/COMPOSER.md create mode 100644 samples/node-demo.js diff --git a/README.md b/README.md index f2f81a1..513cf24 100644 --- a/README.md +++ b/README.md @@ -60,9 +60,9 @@ A composition is typically defined by means of a Javascript expression as illustrated in [samples/demo.js](samples/demo.js): ```javascript composer.if( - composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), - composer.action('success', { action: function main() { return { message: 'success' } } }), - composer.action('failure', { action: function main() { return { message: 'failure' } } })) + composer.action('authenticate', { action: function ({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function () { return { message: 'success' } } }), + composer.action('failure', { action: function () { return { message: 'failure' } } })) ``` Compositions compose actions using _combinator_ methods. These methods implement the typical control-flow constructs of a sequential imperative diff --git a/docs/COMBINATORS.md b/docs/COMBINATORS.md new file mode 100644 index 0000000..7034a09 --- /dev/null +++ b/docs/COMBINATORS.md @@ -0,0 +1,220 @@ +# Combinators + +The `composer` module offers a number of combinators to define compositions: + +| Combinator | Description | Example | +| --:| --- | --- | +| [`action`](#action) | action | `composer.action('echo')` | +| [`function`](#function) | function | `composer.function(({ x, y }) => ({ product: x * y }))` | +| [`literal` or `value`](#literal) | constant value | `composer.literal({ message: 'Hello, World!' })` | +| [`sequence` or `seq`](#sequence) | sequence | `composer.sequence('hello', 'bye')` | +| [`let`](#let) | variable declarations | `composer.let({ count: 3, message: 'hello' }, ...)` | +| [`if`](#if) | conditional | `composer.if('authenticate', 'success', 'failure')` | +| [`while`](#while) | loop | `composer.while('notEnough', 'doMore')` | +| [`dowhile`](#dowhile) | loop at least once | `composer.dowhile('fetchData', 'needMoreData')` | +| [`repeat`](#repeat) | counted loop | `composer.repeat(3, 'hello')` | +| [`try`](#try) | error handling | `composer.try('divideByN', 'NaN')` | +| [`finally`](#finally) | finalization | `composer.finally('tryThis', 'doThatAlways')` | +| [`retry`](#retry) | error recovery | `composer.retry(3, 'connect')` | +| [`retain`](#retain) | persistence | `composer.retain('validateInput')` | + +The `action`, `function`, and `literal` combinators and their synonymous construct compositions respectively from actions, functions, and constant values. The other combinators combine existing compositions to produce new compositions. + +## Shorthands + +Where a composition is expected, the following shorthands are permitted: + - `name` of type `string` stands for `composer.action(name)`, + - `fun` of type `function` stands for `composer.function(fun)`, + - `null` stands for the empty sequence `composer.sequence()`. + +## Action + +`composer.action(name, [options])` is a composition with a single action named _name_. It invokes the action named _name_ on the input parameter object for the composition and returns the output parameter object of this action invocation. + +The action _name_ may specify the namespace and/or package containing the action following the usual OpenWhisk grammar. If no namespace is specified, the default namespace is assumed. If no package is specified, the default package is assumed. + +Examples: +```javascript +composer.action('hello') +composer.action('myPackage/myAction') +composer.action('/whisk.system/utils/echo') +``` +The optional `options` dictionary makes it possible to provide a definition for the action being composed. +```javascript +// specify the code for the action as a function +composer.action('hello', { action: function () { return { message: 'hello' } } }) + +// specify the code for the action as a function reference +function hello() { + return { message: 'hello' } +} +composer.action('hello', { action: hello }) + +// specify the code for the action as a string +composer.action('hello', { action: "const message = 'hello'; function main() { return { message } }" }) + + +// specify the code and runtime for the action +composer.action('hello', { + action: { + kind: 'nodejs:8', + code: "function () { return { message: 'hello' } }" + } +}) + +// specify a file containing the code for the action +composer.action('hello', { filename: 'hello.js' }) + +// specify a sequence of actions +composer.action('helloAndBye', { sequence: ['hello', 'bye'] }) +``` +The action may de defined by providing the code for the action as a string, as a Javascript function, or as a file name. Alternatively, a sequence action may be defined by providing the list of sequenced actions. The code (specified as a string) may be annotated with the kind of the action runtime. + +### Environment capture + +Javascript functions used to define actions cannot capture any part of their declaration environment. The following code is not correct as the declaration of `name` would not be available at invocation time: +```javascript +let name = 'Dave' +composer.action('hello', { action: function main() { return { message: 'Hello ' + name } } }) +``` +In contrast, the following code is correct as it resolves `name`'s value at composition time. +```javascript +let name = 'Dave' +composer.action('hello', { action: `function main() { return { message: 'Hello ' + '${name}' } }` }) +``` + +## Function + +`composer.function(fun)` is a composition with a single Javascript function _fun_. It applies the specified function to the input parameter object for the composition. + - If the function returns a value of type `function`, the composition returns an error object. + - If the function throws an exception, the composition returns an error object. The exception is logged as part of the conductor action invocation. + - If the function returns a value of type other than function, the value is first converted to a JSON value using `JSON.stringify` followed by `JSON.parse`. If the resulting JSON value is not a JSON dictionary, the JSON value is then wrapped into a `{ value }` dictionary. The composition returns the final JSON dictionary. + - If the function does not return a value and does not throw an exception, the composition returns the input parameter object for the composition converted to a JSON dictionary using `JSON.stringify` followed by `JSON.parse`. + +Examples: +```javascript +composer.function(params => ({ message: 'Hello ' + params.name })) +composer.function(function () { return { error: 'error' } }) + +function product({ x, y }) { return { product: x * y } } +composer.function(product) +``` + +### Environment capture + +Functions intended for compositions cannot capture any part of their declaration environment. They may however access and mutate variables in an environment consisting of the variables declared by the [composer.let](#composerletname-value-composition_1-composition_2-) combinator discussed below. + +The following is not legal: +```javascript +let name = 'Dave' +composer.function(params => ({ message: 'Hello ' + name })) +``` +The following is legal: +```javascript +composer.let({ name: 'Dave' }, composer.function(params => ({ message: 'Hello ' + name }))) +``` + +## Literal + +`composer.literal(value)` and its synonymous `composer.value(value)` output a constant JSON dictionary. This dictionary is obtained by first converting the _value_ argument to JSON using `JSON.stringify` followed by `JSON.parse`. If the resulting JSON value is not a JSON dictionary, the JSON value is then wrapped into a `{ value }` dictionary. + +The _value_ argument may be computed at composition time. For instance, the following composition captures the date at the time the composition is encoded to JSON: +```javascript +composer.sequence( + composer.literal(Date()), + composer.action('log', { action: params => ({ message: 'Composition time: ' + params.value }) })) +``` + +## Sequence + +`composer.sequence(composition_1, composition_2, ...)` chains a series of compositions (possibly empty). + +The input parameter object for the composition is the input parameter object of the first composition in the sequence. The output parameter object of one composition in the sequence is the input parameter object for the next composition in the sequence. The output parameter object of the last composition in the sequence is the output parameter object for the composition. + +If one of the components fails (i.e., returns an error object), the remainder of the sequence is not executed. The output parameter object for the composition is the error object produced by the failed component. + +An empty sequence behaves as a sequence with a single function `params => params`. The output parameter object for the empty sequence is its input parameter object unless it is an error object, in which case, as usual, the error object only contains the `error` field of the input parameter object. + +## Let + +`composer.let({ name_1: value_1, name_2: value_2, ... }, composition_1_, _composition_2_, ...)` declares one or more variables with the given names and initial values, and runs a sequence of compositions in the scope of these declarations. + +Variables declared with `composer.let` may be accessed and mutated by functions __running__ as part of the following sequence (irrespective of their place of definition). In other words, name resolution is [dynamic](https://en.wikipedia.org/wiki/Name_resolution_(programming_languages)#Static_versus_dynamic). If a variable declaration is nested inside a declaration of a variable with the same name, the innermost declaration masks the earlier declarations. + +For example, the following composition invokes composition `composition` repeatedly `n` times. +```javascript +composer.let({ i: n }, composer.while(() => i-- > 0, composition)) +``` +Variables declared with `composer.let` are not visible to invoked actions. However, they may be passed as parameters to actions as for instance in: +```javascript +composer.let({ n: 42 }, () => ({ n }), 'increment', params => { n = params.n }) +``` + +In this example, the variable `n` is exposed to the invoked action as a field of the input parameter object. Moreover, the value of the field `n` of the output parameter object is assigned back to variable `n`. + +## If + +`composer.if(condition, consequent, [alternate], [options])` runs either the _consequent_ composition if the _condition_ evaluates to true or the _alternate_ composition if not. + +A _condition_ composition evaluates to true if and only if it produces a JSON dictionary with a field `value` with value `true`. Other fields are ignored. Because JSON values other than dictionaries are implicitly lifted to dictionaries with a `value` field, _condition_ may be a Javascript function returning a Boolean value. An expression such as `params.n > 0` is not a valid condition (or in general a valid composition). One should write instead `params => params.n > 0`. The input parameter object for the composition is the input parameter object for the _condition_ composition. + +The _alternate_ composition may be omitted. If _condition_ fails, neither branch is executed. + +The optional `options` dictionary supports a `nosave` option. If `options.nosave` is thruthy, the _consequent_ composition or _alternate_ composition is invoked on the output parameter object of the _condition_ composition. Otherwise, the output parameter object of the _condition_ composition is discarded and the _consequent_ composition or _alternate_ composition is invoked on the input parameter object for the composition. For example, the following compositions divide parameter `n` by two if `n` is even: +```javascript +composer.if(params => params.n % 2 === 0, params => { params.n /= 2 }) +composer.if(params => { params.value = params.n % 2 === 0 }, params => { params.n /= 2 }, null, { nosave: true }) +``` +In the first example, the condition function simply returns a Boolean value. The consequent function uses the saved input parameter object to compute `n`'s value. In the second example, the condition function adds a `value` field to the input parameter object. The consequent function applies to the resulting object. In particular, in the second example, the output parameter object for the condition includes the `value` field. + +While, the default `nosave == false` behavior is typically more convenient, preserving the input parameter object is not free as it counts toward the parameter size limit for OpenWhisk actions. In essence, the limit on the size of parameter objects processed during the evaluation of the condition is reduced by the size of the saved parameter object. The `nosave` option omits the parameter save, hence preserving the parameter size limit. + +## While + +`composer.while(condition, body, [options])` runs _body_ repeatedly while _condition_ evaluates to true. The _condition_ composition is evaluated before any execution of the _body_ composition. See [composer.if](#composerifcondition-consequent-alternate) for a discussion of conditions. + +A failure of _condition_ or _body_ interrupts the execution. The composition returns the error object from the failed component. + +Like `composer.if`, `composer.while` supports a `nosave` option. By default, the output parameter object of the _condition_ composition is discarded and the input parameter object for the _body_ composition is either the input parameter object for the whole composition the first time around or the output parameter object of the previous iteration of _body_. However if `options.nosave` is thruthy, the input parameter object for _body_ is the output parameter object of _condition_. Moreover, the output parameter object for the whole composition is the output parameter object of the last _condition_ evaluation. + +For instance, the following composition invoked on dictionary `{ n: 28 }` returns `{ n: 7 }`: +```javascript +composer.while(params => params.n % 2 === 0, params => { params.n /= 2 }) +``` +For instance, the following composition invoked on dictionary `{ n: 28 }` returns `{ n: 7, value: false }`: +```javascript +composer.while(params => { params.value = params.n % 2 === 0 }, params => { params.n /= 2 }, { nosave: true }) +``` + +## Dowhile + +`composer.dowhile(condition, body, [options])` is similar to `composer.while(body, condition, [options])` except that _body_ is invoked before _condition_ is evaluated, hence _body_ is always invoked at least once. + +## Repeat + +`composer.repeat(count, body)` invokes _body_ _count_ times. + +## Try + +`composer.try(body, handler)` runs _body_ with error handler _handler_. + +If _body_ returns an error object, _handler_ is invoked with this error object as its input parameter object. Otherwise, _handler_ is not run. + +## Finally + +`composer.finally(body, finalizer)` runs _body_ and then _finalizer_. + +The _finalizer_ is invoked in sequence after _body_ even if _body_ returns an error object. + +## Retry + +`composer.retry(count, body)` runs _body_ and retries _body_ up to _count_ times if it fails. The output parameter object for the composition is either the output parameter object of the successful _body_ invocation or the error object produced by the last _body_ invocation. + +## Retain + +`composer.retain(body, [options])` runs _body_ on the input parameter object producing an object with two fields `params` and `result` such that `params` is the input parameter object of the composition and `result` is the output parameter object of _body_. + +An `options` dictionary object may be specified to alter the default behavior of `composer.retain` in the following ways: +- If `options.catch` is thruthy, the `retain` combinator behavior will be the same even if _body_ returns an error object. Otherwise, if _body_ fails, the output of the `retain` combinator is only the error object (i.e., the input parameter object is not preserved). +- If `options.filter` is a function, the combinator only persists the result of the function application to the input parameter object. +- If `options.field` is a string, the combinator only persists the value of the field of the input parameter object with the given name. diff --git a/docs/COMPOSE.md b/docs/COMPOSE.md index a60145a..2dd8868 100644 --- a/docs/COMPOSE.md +++ b/docs/COMPOSE.md @@ -1,6 +1,11 @@ # Compose Command -The `compose` command makes it possible to encode and deploy compositions. +The `compose` command makes it possible to deploy compositions from the command line. + +The `compose` command is intended is as a minimal complement to the OpenWhisk CLI. The OpenWhisk CLI already has the capability to configure, invoke, and delete compositions (since these are just OpenWhisk actions) but lacks the capability to create composition actions. The `compose` command bridges this gap making it possible to deploy compositions as part of the development cycle or in shell scripts. It is not a replacement for the OpenWhisk CLI however as it does not duplicate existing OpenWhisk CLI capabilities. For a much richer devops experience, we recommend using [Shell](https://github.com/ibm-functions/shell). + +## Usage + ``` compose ``` @@ -20,12 +25,12 @@ The `compose` command requires either a Javascript file that evaluates to a comp The `compose` command has three mode of operation: - By default or when the `--json` option is specified, the command returns the composition encoded as a JSON dictionary. -- When the `--deploy` option is specified, the command deploys the composition with the desired name. +- When the `--deploy` option is specified, the command deploys the composition given the desired name for the composition. - When the `--encode` option is specified, the command returns the Javascript code for the [conductor action](https://github.com/apache/incubator-openwhisk/blob/master/docs/conductors.md) for the composition. ## Encoding -By default, the `compose` command returns the composition encoded as a JSON dictionary: +By default, the `compose` command evaluates the composition code and outputs the resulting JSON dictionary: ``` compose demo.js ``` @@ -37,7 +42,7 @@ compose demo.js "action": { "exec": { "kind": "nodejs:default", - "code": "function main({ password }) { return { value: password === 'abc123' } }" + "code": "const main = function ({ password }) { return { value: password === 'abc123' } }" } } }, @@ -46,7 +51,7 @@ compose demo.js "action": { "exec": { "kind": "nodejs:default", - "code": "function main() { return { message: 'success' } }" + "code": "const main = function () { return { message: 'success' } }" } } }, @@ -55,7 +60,7 @@ compose demo.js "action": { "exec": { "kind": "nodejs:default", - "code": "function main() { return { message: 'failure' } }" + "code": "const main = function () { return { message: 'failure' } }" } } } @@ -85,10 +90,11 @@ compose demo.js ] } ``` -The evaluation context for the Javascript code includes the `composer` object implicitly defined as: +The evaluation context includes the `composer` object implicitly defined as: ```javascript composer = require('@ibm-functions/composer') ``` +In other words, there is no need to require the `composer` module explicitly in the composition code. ## Deployment diff --git a/docs/COMPOSER.md b/docs/COMPOSER.md new file mode 100644 index 0000000..4217df3 --- /dev/null +++ b/docs/COMPOSER.md @@ -0,0 +1,84 @@ +# Composer Module + +The [`composer`](../composer.js) Node.js module makes it possible define, deploy, and invoke compositions. + +- [Installation](#installation) +- [Example](#example) +- [Combinators](#combinator-methods) +- [Deployment](#deployment) + +## Installation + +To install the `composer` module, use the Node Package Manager: +``` +npm install @ibm-functions/composer +``` + +## Example + +The [samples/node-demo.js](../samples/node-demo.js) file illustrates how to define, deploy, and invoke a composition using `node`: +```javascript +// require the composer module +const composer = require('@ibm-functions/composer') + +// define the composition +const composition = composer.if( + composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function main() { return { message: 'success' } } }), + composer.action('failure', { action: function main() { return { message: 'failure' } } })) + +// instantiate OpenWhisk client +const wsk = composer.openwhisk({ ignore_certs: true }) + +wsk.compositions.deploy(composition, 'demo') // deploy composition + .then(() => wsk.actions.invoke({ name: 'demo', params: { password: 'abc123' }, blocking: true })) // invoke composition + .then(({ response }) => console.log(JSON.stringify(response.result, null, 4)), console.error) +``` +``` +node samples/node-demo.js +``` +```json +{ + "message": "success" +} +``` +Alternatively, the `compose` command can deploy compositions and the OpenWhisk CLI can invoke compositions. See [COMPOSE.md](COMPOSE.md) for details. + +# Combinator methods + +The `composer` object offers a number of combinator methods to define composition objects, e.g., `composer.if`. Combinators are documented in [COMBINATORS.md](COMBINATORS.md). + +# Deployment + +The `composer` object offers an extension to the [OpenWhisk Client for Javascript](https://github.com/apache/incubator-openwhisk-client-js) that supports deploying compositions. + +## OpenWhisk method + +A client instance is obtained by invoking `composer.openwhisk(options)`, for instance with: +```javascript +const wsk = composer.openwhisk({ ignore_certs: true }) + +``` +The specific OpenWhisk deployment to use may be specified via options, environment variables, or the OpenWhisk property file. Options have priority over environment variables, which have priority over the OpenWhisk property file. Options and environment variables are documented [here](https://github.com/apache/incubator-openwhisk-client-js#constructor-options). The default path for the whisk property file is `$HOME/.wskprops`. It can be altered by setting the `WSK_CONFIG_FILE` environment variable. + +The `composer` module adds to the OpenWhisk client instance a new top-level category named `compositions` with a method named `deploy`. + +## Deploy method + +`wsk.deploy(composition, [name])` deploys the composition `composition` with name `name`. More precisely, it successively deploys all the actions and compositions defined in `composition` as well as `composition` itself. + +The compositions are encoded into conductor actions prior to deployment. In other words, the `deploy` method deploys one or several actions. + +The `deploy` method returns a successful promise if all the actions were deployed successfully, or a rejected promise otherwise. In the later, the state of the various actions is unknown. + +The `deploy` method deletes the deployed actions before recreating them if necessary. As a result, default parameters, limits, and annotations on preexisting actions are lost. + +The `name` argument may be omitted if the `composition` consists of a single action invocation. In this case, `deploy` method only deploys the actions and compositions whose definitions are nested inside `composition`. + +## Invoke, Update, Delete + +Since compositions are deployed as conductor actions, other management tasks for compositions can be implemented by invoking methods of `wsk.actions`, for instance: +```javascript +wsk.actions.delete('demo') +``` +Updating or deleting a conductor action only affect the action itself. It does not affect any other action deployed as part of the composition. diff --git a/docs/README.md b/docs/README.md index fefeeda..c270c5d 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,360 +1,13 @@ -# Composer Reference - -The [`composer`](../composer.js) Node.js module makes it possible define action [compositions](#example) using [combinators](#combinators) and [deploy](#deployment) them. - -## Installation - -To install the `composer` module use the Node Package Manager: -``` -npm -g install @ibm-functions/composer -``` -We recommend to install the module globally (with `-g` option) so the `compose` -command is added to the path. Otherwise, it can be found in the `bin` folder of -the module installation. - -## Example - -A composition is typically defined by means of a Javascript file as illustrated -in [samples/demo.js](samples/demo.js): -```javascript -composer.if( - composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), - composer.action('success', { action: function main() { return { message: 'success' } } }), - composer.action('failure', { action: function main() { return { message: 'failure' } } })) -``` -Composer offers traditional control-flow concepts as methods. These methods -are called _combinators_. This example composition composes three actions named -`authenticate`, `success`, and `failure` using the `composer.if` combinator, -which implements the usual conditional construct. It take three actions (or -compositions) as parameters. It invokes the first one and, depending on the -result of this invocation, invokes either the second or third action. - - This composition includes the definitions of the three composed actions. If the - actions are defined and deployed elsewhere, the composition code can be shorten - to: -```javascript -composer.if('authenticate', 'success', 'failure') -``` -To deploy this composition use the `compose` command: -``` -compose demo.js --deploy demo -``` -The `compose` command synthesizes and deploy an action named `demo` that -implements the composition. It also deploys the composed actions if definitions -are provided for them. - -The `demo` composition may be invoked like any action, for instance using the -OpenWhisk CLI: -``` -wsk action invoke demo -r -p password passw0rd -``` -```json -{ - "message": "failure" -} -``` - -## Activation Records -An invocation of a composition creates a series of activation records: -``` -wsk action invoke demo -p password passw0rd -``` -``` -ok: invoked /_/demo with id 4f91f9ed0d874aaa91f9ed0d87baaa07 -``` -``` -wsk activation list -``` -``` -activations -fd89b99a90a1462a89b99a90a1d62a8e demo -eaec119273d94087ac119273d90087d0 failure -3624ad829d4044afa4ad829d40e4af60 demo -a1f58ade9b1e4c26b58ade9b1e4c2614 authenticate -3624ad829d4044afa4ad829d40e4af60 demo -4f91f9ed0d874aaa91f9ed0d87baaa07 demo -``` -The entry with the earliest start time (`4f91f9ed0d874aaa91f9ed0d87baaa07`) summarizes the invocation of the composition while other entries record later activations caused by the composition invocation. There is one entry for each invocation of a composed action (`a1f58ade9b1e4c26b58ade9b1e4c2614` and `eaec119273d94087ac119273d90087d0`). The remaining entries record the beginning and end of the composition as well as the transitions between the composed actions. - -Compositions are implemented by means of OpenWhisk conductor actions. The [documentation of conductor actions](https://github.com/apache/incubator-openwhisk/blob/master/docs/conductors.md) discusses activation records in greater details. - -## Deployment - -The `compose` command when not invoked with the `--deploy` option returns the composition encoded as a JSON dictionary: -``` -compose demo.js -``` -```json -{ - "actions": [ - { - "name": "/_/authenticate", - "action": { - "exec": { - "kind": "nodejs:default", - "code": "function main({ password }) { return { value: password === 'abc123' } }" - } - } - }, - { - "name": "/_/success", - "action": { - "exec": { - "kind": "nodejs:default", - "code": "function main() { return { message: 'success' } }" - } - } - }, - { - "name": "/_/failure", - "action": { - "exec": { - "kind": "nodejs:default", - "code": "function main() { return { message: 'failure' } }" - } - } - } - ], - "composition": [ - { - "type": "if", - "test": [ - { - "type": "action", - "name": "/_/authenticate" - } - ], - "consequent": [ - { - "type": "action", - "name": "/_/success" - } - ], - "alternate": [ - { - "type": "action", - "name": "/_/failure" - } - ] - } - ] -} -``` -The JSON format is documented in [FORMAT.md](FORMAT.md). The format is meant to be stable, self-contained, language-independent, and human-readable. The JSON dictionary includes the definition for the composition as well as definitions of nested actions and compositions (if any). - -A JSON-encoded composition may be deployed using the `compose` command: -``` -compose demo.js > demo.json -compose demo.json --deploy demo -``` -The `compose` command can also produce the code of the conductor action generated for the composition: -``` -compose demo.js --encode -``` -```javascript -const main=(function init(e,t){function r(e,t){return e.slice(-1)[0].next=1,e.push(...t),e}const a=function e(t,a=""){if(Array.isArray(t))return 0===t.length?[{type:"pass",path:a}]:t.map((t,r)=>e(t,a+"["+r+"]")).reduce(r);const n=t.options||{};switch(t.type){case"action":return[{type:"action",name:t.name,path:a}];case"function":return[{type:"function",exec:t.exec,path:a}];case"literal":return[{type:"literal",value:t.value,path:a}];case"finally":var s=e(t.body,a+".body");const l=e(t.finalizer,a+".finalizer");return(o=[[{type:"try",path:a}],s,[{type:"exit",path:a}],l].reduce(r))[0].catch=o.length-l.length,o;case"let":return s=e(t.body,a+".body"),[[{type:"let",let:t.declarations,path:a}],s,[{type:"exit",path:a}]].reduce(r);case"retain":s=e(t.body,a+".body");var o=[[{type:"push",path:a}],s,[{type:"pop",collect:!0,path:a}]].reduce(r);return n.field&&(o[0].field=n.field),o;case"try":s=e(t.body,a+".body");const h=r(e(t.handler,a+".handler"),[{type:"pass",path:a}]);return(o=[[{type:"try",path:a}],s].reduce(r))[0].catch=o.length,o.slice(-1)[0].next=h.length,o.push(...h),o;case"if":var p=e(t.consequent,a+".consequent"),c=r(e(t.alternate,a+".alternate"),[{type:"pass",path:a}]);return n.nosave||(p=r([{type:"pop",path:a}],p)),n.nosave||(c=r([{type:"pop",path:a}],c)),o=r(e(t.test,a+".test"),[{type:"choice",then:1,else:p.length+1,path:a}]),n.nosave||(o=r([{type:"push",path:a}],o)),p.slice(-1)[0].next=c.length,o.push(...p),o.push(...c),o;case"while":return p=e(t.body,a+".body"),c=[{type:"pass",path:a}],n.nosave||(p=r([{type:"pop",path:a}],p)),n.nosave||(c=r([{type:"pop",path:a}],c)),o=r(e(t.test,a+".test"),[{type:"choice",then:1,else:p.length+1,path:a}]),n.nosave||(o=r([{type:"push",path:a}],o)),p.slice(-1)[0].next=1-o.length-p.length,o.push(...p),o.push(...c),o;case"dowhile":var i=e(t.test,a+".test");return n.nosave||(i=r([{type:"push",path:a}],i)),o=[e(t.body,a+".body"),i,[{type:"choice",then:1,else:2,path:a}]].reduce(r),n.nosave?(o.slice(-1)[0].then=1-o.length,o.slice(-1)[0].else=1):(o.push({type:"pop",path:a}),o.slice(-1)[0].next=1-o.length),c=[{type:"pass",path:a}],n.nosave||(c=r([{type:"pop",path:a}],c)),o.push(...c),o}}(t),n=e=>"object"==typeof e&&null!==e&&!Array.isArray(e),s=e=>Promise.reject({code:400,error:e}),o=e=>Promise.reject((e=>({code:"number"==typeof e.code&&e.code||500,error:"string"==typeof e.error&&e.error||e.message||"string"==typeof e&&e||"An internal error occurred"}))(e));return t=>Promise.resolve().then(()=>(function(t){let r=0,p=[];if(void 0!==t.$resume){if(!n(t.$resume))return s("The type of optional $resume parameter must be object");if(r=t.$resume.state,p=t.$resume.stack,void 0!==r&&"number"!=typeof r)return s("The type of optional $resume.state parameter must be number");if(!Array.isArray(p))return s("The type of $resume.stack must be an array");delete t.$resume,c()}function c(){if(n(t)||(t={value:t}),void 0!==t.error)for(t={error:t.error},r=void 0;p.length>0&&"number"!=typeof(r=p.shift().catch););}function i(r){function a(e,t){const r=p.find(t=>void 0!==t.let&&void 0!==t.let[e]);void 0!==r&&(r.let[e]=JSON.parse(JSON.stringify(t)))}const n=p.reduceRight((e,t)=>"object"==typeof t.let?Object.assign(e,t.let):e,{});let s="(function(){try{";for(const e in n)s+=`var ${e}=arguments[1]['${e}'];`;s+=`return eval((${r}))(arguments[0])}finally{`;for(const e in n)s+=`arguments[1]['${e}']=${e};`;s+="}})";try{return e(s)(t,n)}finally{for(const e in n)a(e,n[e])}}for(;;){if(void 0===r)return console.log("Entering final state"),console.log(JSON.stringify(t)),t.error?t:{params:t};const e=a[r];console.log(`Entering state ${r} at path fsm${e.path}`);const n=r;switch(r=void 0===e.next?void 0:n+e.next,e.type){case"choice":r=n+(t.value?e.then:e.else);break;case"try":p.unshift({catch:n+e.catch});break;case"let":p.unshift({let:JSON.parse(JSON.stringify(e.let))});break;case"exit":if(0===p.length)return o(`State ${n} attempted to pop from an empty stack`);p.shift();break;case"push":p.unshift(JSON.parse(JSON.stringify({params:e.field?t[e.field]:t})));break;case"pop":if(0===p.length)return o(`State ${n} attempted to pop from an empty stack`);t=e.collect?{params:p.shift().params,result:t}:p.shift().params;break;case"action":return{action:e.name,params:t,state:{$resume:{state:r,stack:p}}};case"literal":t=JSON.parse(JSON.stringify(e.value)),c();break;case"function":let a;try{a=i(e.exec.code)}catch(e){console.error(e),a={error:`An exception was caught at state ${n} (see log for details)`}}"function"==typeof a&&(a={error:`State ${n} evaluated to a function`}),t=JSON.parse(JSON.stringify(void 0===a?t:a)),c();break;case"pass":c();break;default:return o(`State ${n} has an unknown type`)}}})(t)).catch(o)})(eval,[{"type":"if","test":[{"type":"action","name":"/_/authenticate"}],"consequent":[{"type":"action","name":"/_/success"}],"alternate":[{"type":"action","name":"/_/failure"}]}]) -``` -This code may be deployed using the OpenWhisk CLI: -``` -compose demo.js > demo-conductor.js -wsk action create demo demo-conductor.js -a conductor true -``` -In contrast to the JSON format, the conductor action code does not include definitions for nested actions or compositions. - -## Parameter Objects and Error Objects - -A composition, like any action, accepts a JSON dictionary (the _input parameter object_) and produces a JSON dictionary (the _output parameter object_). An output parameter object with an `error` field is an _error object_. A composition _fails_ if it produces an error object. - -By convention, an error object returned by a composition is stripped from all fields except from the `error` field. This behavior is consistent with the OpenWhisk action semantics, e.g., the action with code `function main() { return { error: 'KO', message: 'OK' } }` outputs `{ error: 'KO' }`. - -## Combinators - -The `composer` module offers a number of combinators to define compositions: - -| Combinator | Description | Example | -| --:| --- | --- | -| [`action`](#action) | action | `composer.action('echo')` | -| [`function`](#function) | function | `composer.function(({ x, y }) => ({ product: x * y }))` | -| [`literal` or `value`](#literal) | constant value | `composer.literal({ message: 'Hello, World!' })` | -| [`sequence` or `seq`](#sequence) | sequence | `composer.sequence('hello', 'bye')` | -| [`let`](#let) | variable declarations | `composer.let({ count: 3, message: 'hello' }, ...)` | -| [`if`](#if) | conditional | `composer.if('authenticate', 'success', 'failure')` | -| [`while`](#while) | loop | `composer.while('notEnough', 'doMore')` | -| [`dowhile`](#dowhile) | loop at least once | `composer.dowhile('fetchData', 'needMoreData')` | -| [`repeat`](#repeat) | counted loop | `composer.repeat(3, 'hello')` | -| [`try`](#try) | error handling | `composer.try('divideByN', 'NaN')` | -| [`finally`](#finally) | finalization | `composer.finally('tryThis', 'doThatAlways')` | -| [`retry`](#retry) | error recovery | `composer.retry(3, 'connect')` | -| [`retain`](#retain) | persistence | `composer.retain('validateInput')` | - -The `action`, `function`, and `literal` combinators and their synonymous construct compositions respectively from actions, functions, and constant values. The other combinators combine existing compositions to produce new compositions. - -Where a composition is expected, the following shorthands are permitted: - - `name` of type `string` stands for `composer.action(name)`, - - `fun` of type `function` stands for `composer.function(fun)`, - - `null` stands for the empty sequence `composer.sequence()`. - -### Action - -`composer.action(name, [options])` is a composition with a single action named _name_. It invokes the action named _name_ on the input parameter object for the composition and returns the output parameter object of this action invocation. - -The action _name_ may specify the namespace and/or package containing the action following the usual OpenWhisk grammar. If no namespace is specified, the default namespace is assumed. If no package is specified, the default package is assumed. - -Examples: -```javascript -composer.action('hello') -composer.action('myPackage/myAction') -composer.action('/whisk.system/utils/echo') -``` -The optional `options` dictionary makes it possible to provide a definition for the action being composed: -```javascript -// specify the code for the action -composer.action('hello', { action: function main() { return { message: 'hello' } } }) -composer.action('hello', { action: "function main() { return { message: 'hello' } }" }) -composer.action('hello', { - action: { - kind: 'nodejs:default', - code: "function main() { return { message: 'hello' } }" - } -}) - -// specify a file containing the code for the action -composer.action('hello', { filename: 'hello.js' }) - -// define an action sequence -composer.action('helloAndBye', { sequence: ['hello', 'bye'] }) -``` - -### Function - -`composer.function(fun)` is a composition with a single Javascript function _fun_. It applies the specified function to the input parameter object for the composition. - - If the function returns a value of type `function`, the composition returns an error object. - - If the function throws an exception, the composition returns an error object. The exception is logged as part of the conductor action invocation. - - If the function returns a value of type other than function, the value is first converted to a JSON value using `JSON.stringify` followed by `JSON.parse`. If the resulting JSON value is not a JSON dictionary, the JSON value is then wrapped into a `{ value }` dictionary. The composition returns the final JSON dictionary. - - If the function does not return a value and does not throw an exception, the composition returns the input parameter object for the composition converted to a JSON dictionary using `JSON.stringify` followed by `JSON.parse`. - -Examples: -```javascript -composer.function(params => ({ message: 'Hello ' + params.name })) -composer.function(function (params) { return { error: 'error' } }) - -function product({ x, y }) { return { product: x * y } } -composer.function(product) -``` - -#### Environment capture - -Functions intended for compositions cannot capture any part of their declaration environment. They may however access and mutate variables in an environment consisting of the variables declared by the [composer.let](#composerletname-value-composition_1-composition_2-) combinator discussed below. - -The following is not legal: -```javascript -let name = 'Dave' -composer.function(params => ({ message: 'Hello ' + name })) -``` -The following is legal: -```javascript -composer.let({ name: 'Dave' }, composer.function(params => ({ message: 'Hello ' + name }))) -``` - -### Literal - -`composer.literal(value)` and its synonymous `composer.value(value)` output a constant JSON dictionary. This dictionary is obtained by first converting the _value_ argument to JSON using `JSON.stringify` followed by `JSON.parse`. If the resulting JSON value is not a JSON dictionary, the JSON value is then wrapped into a `{ value }` dictionary. - -The _value_ argument may be computed at composition time. For instance, the following composition captures the date at the time the composition is encoded to JSON: -```javascript -composer.literal(Date()) -``` - -### Sequence - -`composer.sequence(composition_1, composition_2, ...)` chains a series of compositions (possibly empty). - -The input parameter object for the composition is the input parameter object of the first composition in the sequence. The output parameter object of one composition in the sequence is the input parameter object for the next composition in the sequence. The output parameter object of the last composition in the sequence is the output parameter object for the composition. - -If one of the components fails, the remainder of the sequence is not executed. The output parameter object for the composition is the error object produced by the failed component. - -An empty sequence behaves as a sequence with a single function `params => params`. The output parameter object for the empty sequence is its input parameter object unless it is an error object, in which case, as usual, the error object only contains the `error` field of the input parameter object. - -### Let - -`composer.let({ name_1: value_1, name_2: value_2, ... }, composition_1_, _composition_2_, ...)` declares one or more variables with the given names and initial values, and runs runs a sequence of compositions in the scope of these declarations. - -Variables declared with `composer.let` may be accessed and mutated by functions __running__ as part of the following sequence (irrespective of their place of definition). In other words, name resolution is [dynamic](https://en.wikipedia.org/wiki/Name_resolution_(programming_languages)#Static_versus_dynamic). If a variable declaration is nested inside a declaration of a variable with the same name, the innermost declaration masks the earlier declarations. - -For example, the following composition invokes composition `composition` repeatedly `n` times. -```javascript -composer.let({ i: n }, composer.while(() => i-- > 0, composition)) -``` -Variables declared with `composer.let` are not visible to invoked actions. However, they may be passed as parameters to actions as for instance in: -```javascript -composer.let({ n: 42 }, () => ({ n }), 'increment', params => { n = params.n }) -``` - -In this example, the variable `n` is exposed to the invoked action as a field of the input parameter object. Moreover, the value of the field `n` of the output parameter object is assigned back to variable `n`. - -### If - -`composer.if(condition, consequent, [alternate], [options])` runs either the _consequent_ composition if the _condition_ evaluates to true or the _alternate_ composition if not. - -A _condition_ composition evaluates to true if and only if it produces a JSON dictionary with a field `value` with value `true`. Other fields are ignored. Because JSON values other than dictionaries are implicitly lifted to dictionaries with a `value` field, _condition_ may be a Javascript function returning a Boolean value. An expression such as `params.n > 0` is not a valid condition (or in general a valid composition). One should write instead `params => params.n > 0`. The input parameter object for the composition is the input parameter object for the _condition_ composition. - -The _alternate_ composition may be omitted. If _condition_ fails, neither branch is executed. - -The optional `options` dictionary supports a `nosave` option. If `options.nosave` is thruthy, the _consequent_ composition or _alternate_ composition is invoked on the output parameter object of the _condition_ composition. Otherwise, the output parameter object of the _condition_ composition is discarded and the _consequent_ composition or _alternate_ composition is invoked on the input parameter object for the composition. For example, the following compositions divide parameter `n` by two if `n` is even: -```javascript -composer.if(params => params.n % 2 === 0, params => { params.n /= 2 }) -composer.if(params => { params.value = params.n % 2 === 0 }, params => { params.n /= 2 }, null, { nosave: true }) -``` -In the first example, the condition function simply returns a Boolean value. The consequent function uses the saved input parameter object to compute `n`'s value. In the second example, the condition function adds a `value` field to the input parameter object. The consequent function applies to the resulting object. In particular, in the second example, the output parameter object for the condition includes the `value` field. - -While, the default `nosave == false` behavior is typically more convenient, preserving the input parameter object is not free as it counts toward the parameter size limit for OpenWhisk actions. In essence, the limit on the size of parameter objects processed during the evaluation of the condition is reduced by the size of the saved parameter object. The `nosave` option omits the parameter save, hence preserving the parameter size limit. - -### While - -`composer.while(condition, body, [options])` runs _body_ repeatedly while _condition_ evaluates to true. The _condition_ composition is evaluated before any execution of the _body_ composition. See [composer.if](#composerifcondition-consequent-alternate) for a discussion of conditions. - -A failure of _condition_ or _body_ interrupts the execution. The composition returns the error object from the failed component. - -Like `composer.if`, `composer.while` supports a `nosave` option. By default, the output parameter object of the _condition_ composition is discarded and the input parameter object for the _body_ composition is either the input parameter object for the whole composition the first time around or the output parameter object of the previous iteration of _body_. However if `options.nosave` is thruthy, the input parameter object for _body_ is the output parameter object of _condition_. Moreover, the output parameter object for the whole composition is the output parameter object of the last _condition_ evaluation. - -For instance, the following composition invoked on dictionary `{ n: 28 }` outputs `{ n: 7 }`: -```javascript -composer.while(params => params.n % 2 === 0, params => { params.n /= 2 }) -``` -For instance, the following composition invoked on dictionary `{ n: 28 }` outputs `{ n: 7, value: false }`: -```javascript -composer.while(params => { params.value = params.n % 2 === 0 }, params => { params.n /= 2 }, { nosave: true }) -``` - -### Dowhile - -`composer.dowhile(condition, body, [options])` is similar to `composer.while(body, condition, [options])` except that _body_ is invoked before _condition_ is evaluated, hence _body_ is always invoked at least once. - -### Repeat - -`composer.repeat(count, body)` invokes _body_ _count_ times. - -### Try - -`composer.try(body, handler)` runs _body_ with error handler _handler_. - -If _body_ outputs an error object, _handler_ is invoked with this error object as its input parameter object. Otherwise, _handler_ is not run. - -### Finally - -`composer.finally(body, finalizer)` runs _body_ and then _finalizer_. - -The _finalizer_ is invoked in sequence after _body_ even if _body_ returns an error object. - -### Retry - -`composer.retry(count, body)` runs _body_ and retries _body_ up to _count_ times if it fails. The output parameter object for the composition is either the output parameter object of the successful _body_ invocation or the error object produced by the last _body_ invocation. - -### Retain - -`composer.retain(body, [options])` runs _body_ on the input parameter object producing an object with two fields `params` and `result` such that `params` is the input parameter object of the composition and `result` is the output parameter object of _body_. - -An `options` dictionary object may be specified to alter the default behavior of `composer.retain` in the following ways: -- If `options.catch` is thruthy, the `retain` combinator behavior will be the same even if _body_ returns an error object. Otherwise, if _body_ fails, the output of the `retain` combinator is only the error object (i.e., the input parameter object is not preserved). -- If `options.filter` is a function, the combinator only persists the result of the function application to the input parameter object. -- If `options.field` is a string, the combinator only persists the value of the field of the input parameter object with the given name. +# Composer Package + +The Composer package consists of: +* the [composer](composer.js) Node.js module for authoring, deploying, and invoking compositions, +* the [compose](bin/compose) command for managing compositions from the command line. + +The documentation for the Composer package is organized as follows: +- [../README.md](../README.md) gives a brief introduction to compositions. +- [COMPOSER.md](COMPOSER.md) documents the `composer` module. +- [COMPOSE.md](COMPOSE.md) documents the `compose` command. +- [COMBINATORS.md](COMBINATORS.md) documents the methods of the `composer` object. +- [FORMAT.md](FORMAT.md) documents the JSON format for encoding compositions. +- The [tutorials](#tutorials) folder includes various tutorials. diff --git a/samples/demo.js b/samples/demo.js index d58cc0f..cf44a20 100644 --- a/samples/demo.js +++ b/samples/demo.js @@ -15,6 +15,6 @@ */ composer.if( - composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), - composer.action('success', { action: function main() { return { message: 'success' } } }), - composer.action('failure', { action: function main() { return { message: 'failure' } } })) \ No newline at end of file + composer.action('authenticate', { action: function ({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function () { return { message: 'success' } } }), + composer.action('failure', { action: function () { return { message: 'failure' } } })) \ No newline at end of file diff --git a/samples/demo.json b/samples/demo.json index c1b6997..1aefb82 100644 --- a/samples/demo.json +++ b/samples/demo.json @@ -5,7 +5,7 @@ "action": { "exec": { "kind": "nodejs:default", - "code": "function main({ password }) { return { value: password === 'abc123' } }" + "code": "const main = function ({ password }) { return { value: password === 'abc123' } }" } } }, @@ -14,7 +14,7 @@ "action": { "exec": { "kind": "nodejs:default", - "code": "function main() { return { message: 'success' } }" + "code": "const main = function () { return { message: 'success' } }" } } }, @@ -23,7 +23,7 @@ "action": { "exec": { "kind": "nodejs:default", - "code": "function main() { return { message: 'failure' } }" + "code": "const main = function () { return { message: 'failure' } }" } } } diff --git a/samples/node-demo.js b/samples/node-demo.js new file mode 100644 index 0000000..2007da5 --- /dev/null +++ b/samples/node-demo.js @@ -0,0 +1,31 @@ +/* + * Copyright 2017 IBM Corporation + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// require the composer module +const composer = require('@ibm-functions/composer') + +// define the composition +const composition = composer.if( + composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function main() { return { message: 'success' } } }), + composer.action('failure', { action: function main() { return { message: 'failure' } } })) + +// instantiate OpenWhisk client +const wsk = composer.openwhisk({ ignore_certs: true }) + +wsk.compositions.deploy(composition, 'demo') // deploy composition + .then(() => wsk.actions.invoke({ name: 'demo', params: { password: 'abc123' }, blocking: true })) // invoke composition + .then(({ response }) => console.log(JSON.stringify(response.result, null, 4)), console.error) From 5c052911b2e171ad0f03755ab7d770adcdbc882d Mon Sep 17 00:00:00 2001 From: Olivier Tardieu Date: Tue, 27 Mar 2018 13:54:30 -0400 Subject: [PATCH 2/3] Expand documentation --- README.md | 8 ++--- docs/COMBINATORS.md | 2 +- docs/COMPOSE.md | 12 +++---- docs/COMPOSER.md | 22 +++++------- docs/COMPOSITIONS.md | 72 ++++++++++++++++++++++++++++++++++++++++ docs/README.md | 8 ++--- docs/tutorials/README.md | 10 ++++++ samples/demo.js | 2 +- samples/node-demo.js | 6 ++-- 9 files changed, 110 insertions(+), 32 deletions(-) create mode 100644 docs/COMPOSITIONS.md create mode 100644 docs/tutorials/README.md diff --git a/README.md b/README.md index 513cf24..e2b66cf 100644 --- a/README.md +++ b/README.md @@ -18,7 +18,7 @@ tool called [IBM Cloud Shell](https://github.com/ibm-functions/shell), or just _Shell_. Shell offers a CLI and graphical interface for fast, incremental, iterative, and local development of serverless applications. While we recommend using Shell, Shell is not required to work with compositions. Compositions may -be managed using a combination of the Composer [compose](bin/compose) command +be managed using a combination of the Composer [compose](docs/COMPOSE.md) command (for deployment) and the [OpenWhisk CLI](https://console.bluemix.net/openwhisk/learn/cli) (for configuration, invocation, and life-cycle management). @@ -31,9 +31,9 @@ of an action (e.g., default parameters, limits, blocking invocation, web export). This repository includes: -* the [composer](composer.js) Node.js module for authoring compositions using +* the [composer](docs/COMPOSER.md) Node.js module for authoring compositions using JavaScript, -* the [compose](bin/compose) command for deploying compositions, +* the [compose](docs/COMPOSE.md) command for deploying compositions, * [documentation](docs), [examples](samples), and [tests](test). Composer and Shell are currently available as _IBM Research previews_. As @@ -81,7 +81,7 @@ composer.if('authenticate', 'success', 'failure') ## Deploying a composition -One way to deploy a composition is to use the `compose` command: +One way to deploy a composition is to use the [compose](docs/COMPOSE.md) command: ``` compose demo.js --deploy demo ``` diff --git a/docs/COMBINATORS.md b/docs/COMBINATORS.md index 7034a09..972913b 100644 --- a/docs/COMBINATORS.md +++ b/docs/COMBINATORS.md @@ -18,7 +18,7 @@ The `composer` module offers a number of combinators to define compositions: | [`retry`](#retry) | error recovery | `composer.retry(3, 'connect')` | | [`retain`](#retain) | persistence | `composer.retain('validateInput')` | -The `action`, `function`, and `literal` combinators and their synonymous construct compositions respectively from actions, functions, and constant values. The other combinators combine existing compositions to produce new compositions. +The `action`, `function`, and `literal` combinators construct compositions respectively from actions, functions, and constant values. The other combinators combine existing compositions to produce new compositions. ## Shorthands diff --git a/docs/COMPOSE.md b/docs/COMPOSE.md index 2dd8868..1a5956b 100644 --- a/docs/COMPOSE.md +++ b/docs/COMPOSE.md @@ -2,7 +2,7 @@ The `compose` command makes it possible to deploy compositions from the command line. -The `compose` command is intended is as a minimal complement to the OpenWhisk CLI. The OpenWhisk CLI already has the capability to configure, invoke, and delete compositions (since these are just OpenWhisk actions) but lacks the capability to create composition actions. The `compose` command bridges this gap making it possible to deploy compositions as part of the development cycle or in shell scripts. It is not a replacement for the OpenWhisk CLI however as it does not duplicate existing OpenWhisk CLI capabilities. For a much richer devops experience, we recommend using [Shell](https://github.com/ibm-functions/shell). +The `compose` command is intended as a minimal complement to the OpenWhisk CLI. The OpenWhisk CLI already has the capability to configure, invoke, and delete compositions (since these are just OpenWhisk actions) but lacks the capability to create composition actions. The `compose` command bridges this gap. It makes it possible to deploy compositions as part of the development cycle or in shell scripts. It is not a replacement for the OpenWhisk CLI however as it does not duplicate existing OpenWhisk CLI capabilities. Moreover, for a much richer developer experience, we recommend using [Shell](https://github.com/ibm-functions/shell). ## Usage @@ -28,7 +28,7 @@ The `compose` command has three mode of operation: - When the `--deploy` option is specified, the command deploys the composition given the desired name for the composition. - When the `--encode` option is specified, the command returns the Javascript code for the [conductor action](https://github.com/apache/incubator-openwhisk/blob/master/docs/conductors.md) for the composition. -## Encoding +## JSON format By default, the `compose` command evaluates the composition code and outputs the resulting JSON dictionary: ``` @@ -114,8 +114,8 @@ compose demo.json --deploy demo ok: created actions /_/authenticate,/_/success,/_/failure,/_/demo ``` The `compose` command synthesizes and deploys a conductor action that implements the -composition with the given name. It also deploys the composed actions if -definitions are provided for them as part of the composition. +composition with the given name. It also deploys the composed actions for which +definitions are provided as part of the composition. The `compose` command outputs the list of deployed actions or an error result. If an error occurs during deployment, the state of the various actions is unknown. @@ -129,9 +129,9 @@ Like the OpenWhisk CLI, the `compose` command supports the following flags for s -u, --auth KEY authorization KEY -i, --insecure bypass certificate checking ``` -If the `--apihost` flag is absent, the environment variable `__OW_API_HOST` is used in its place. If neither is available, the `compose` command attempts to obtain the api host from the whisk property file for the current user. +If the `--apihost` flag is absent, the environment variable `__OW_API_HOST` is used in its place. If neither is available, the `compose` command extracts the `APIHOST` key from the whisk property file for the current user. -If the `--auth` flag is absent, the environment variable `__OW_API_KEY` is used in its place. If neither is available, the `compose` command attempts to obtain the authorization key information from the whisk property file for the current user. +If the `--auth` flag is absent, the environment variable `__OW_API_KEY` is used in its place. If neither is available, the `compose` command extracts the `AUTH` key from the whisk property file for the current user. The default path for the whisk property file is `$HOME/.wskprops`. It can be altered by setting the `WSK_CONFIG_FILE` environment variable. diff --git a/docs/COMPOSER.md b/docs/COMPOSER.md index 4217df3..1a98918 100644 --- a/docs/COMPOSER.md +++ b/docs/COMPOSER.md @@ -2,17 +2,13 @@ The [`composer`](../composer.js) Node.js module makes it possible define, deploy, and invoke compositions. -- [Installation](#installation) -- [Example](#example) -- [Combinators](#combinator-methods) -- [Deployment](#deployment) - ## Installation To install the `composer` module, use the Node Package Manager: ``` npm install @ibm-functions/composer ``` +To take advantage of the `compose` command, it may be useful to install the module globally as well (`-g` option). ## Example @@ -23,9 +19,9 @@ const composer = require('@ibm-functions/composer') // define the composition const composition = composer.if( - composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), - composer.action('success', { action: function main() { return { message: 'success' } } }), - composer.action('failure', { action: function main() { return { message: 'failure' } } })) + composer.action('authenticate', { action: function ({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function () { return { message: 'success' } } }), + composer.action('failure', { action: function () { return { message: 'failure' } } })) // instantiate OpenWhisk client const wsk = composer.openwhisk({ ignore_certs: true }) @@ -54,18 +50,18 @@ The `composer` object offers an extension to the [OpenWhisk Client for Javascrip ## OpenWhisk method -A client instance is obtained by invoking `composer.openwhisk(options)`, for instance with: +A client instance is obtained by invoking `composer.openwhisk([options])`, for instance with: ```javascript const wsk = composer.openwhisk({ ignore_certs: true }) ``` -The specific OpenWhisk deployment to use may be specified via options, environment variables, or the OpenWhisk property file. Options have priority over environment variables, which have priority over the OpenWhisk property file. Options and environment variables are documented [here](https://github.com/apache/incubator-openwhisk-client-js#constructor-options). The default path for the whisk property file is `$HOME/.wskprops`. It can be altered by setting the `WSK_CONFIG_FILE` environment variable. +The specific OpenWhisk deployment to use may be specified via the optional `options` argument, environment variables, or the OpenWhisk property file. Options have priority over environment variables, which have priority over the OpenWhisk property file. Options and environment variables are documented [here](https://github.com/apache/incubator-openwhisk-client-js#constructor-options). The default path for the whisk property file is `$HOME/.wskprops`. It can be altered by setting the `WSK_CONFIG_FILE` environment variable. The `composer` module adds to the OpenWhisk client instance a new top-level category named `compositions` with a method named `deploy`. ## Deploy method -`wsk.deploy(composition, [name])` deploys the composition `composition` with name `name`. More precisely, it successively deploys all the actions and compositions defined in `composition` as well as `composition` itself. +`wsk.deploy(composition, [name])` deploys the composition `composition`, giving name `name` to the corresponding conductor action. More precisely, it successively deploys all the actions and compositions defined in `composition` as well as `composition` itself. The compositions are encoded into conductor actions prior to deployment. In other words, the `deploy` method deploys one or several actions. @@ -75,9 +71,9 @@ The `deploy` method deletes the deployed actions before recreating them if neces The `name` argument may be omitted if the `composition` consists of a single action invocation. In this case, `deploy` method only deploys the actions and compositions whose definitions are nested inside `composition`. -## Invoke, Update, Delete +## Invoke, Update, and Delete methods -Since compositions are deployed as conductor actions, other management tasks for compositions can be implemented by invoking methods of `wsk.actions`, for instance: +Since compositions are deployed as conductor actions, other management tasks for compositions can be achieved by invoking methods of `wsk.actions`, for instance: ```javascript wsk.actions.delete('demo') ``` diff --git a/docs/COMPOSITIONS.md b/docs/COMPOSITIONS.md new file mode 100644 index 0000000..88f33bd --- /dev/null +++ b/docs/COMPOSITIONS.md @@ -0,0 +1,72 @@ +# Compositions + +Composer makes it possible to assemble actions into rich workflows called _compositions_. An example composition is described in [../README.md](../README.md). + +## Control flow + +Compositions can express the control flow of typical a sequential imperative programming language: sequences, conditionals, loops, error handling. This control flow is specified using _combinator_ methods such as: +- `composer.sequence(firstAction, secondAction)` +- `composer.if(conditionAction, consequentAction, alternateAction)` +- `composer.try(bodyAction, handlerAction)` + +Combinators are described in [COMBINATORS.md](COMBINATORS.md). + +## Composition objects + +Combinators return composition objects. Compositions object offer several helper methods described below: +- `composition.named(name)` +- `composition.encode([name])` + +## Parameter objects and error objects + +A composition, like any action, accepts a JSON dictionary (the _input parameter object_) and produces a JSON dictionary (the _output parameter object_). An output parameter object with an `error` field is an _error object_. A composition _fails_ if it produces an error object. + +By convention, an error object returned by a composition is stripped from all fields except from the `error` field. This behavior is consistent with the OpenWhisk action semantics, e.g., the action with code `function main() { return { error: 'KO', message: 'OK' } }` outputs `{ error: 'KO' }`. + +Error objects play a specific role as they interrupt the normal flow of execution, akin to exceptions in traditional programming languages. For instance, if a component of a sequence returns an error object, the remainder of the sequence is not executed. Moreover, if the sequence is enclosed in an error handling composition like a `composer.try(sequence, handler)` combinator, the execution continues with the error handler. + +## Data flow + +The invocation of a composition triggers a series of computations (possibly empty, e.g., for the empty sequence) obtained by chaining the components of the composition along the path of execution. The input parameter object for the composition is the input parameter object of the first component in the chain. The output parameter object of a component in the chain is the input parameter object for the next component if any or the output parameter object for the composition if this is the final component in the chain. + +For example, the composition `composer.sequence('triple', 'increment')` invokes the `increment` action on the output of the `triple` action. + +## Components + +Components of a compositions can be actions, Javascript functions, or compositions. + +Javascript functions can be viewed as simple, anonymous actions that do not need to be deployed and managed separately from the composition they belong to. Functions are typically used to alter a parameter object between two actions that expect different schemas. + +Compositions may be nested inside compositions in two ways. First, combinators can be nested, e.g., +```javascript +composer.if('isEven', 'half', composer.sequence('triple', 'increment')) +``` +Second, compositions can reference other compositions by name. For instance, assuming we deploy the sequential composition of the `triple` and `increment` actions as the composition `tripleAndIncrement`, the following code behaves identically to the previous example: +```javascript +composer.if('isEven', 'half', 'tripleAndIncrement') +``` +Observe however, that the behavior of the second composition would be altered if we redefine the `tripleAndIncrement` composition to do something else, whereas the first example would not be affected. + +## Nested declarations + +A composition can embed the definitions of none, some, or all the composed actions as illustrated in [demo.js](../samples/demo.js): +```javascript +composer.if( + composer.action('authenticate', { action: function ({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function () { return { message: 'success' } } }), + composer.action('failure', { action: function () { return { message: 'failure' } } })) +) +``` +Deploying such a composition deploys the embedded actions. + +A composition can also include the definition of another composition thank to the `named` method on composition objects. +```javascript +composer.if('isEven', 'half', composer.sequence('triple', 'increment').named('tripleAndIncrement')) +``` +In this example, the `composer.sequence('triple', 'increment')` composition is given the name `tripleAndIncrement` and the enclosing composition references the `tripleAndIncrement` composition by name. In other words, deploying this composition actually deploys two compositions: +- a composition named `tripleAndIncrement` defined as `composer.sequence('triple', 'increment')`, and +- a composition defined as `composer.if('isEven', 'half', 'tripleAndIncrement')` whose name will be specified as deployment time. + +## Serialization and deserialization + + Compositions objects can be serialized to JSON objects by invoking `JSON.stringify` on them. Serialized compositions can be deserialized to composition objects using the `composer.deserialize(serializedComposition)` method. The JSON format is documented in [FORMAT.md](FORMAT.md). diff --git a/docs/README.md b/docs/README.md index c270c5d..ec4d3b2 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,13 +1,13 @@ # Composer Package The Composer package consists of: -* the [composer](composer.js) Node.js module for authoring, deploying, and invoking compositions, -* the [compose](bin/compose) command for managing compositions from the command line. +* the [composer](../composer.js) Node.js module for authoring, deploying, and invoking compositions, +* the [compose](../bin/compose) command for managing compositions from the command line. The documentation for the Composer package is organized as follows: -- [../README.md](../README.md) gives a brief introduction to compositions. +- [COMPOSITIONS.md](COMPOSITIONS.md) gives a brief introduction to compositions. - [COMPOSER.md](COMPOSER.md) documents the `composer` module. - [COMPOSE.md](COMPOSE.md) documents the `compose` command. - [COMBINATORS.md](COMBINATORS.md) documents the methods of the `composer` object. - [FORMAT.md](FORMAT.md) documents the JSON format for encoding compositions. -- The [tutorials](#tutorials) folder includes various tutorials. +- The [tutorials](tutorials) folder includes various tutorials. diff --git a/docs/tutorials/README.md b/docs/tutorials/README.md new file mode 100644 index 0000000..1070fc3 --- /dev/null +++ b/docs/tutorials/README.md @@ -0,0 +1,10 @@ +# Tutorials + +This folder contains a few tutorials to get started: +* [Introduction to Serverless + Composition](introduction/README.md): Setting up your + programming environment and getting started with Shell and Composer. +* [Building a Translation Slack Bot with Serverless + Composition](translateBot/README.md): A more advanced tutorial + using Composition to build a serverless Slack chatbot that does language + translation. diff --git a/samples/demo.js b/samples/demo.js index cf44a20..71cc939 100644 --- a/samples/demo.js +++ b/samples/demo.js @@ -17,4 +17,4 @@ composer.if( composer.action('authenticate', { action: function ({ password }) { return { value: password === 'abc123' } } }), composer.action('success', { action: function () { return { message: 'success' } } }), - composer.action('failure', { action: function () { return { message: 'failure' } } })) \ No newline at end of file + composer.action('failure', { action: function () { return { message: 'failure' } } })) diff --git a/samples/node-demo.js b/samples/node-demo.js index 2007da5..d36c60b 100644 --- a/samples/node-demo.js +++ b/samples/node-demo.js @@ -19,9 +19,9 @@ const composer = require('@ibm-functions/composer') // define the composition const composition = composer.if( - composer.action('authenticate', { action: function main({ password }) { return { value: password === 'abc123' } } }), - composer.action('success', { action: function main() { return { message: 'success' } } }), - composer.action('failure', { action: function main() { return { message: 'failure' } } })) + composer.action('authenticate', { action: function ({ password }) { return { value: password === 'abc123' } } }), + composer.action('success', { action: function () { return { message: 'success' } } }), + composer.action('failure', { action: function () { return { message: 'failure' } } })) // instantiate OpenWhisk client const wsk = composer.openwhisk({ ignore_certs: true }) From 4107a8996a5acdbb9dbb21c0b5089172f2865500 Mon Sep 17 00:00:00 2001 From: Olivier Tardieu Date: Tue, 27 Mar 2018 17:20:11 -0400 Subject: [PATCH 3/3] More documentation --- composer.js | 2 +- docs/COMPOSITIONS.md | 22 +++++++++++++++++----- 2 files changed, 18 insertions(+), 6 deletions(-) diff --git a/composer.js b/composer.js index 3827e14..74f43f7 100644 --- a/composer.js +++ b/composer.js @@ -109,7 +109,6 @@ class Composition { if (arguments.length > 1) throw new ComposerError('Too many arguments') if (typeof name !== 'undefined' && typeof name !== 'string') throw new ComposerError('Invalid argument', name) const obj = typeof name === 'string' ? this.named(name) : this - if (obj.composition.length !== 1 || obj.composition[0].type !== 'action') throw new ComposerError('Cannot encode anonymous composition') return new Composition(obj.composition, null, obj.actions.map(encode)) } } @@ -123,6 +122,7 @@ class Compositions { if (arguments.length > 2) throw new ComposerError('Too many arguments') if (!(composition instanceof Composition)) throw new ComposerError('Invalid argument', composition) const obj = composition.encode(name) + if (obj.composition.length !== 1 || obj.composition[0].type !== 'action') throw new ComposerError('Cannot deploy anonymous composition') return obj.actions.reduce((promise, action) => promise.then(() => this.actions.delete(action).catch(() => { })) .then(() => this.actions.update(action)), Promise.resolve()) } diff --git a/docs/COMPOSITIONS.md b/docs/COMPOSITIONS.md index 88f33bd..7663ea9 100644 --- a/docs/COMPOSITIONS.md +++ b/docs/COMPOSITIONS.md @@ -14,8 +14,8 @@ Combinators are described in [COMBINATORS.md](COMBINATORS.md). ## Composition objects Combinators return composition objects. Compositions object offer several helper methods described below: -- `composition.named(name)` -- `composition.encode([name])` +- [`composition.named(name)`](#nested-declarations) to nest one composition definition inside another, +- [`composition.encode([name])`](#conductor-actions) to synthesize conductor actions for compositions. ## Parameter objects and error objects @@ -27,15 +27,20 @@ Error objects play a specific role as they interrupt the normal flow of executio ## Data flow -The invocation of a composition triggers a series of computations (possibly empty, e.g., for the empty sequence) obtained by chaining the components of the composition along the path of execution. The input parameter object for the composition is the input parameter object of the first component in the chain. The output parameter object of a component in the chain is the input parameter object for the next component if any or the output parameter object for the composition if this is the final component in the chain. +The invocation of a composition triggers a series of computations (possibly empty, e.g., for the empty sequence) obtained by chaining the components of the composition along the path of execution. The input parameter object for the composition is the input parameter object of the first component in the chain. The output parameter object of a component in the chain is typically the input parameter object for the next component if any or the output parameter object for the composition if this is the final component in the chain. For example, the composition `composer.sequence('triple', 'increment')` invokes the `increment` action on the output of the `triple` action. +Some combinators however are designed to alter the default flow of data. For instance, the `composer.retain(myAction)` composition returns a combination of the input parameter object and the output parameter object of `myAction`. + ## Components Components of a compositions can be actions, Javascript functions, or compositions. -Javascript functions can be viewed as simple, anonymous actions that do not need to be deployed and managed separately from the composition they belong to. Functions are typically used to alter a parameter object between two actions that expect different schemas. +Javascript functions can be viewed as simple, anonymous actions that do not need to be deployed and managed separately from the composition they belong to. Functions are typically used to alter a parameter object between two actions that expect different schemas, as in: +```javascript +composer.if('getUserNameAndPassword', params => ({ key = btoa(params.user + ':' + params.password) }), 'authenticate') +``` Compositions may be nested inside compositions in two ways. First, combinators can be nested, e.g., ```javascript @@ -69,4 +74,11 @@ In this example, the `composer.sequence('triple', 'increment')` composition is g ## Serialization and deserialization - Compositions objects can be serialized to JSON objects by invoking `JSON.stringify` on them. Serialized compositions can be deserialized to composition objects using the `composer.deserialize(serializedComposition)` method. The JSON format is documented in [FORMAT.md](FORMAT.md). + Compositions objects can be serialized to JSON dictionaries by invoking `JSON.stringify` on them. Serialized compositions can be deserialized to composition objects using the `composer.deserialize(serializedComposition)` method. The JSON format is documented in [FORMAT.md](FORMAT.md). + In short, the JSON dictionary for a composition contains a representation of the syntax tree for this composition as well as the definition of all the actions and compositions embedded inside the composition. + +## Conductor actions + +Compositions are implemented by means of OpenWhisk [conductor actions](https://github.com/apache/incubator-openwhisk/blob/master/docs/conductors.md). The conductor actions are implicitly synthesized when compositions are deployed using the `compose` command or the `composer` module. The `encode` method on compositions may also be used to generate the conductor actions for compositions. +- `composition.encode()` replaces all the compositions nested inside `composition` with conductor actions. It does not alter the composition itself, only its components. +- `composition.encode(name)` is a shorthand for `composition.named(name).encode()`. It encode the composition and all its components into conductor actions, replacing the composition with an invocation of the action named `name` bound to the conductor action for `composition`.