Tutorial 3. Extended Example

tmcphillips edited this page Feb 15, 2013 · 10 revisions

In this tutorial we will begin to see some of the advantages of programming with data flows. We will do this by incrementally extending in various ways the Hello World example from Tutorial 2. Hello World.

You have probably already tried modifying the hello1.yaml workflow file to output a different greeting. Indeed, this can be done easily by modifying the greeting given to the value constant of the CreateGreeting node. For example, to introduce a Shakespearean theme modify the sixth line of the CreateGreeting node as shown and save the file as hello2.yaml:

imports:

- classpath:/common/groovy/actors.yaml
- classpath:/common/directors.yaml

components:

- id: HelloWorld
  type: Workflow
  properties:
    director: !ref MTDataDrivenDirector
    nodes:
    - !ref CreateGreeting
    - !ref RenderGreeting

- id: CreateGreeting
  type: Node
  properties:   
    actor: !ref ConstantSource
    sequences:
      value:
      - Hello World
      - Good Afternoon, Cosmos
      - Good night, and good luck
    outflows:
      value: /messages/greeting/

- id: RenderGreeting
  type: Node
  properties:
    actor: !ref PrintStreamWriter
    inflows:
      message: /messages/greeting/

Now run the workflow as before, and you should see something like the following:

$ restflow -f hello2.yaml
Good night, sweet prince.
$

Creating a sequence of greetings

Perhaps more interesting, you can provide CreateGreeting with a list of greetings to send. Replace the constants property of the CreateGreeting node with a sequences property, and assign it a list of greetings as shown below. Save the new workflow as hello3.yaml. (Only the CreateGreeting component is shown here. The rest of the workflow definition is still required).

- id: CreateGreeting
  type: Node
  properties:   
    actor: !ref ConstantSource
    sequences:
      value:
      - Hello World
      - Good Afternoon, Cosmos
      - Good night, and good luck
    outflows:
      value: /messages/greeting/

Running this workflow will then yield the following:

$ restflow -f hello3.yaml
Hello World 
Good Afternoon, Cosmos
Good night, and good luck
$ 

What happened? The CreateGreeting component stepped three times, once for each item in the list of greetings provided to it via the value sequence. The RenderMessage node received each greeting from CreateGreeting and printed the messages to the terminal. Consequently, RenderMessage, stepped three times as well.

This illustrates a key point. RestFlow workflows and components generally are data driven. They operate on all data provided to them--through constants, sequences, or inflows--and keep working until they have no more data to process.

Adding some emphasis

So far we have only printed some constant strings to the terminal. Now we will add some basic computation. Add the following new component somewhere within the components section of the workflow specification file. The order of the components in the file does not matter.

- id: EmphasizeGreeting
  type: Node
  properties:
    actor: !ref StringConcatenator 
    inflows:
      stringOne: /messages/greeting/
    constants:
      stringTwo: '!!'
    outflows:
      concatenatedString: /messages/emphasizedGreeting/

The function of this new node generally is to take two strings, concatenate them, and output the result. In this case the inflows and constants properties configure it to take the greetings provided by CreateGreeting and append two exclamation points to each.

Now change the flow expression for the message inflow on RenderGreeting to /messages/emphasizedGreeting/. This configures RenderGreeting to receive the outputs of the EmphasizeGreeting node, rather than those of CreateGreeting:

- id: RenderGreeting
  type: Node
  properties:
    actor: !ref PrintStreamWriter
    inflows:
      message: /messages/emphasizedGreeting/

Finally, add a reference to the new EmphasizeGreeting component to the list of nodes given to the HelloWorld component. This tells RestFlow that EmphasizeGreeting is one of the nodes that must be triggered during a run of the workflow. Note that here, too, the order of components in the list does not matter:

- id: HelloWorld
  type: Workflow
  properties:
    director: !ref MTDataDrivenDirector
    nodes:
    - !ref CreateGreeting
    - !ref RenderGreeting
    - !ref EmphasizeGreeting

Save the modified workflow as hello4.yaml and then run it. You should see the following output:

$ restflow -f hello4.yaml
Hello World!!
Good Afternoon, Cosmos!!
Good night, and good luck!!
$

The Hello World workflow is now a functioning three-stage software pipeline.

Two data flows to one node

You may have noticed that the names of the inflow and constant for EmphasizeGreeting are stringOne and stringTwo, respectively. The function of EmphasizeGreeting is to concatenate stringOne and StringTwo to produce concatentatedString (the name of the outflow of EmphasizeGreeting). Must stringOne be provided a dynamic value, sent by an upstream workflow node as it is here? And does stringTwo need to be provided a static value via a parameter? The answer to both questions is no. Any information used by a workflow node can in fact come from one of three different places depending on how it is configured.

First, it can be provided via a static value, as stringTwo is here. By static, we mean that the value(s) to be provided are fixed before the workflow is run. To provide a static value for stringOne, you only need to delete the stringOne inflow and create a stringOne constant analogous to the stringTwo constant. (But then, of course, the node would not process any information from CreateGreeting!) Second, data can be provided to a node via an inflow. Finally, the node may use a default value assigned in the definition of the actor underlying the node. We will look at actors and how they are defined in Chapter 5.

When you declare more than one inflow for a node, the data flowing to the two inflows may come from the same or different upstream workflow nodes. To see this in practice, consider this updated version of Hello World, hello5.yaml:

imports:

- actors:actors.yaml
- classpath:/common/directors.yaml

components: 

- id: HelloWorld
  type: Workflow
  properties:
    director: !ref MTDataDrivenDirector
    nodes:
    - !ref CreateGreeting
    - !ref EmphasizeGreeting
    - !ref RenderGreeting
    - !ref CreateEmphasis

- id: CreateGreeting
  type: Node
  properties:  
    actor: !ref ConstantSource
    sequences:
      value:
      - Hello World
      - Good Afternoon, Cosmos
      - Good night, and good luck
    outflows:
      value: /messages/greeting/
 
- id: CreateEmphasis
  type: Node
  properties:  
    actor: !ref ConstantSource
    sequences:
      value:
      - "!"
      - "!!"
      - "!!!"
    outflows:
      value: /messages/emphasis/
 
- id: EmphasizeGreeting
  type: Node
  properties:
    actor: !ref StringConcatenator 
    inflows:
      stringOne: /messages/greeting/
      stringTwo: /messages/emphasis/
    outflows:
      concatenatedString: /messages/emphasizedGreeting/
       
- id: RenderGreeting
  type: Node
  properties:
    actor: !ref PrintStreamWriter
    inflows:
      message: /messages/emphasizedGreeting/

We have added a new component, CreateEmphasis, that produces three strings, "!", "!!", and "!!!", and sends them to the new stringTwo inflow of the EmphasizeGreeting node. Running the updated workflow gives these results:

$ restflow -f hello5.yaml
Hello World!
Good Afternoon, Cosmos!!
Good night, and good luck!!!
$

When workflows become more complex than this it is often useful to see what data each node produces during a run and in what order. Tutorial 4. Workflow Traces will show you how to do this.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.