Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Tutorial 3. Extended Example
In this tutorial we will begin to see some of the advantages of programming with data flows. We will do this by incrementally extending in various ways the Hello World example from Tutorial 2. Hello World.
You have probably already tried modifying the hello1.yaml workflow file to output a different greeting. Indeed, this can be done easily by modifying the greeting given to the value constant of the CreateGreeting node. For example, to introduce a Shakespearean theme modify the sixth line of the CreateGreeting node as shown and save the file as hello2.yaml:
imports: - classpath:/common/groovy/actors.yaml - classpath:/common/directors.yaml components: - id: HelloWorld type: Workflow properties: director: !ref MTDataDrivenDirector nodes: - !ref CreateGreeting - !ref RenderGreeting - id: CreateGreeting type: Node properties: actor: !ref ConstantSource sequences: value: - Hello World - Good Afternoon, Cosmos - Good night, and good luck outflows: value: /messages/greeting/ - id: RenderGreeting type: Node properties: actor: !ref PrintStreamWriter inflows: message: /messages/greeting/
Now run the workflow as before, and you should see something like the following:
$ restflow -f hello2.yaml Good night, sweet prince. $
Creating a sequence of greetings
Perhaps more interesting, you can provide CreateGreeting with a list of greetings to send. Replace the constants property of the CreateGreeting node with a sequences property, and assign it a list of greetings as shown below. Save the new workflow as hello3.yaml. (Only the CreateGreeting component is shown here. The rest of the workflow definition is still required).
- id: CreateGreeting type: Node properties: actor: !ref ConstantSource sequences: value: - Hello World - Good Afternoon, Cosmos - Good night, and good luck outflows: value: /messages/greeting/
Running this workflow will then yield the following:
$ restflow -f hello3.yaml Hello World Good Afternoon, Cosmos Good night, and good luck $
What happened? The CreateGreeting component stepped three times, once for each item in the list of greetings provided to it via the value sequence. The RenderMessage node received each greeting from CreateGreeting and printed the messages to the terminal. Consequently, RenderMessage, stepped three times as well.
This illustrates a key point. RestFlow workflows and components generally are data driven. They operate on all data provided to them--through constants, sequences, or inflows--and keep working until they have no more data to process.
Adding some emphasis
So far we have only printed some constant strings to the terminal. Now we will add some basic computation. Add the following new component somewhere within the components section of the workflow specification file. The order of the components in the file does not matter.
- id: EmphasizeGreeting type: Node properties: actor: !ref StringConcatenator inflows: stringOne: /messages/greeting/ constants: stringTwo: '!!' outflows: concatenatedString: /messages/emphasizedGreeting/
The function of this new node generally is to take two strings, concatenate them, and output the result. In this case the inflows and constants properties configure it to take the greetings provided by CreateGreeting and append two exclamation points to each.
Now change the flow expression for the message inflow on RenderGreeting to /messages/emphasizedGreeting/. This configures RenderGreeting to receive the outputs of the EmphasizeGreeting node, rather than those of CreateGreeting:
- id: RenderGreeting type: Node properties: actor: !ref PrintStreamWriter inflows: message: /messages/emphasizedGreeting/
Finally, add a reference to the new EmphasizeGreeting component to the list of nodes given to the HelloWorld component. This tells RestFlow that EmphasizeGreeting is one of the nodes that must be triggered during a run of the workflow. Note that here, too, the order of components in the list does not matter:
- id: HelloWorld type: Workflow properties: director: !ref MTDataDrivenDirector nodes: - !ref CreateGreeting - !ref RenderGreeting - !ref EmphasizeGreeting
Save the modified workflow as hello4.yaml and then run it. You should see the following output:
$ restflow -f hello4.yaml Hello World!! Good Afternoon, Cosmos!! Good night, and good luck!! $
The Hello World workflow is now a functioning three-stage software pipeline.
Two data flows to one node
You may have noticed that the names of the inflow and constant for EmphasizeGreeting are stringOne and stringTwo, respectively. The function of EmphasizeGreeting is to concatenate stringOne and StringTwo to produce concatentatedString (the name of the outflow of EmphasizeGreeting). Must stringOne be provided a dynamic value, sent by an upstream workflow node as it is here? And does stringTwo need to be provided a static value via a parameter? The answer to both questions is no. Any information used by a workflow node can in fact come from one of three different places depending on how it is configured.
First, it can be provided via a static value, as stringTwo is here. By static, we mean that the value(s) to be provided are fixed before the workflow is run. To provide a static value for stringOne, you only need to delete the stringOne inflow and create a stringOne constant analogous to the stringTwo constant. (But then, of course, the node would not process any information from CreateGreeting!) Second, data can be provided to a node via an inflow. Finally, the node may use a default value assigned in the definition of the actor underlying the node. We will look at actors and how they are defined in Chapter 5.
When you declare more than one inflow for a node, the data flowing to the two inflows may come from the same or different upstream workflow nodes. To see this in practice, consider this updated version of Hello World, hello5.yaml:
imports: - actors:actors.yaml - classpath:/common/directors.yaml components: - id: HelloWorld type: Workflow properties: director: !ref MTDataDrivenDirector nodes: - !ref CreateGreeting - !ref EmphasizeGreeting - !ref RenderGreeting - !ref CreateEmphasis - id: CreateGreeting type: Node properties: actor: !ref ConstantSource sequences: value: - Hello World - Good Afternoon, Cosmos - Good night, and good luck outflows: value: /messages/greeting/ - id: CreateEmphasis type: Node properties: actor: !ref ConstantSource sequences: value: - "!" - "!!" - "!!!" outflows: value: /messages/emphasis/ - id: EmphasizeGreeting type: Node properties: actor: !ref StringConcatenator inflows: stringOne: /messages/greeting/ stringTwo: /messages/emphasis/ outflows: concatenatedString: /messages/emphasizedGreeting/ - id: RenderGreeting type: Node properties: actor: !ref PrintStreamWriter inflows: message: /messages/emphasizedGreeting/
We have added a new component, CreateEmphasis, that produces three strings, "!", "!!", and "!!!", and sends them to the new stringTwo inflow of the EmphasizeGreeting node. Running the updated workflow gives these results:
$ restflow -f hello5.yaml Hello World! Good Afternoon, Cosmos!! Good night, and good luck!!! $
When workflows become more complex than this it is often useful to see what data each node produces during a run and in what order. Tutorial 4. Workflow Traces will show you how to do this.