Flow Design in Xtend
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
FlowDesignInXtend-ActiveAnnotations
FlowDesignInXtend
.gitignore
.travis.yml
README.md

README.md

XtendFlow Build Status Download Get automatic notifications about new "de.grammarcraft.xtend.flow" versions

This library enables Xtend for Flow Design based programming, the natural approach for IODA Architectures.

The notation tries to follow the intention of the inventor of Flow Design Ralf Westphal as close as possible. This programming paradigm and its notation was initially explained by him in English in his articles "Flow-Design Cheat Sheet – Part I, Notation" and "Flow-Design Cheat Sheet – Part II, Notation". He introduces the underlying paradigm in detail in his article series "The Incremental Architect´s Napkin" and in his book "Messaging as a Programming Model". The according architectural approach he describes in his article "The IODA Architecture". However, this architectural approach is more general and is going beyond Flow Design and may be applicable to any programming platform. All, absolutely worth to read!

Who understands German may read my blog article on how this notation was initially mapped to Xtend.

However, since version 0.2.0 the Xtend Flow library supports a notation which takes the advantages of Xtend regarding operator overriding forming up a tiny DSL which is very concise and quite closer to the original flow notation than my first attempt. Since version 0.4.0 the libraries domain language officially follows the nomenclature of the IODA Architecture.

Xtend Flow DSL

The Xtend flow DSL consists of active annotations for declaring function units as operations and integrations including their input and output ports, as well as of operators for wiring ports and for forwarding message to output ports.

Function Unit Annotation @Unit

The annotation @Unit is applied to Xtend classes marking them as function units and declares input and output ports for them. E.g., the function unit

Simple Function Unit

would be declared in Xtend as the follows:

@Unit(
    inputPorts = #[
        @Port(name="input", type=String)
    ],
    outputPorts = #[
        @Port(name="output", type=String)
    ]
)
class ToUpper {
...
}

Organizing software systems in function units is one of the main design keys of Flow Design. It follows closely the Principle of Mutual Oblivion (PoMO). There is no dependency between the implementation of different function units. Function units does not know each other. There are only global dependencies to message data types flowing through the ports.

Operation Unit Annotation @Operation

Marking an function unit with annotation @Operation declares an operation in the sense of the IODA architecture and automatically adds particular Xtend class properties and class operations for allowing to program message processing and message forwarding.

The ToUpper function unit from above is intended to be a operation unit in the sense of IODA. Therefore, this function unit should be defined with the @Operation annotation in front:

@Operation @Unit(
    inputPorts = #[
        @Port(name="input", type=String)
    ],
    outputPorts = #[
        @Port(name="output", type=String)
    ]
)
class ToUpper {
...
}

How the string messages arriving over the port input are processed is implemented by the method process annotated by the port name at the end:

class ToUpper {
    override process$input(String msg) {
       ...
    }

Having added the according function unit annotation to a Xtend class, the implementation of the right named method for processing the input arriving over the input port is also requested by compiler and supported by quick fix proposals in IDEs like Eclipse.

Message Forwarding Operator <=

The computed messages may be forwarded to the declared output port using the message forwarding operator <=:

output <= msg.toUpperCase;

The message forwarding operator also accepts closures computing the message to be forwarded:

output <= [
    if (msg.startsWith("_"))
        msg.toFirstUpper
    else
        msg.toUpperCase
]

Integration Unit Annotation @Integration

While operation units are used for implementing the pieces of logic a system is composed of, the only reason of integration units following the IODA architecture approach is the integration of other function units, operation or integration units. Integration units are used to compose systems and parts of systems. Nevertheless they are treated as function units as well, having input and output ports. Therefore input ports and output ports are declared in the same way.

@Integration @Unit(
    inputPorts = #[
        @Port(name="input", type=String)
    ],
    outputPorts = #[
        @Port(name="lower", type=String),
        @Port(name="upper", type=String)
    ]
)
class Normalize {
  ...
}

Separating operation units - for implementing functionality - from integration units for integration purposes is one of the main goals of the IODA architecture and within Flow Design. It allows to structure software systems in several levels of abstraction having implementation details only on the lowest level of abstraction. On upper level of abstraction the implementation consist of integration functionality only, represented by integration units. This is the second main principle of Flow Design and follows closely the Integration Operation Segregation Principle (IOSP).

Port Wiring Operator ->

The function units are either instantiated by the board or passed via constructor injection. Any [Dependency Injection](TODO add DP wiki link) approach could be applied here.

As integrating is the only reason for an integration unit, it normally has only a constructor method, nothing else. This constructor connects output to input ports of the integrated function units, as well as the integration unit's own input ports to the integrated function unit's input ports, and the integrated function unit's output ports to the integration unit's own output ports. For this the wiring operator -> is used. Here comes an example:

class Normalize {
    
    val toLower = new ToLower
    val toUpper = new ToUpper
    
    new() { 
        toLower.output -> lower
        toUpper.output -> upper
        input -> toLower
        input -> toUpper
    }

}

The wiring follows the data flow implemented by the system to be designed. The Xtend class above integrates the two instances of function units ToLower and ToUpper as shown in the following figure. In fact, it does not matter whether the integrated function units are integration or operation function units!

Integrating Function Board

The wiring operator works for full qualified port names, like

fu1.output -> fu2.input

as well as for function units with only one input or output port, like

fu1 -> fu2

as well as for mixed combinations, like

fu1.output -> fu2
fu1 -> fu2.output

Unfortunately, no error is shown by the Xtend compiler if the wiring operator is applied to a function unit with more than one input port or output port without using full qualified port names as then the operator is treated as Xtend's pairing operator.

Implicit Input Ports

Under circumstances a flow needs to be started by an event without any data delivering; e.g. on an initial start of a program. In this case the definition of the input port may be omitted. The framework will create an implicit input port of type de.grammarcraft.xtend.flow.data.None.

Let's assume a function unit with no input port declared. It reads user input from the console - a typical use case for starting a flow without explicit input data delivery:

@Operation @Unit(
    outputPorts = #[
        @Port(name="readNumber", type=String)
    ]
)
class ReadNumberToConvert {..}

The flow trough this function unit may be started by forwarding the predefined constant value None to it:

class Program 
{
    def static void main(String[] args) 
    {
        ReadNumberToConvert entry_point
        ...
        entry_point <= None
    }
}

For this example the following method for processing the data less input event have to be overridden:

class ReadNumberToConvert {

    override process$start(None msg) {
        output <= this.inputProvider.read_number_to_convert()
    }
    
}

Error Handling

Integration Error Handling

An integration error is given if a declared output port is not connected to any input port or any closure processing message from it. This is a meta-model error and is treated by the library by the special port intergationError of type java.lang.Exception predefined for each function unit and function board. By default this port is always connected to a closure printing a fatal error message on the standard console.

However, this connection may be overridden by the library user, explicitly forwarding the exception to an user defined closure or input port.

fu.integrationError -> [ log.fatal("integration error happened: {0}", exception.message) ] 

as this may become a hassle if for all instantiated function units the same closure has to be connected, there is an helper expression implemented where a bunch of function units may be connected at once to one closure:

      onIntegrationErrorAt(#[reverse, collector]) [ 
          exception | log.fatal("integration error happened: {0}", exception.message)
      ]

Here, the variables reverse and collector are referencing function unit instances.

Model Error Handling

The more interesting part for system designers is, how to handle errors which are inherent part of the system to be modeled. In fact, this is quite easy: All errors implicated by the system's model should be design as ordinary ports. They are still messages flowing through the system and must be handled explicitly by system's design.

Example

A simple example applying this notation may be found as sub project at this repository on GitHub.

Runtime Cost Considerations

In fact, wiring up ports results in method call implementations which are not very expensive and are almost well optimized by the Hotspot JVM.

E.g., the wiring of the function units from the example above into a chain, throwing an exception at the end, like

reverse -> toUpper
toUpper -> toLower
toLower -> [msg | throw new RuntimeException(msg)]
  
try {
    reverse <= "some input"
}
catch (RuntimeException re) {
    re.printStackTrace
}

will result in an exception with the following stack trace (shorten to show the important calls):

java.lang.RuntimeException: tupni emos
    at de.grammarcraft.xtend.annotatedflow.CallStackExample$1.apply(CallStackExample.java:20)
    ...
    at de.grammarcraft.xtend.flow.OutputPort.forward(OutputPort.java:52)
    ...
    at de.grammarcraft.xtend.annotatedflow.ToLower.process$input(ToLower.java:27)
    ...
    at de.grammarcraft.xtend.flow.OutputPort.forward(OutputPort.java:52)
    ...
    at de.grammarcraft.xtend.annotatedflow.ToUpper.process$input(ToUpper.java:27)
    ...
    at de.grammarcraft.xtend.flow.OutputPort.forward(OutputPort.java:52)
    ...
    at de.grammarcraft.xtend.annotatedflow.Reverse.process$input(Reverse.java:45)
    ...
    at de.grammarcraft.xtend.annotatedflow.CallStackExample.main(CallStackExample.java:25)

Concurrency Considerations

The message forwarding mechanisms of this Flow Design implementation are not thread safe at all. So, if messages may be forwarded to an input port of a function unit instance in concurrent situations, means, the particular process method of that input port may be called concurrently from different threads, the implementation of this method must handle such concurrent calls properly by a thread safe internal implementation according to the Java synchronization rules.

Detecting concurrent situations in the system to be designed is an inherent mission of the system designer. Flow design does not help him in that challenge. However, support may be added in later versions of the library, when the actors concept (like in Scala or Erlang) is integrated letting function units becomes actors for concurrent designs. An example for combining the Flow Design approach with the Actors Design may be found in Ralf Westphal's article "Actors in a IODA Architecture by Example"

Flow Cycle Considerations

The designer has strictly to avoid specifying direct or indirect cycles in the message flow! Cycles cannot be detected by the compiler or the annotation processor as the library and its notation has been realized as an internal DSL utilizing Xtend's active annotations. Active annotation processors are working in the context of classes not in the global context of all compilation units. For detecting cycles, the analysis of the global context would be needed. This may only be done by an external DSL.

Who understands German may read my blog article about that issue.

Build Integration

The library is provided for arbitrary build systems via Bintray and via Maven Central thanks to Stefan Öhme's article.

E.g., the integration via Gradle is as follows:

dependencies {
  compile 'org.eclipse.xtend:org.eclipse.xtend.lib:2.8.+'
  compile 'de.grammarcraft:de.grammarcraft.xtend.flow:0.2.+'
}

How It Gets Released

The building is done applying Gradle plugins for Xtend compiling and for releasing to Bintray and Maven Central.

The build is running on Travis CI.

Example Implementations

Example implentations applying this library may be found on GitHub in the repository XtendFlow-Examples.