Event model

[tliron]

I went back to some earlier proposals and came up with something a bit new.

There are various assumptions made along the way here. My goal is just to put out something that encompasses all the features I think we need: 1) a unified event model, 2) pluggable event handlers, and 3) the ability to send events to specific targets in response to internal and external triggers. I had to make up a bunch of $ functions and magic words along the way, I hope they make sense in context.

Classical "workflows" are emergent in this scheme. I'm far more interested in the primitives here, the underlying event model. Perhaps some sugar can be added for those who are addicted to workflows, to automatically build upon these primitives.

Also note that this is not an attempt create a state machine per se, because I see state machines as a very small subset of what you'd want to achieve with events. However, this example does show some state management.

event_handler_types:

  SendEvent:
    properties:
      target: { type: entity } # TOSCA-path reference
      interface_type: { type: entity, required: false }
      name: { type: string }

  UpdateAttribute:
    properties:
      target: { type: entity }
      name: { type: string }
      value: { type: any }

  Execute:
    properties:
      command: { type: list, entry_schema: string }
      artifact: { type: string, required: false }
      inputs: { type: string, default: args, required: false } # "env" for environment vars, "json" for JSONified stdin, or "args" for CLI arguments
      results: { type: list, entry_schema: AttributeReference, required: false }

  CallJsonRpc:
    properties:
      url: { type: string }

data_types:

  AttributeReference:
    properties:
      target: { type: entity } # TOSCA-path reference
      name: { type: string }

interface_types:

  NodeLifecycle:

    attributes: # yes, attributes in interfaces

      state: { type: string }

    events: # replaces "operations" and "notifications"

      create:

        inputs: # for all handlers
        - source: tosca

        handlers: # for handling the event; be pessimistic: assume that handlers are called concurrently in arbitrary order
        - type: UpdateAttribute
          properties:
            target: $self # = this interface implementation, otherwise use TOSCA path
            name: state
            value: creating
        - type: Execute
          properties: # per handler type
            command:
            - /opt/log-event.py
            inputs: env
          inputs: # you can also add inputs per handler
          - topology: { $json: [ $topology ] } # the entire topology
          - node: $node_id
          outputs:
          - # TODO: map an output to an attribute somehow?
          wait: false # does not wait for it to finish execution and return status is ignored

        triggers: # for sending events (you can declare triggers in lots of places, what matters is the function calling context)
        - condition: $all_handlers_succeeded # for *this* event
          handlers: # these are "ad hoc" handlers just for this trigger condition, not hooked to a specific event
          - type: SendEvent
            properties:
              target: $self # where to send the event; self = *this* interface implementation (in this function calling context)
              name: created
        - condition: $any_handler_failed
          handlers:
          - type: SendEvent
            properties:
              target: $self
              name: failed

      created:

        handlers:
        - type: UpdateAttribute
          properties:
            name: state
            value: created

        triggers:
        - condition: $all_handlers_succeeded
          handle_send_events:
          - target: $self
            name: configure
          - target: $all_outgoing_relationships
            interface_type: RelationshipLifecycle # will be sent to *all* implementations of the interface type, including derived interface types
            name: source-created
          - target: $all_incoming_relationships
            interface_type: RelationshipLifecycle
            name: target-created
        - condition: $any_handler_failed
          handlers:
          - type: SendEvent
            properties:
              target: $self
              name: failed

      configure:

        triggers:
        - condition: $all_handlers_succeeded
          handlers:
          - type: SendEvent
            properties:
              target: $self
              name: configured
        - condition: $any_handler_failed
          handlers:
          - type: SendEvent
            properties:
              target: $self
              name: failed

      configured:

        triggers:
        - condition: $all_handlers_succeeded
          handlers:
          - type: SendEvent
            properties:
              target: $all_outgoing_relationships
              interface_type: RelationshipLifecycle
              name: source-configured
          - type: SendEvent
            properties:
              target: $all_incoming_relationships
              interface_type: RelationshipLifecycle
              name: target-configured
        - condition: $any_handler_failed
          handlers:
          - type: SendEvent
            properties:
              target: $self
              name: failed

      failed:

        handlers:
        - type: UpdateAttribute
          properties:
            name: state
            value: failed

  RelationshipLifecycle:
    events:
      source-created: {}
      target-created: {}
      source-configured: {}
      target-configured: {}

node_types:

  WebApp:
    interfaces:
      product-lifecycle:
        type: NodeLifecycle
        events:
          create:
            handlers: # will be added to handlers in interface declaration
            - type: Execute
              properties:
                command:
                - /opt/create-web-app.py
                results:
                - target: { $get_entity: [ $self, CONTAINER ] }
                  name: id
      infra-lifecycle:
        type: NodeLifecycle

events: # global calling context

  create-all-node: # these names are unique for this service template
    handlers:
    - type: SendEvent
      properties:
        target: $all_nodes
        interface_type: NodeLifecycle
        name: create

[lauwers]

Very good discussion. It looks like we're all working towards the common goal of having system-wide behavior emerge from elementary behaviors associated with smaller building blocks (i.e. nodes and relationships). This is fundamentally different from workflow-based orchestration systems. We should make a big deal about this in the introductory section of the specification where we highlight the unique capabilities of TOSCA.

Here is a collection of initial comments and questions that I'll try to better organize over time. Most of my comments related to the fact that many of the concepts presented by Tal are somewhat similar to concepts already in 1.3. For example:

• EventHandlers feel somewhat similar to artifact processors in 1.3, although it looks like event handles don't need artifacts.
• The handlers defined in interfaces feel similar to the primary and dependent implementation artifacts defined in 1.3.
• The triggers concept in Tal's proposal feels similar to the on_success and on_failure concept in my proposal
• My proposal associates preconditions with operations (events in Tal's proposal). I believe there is value in defining the state the interface must be in before an operation (or event) can be called.
• v1.3 defines triggers in the context of (imperative) policies. Can Tal's triggers be used to harmonize interface definitions and policies?
• v1.3 defines actions (previously also called activities) that could be taken by workflows and policies. How do these actions relate to handlers?
• Tal's example hardcodes one unique path through the state machine associated with an interface. How do we accommodate scenarios where different paths may need to be taken? I continue to think that we need to differentiate between:
  • the interface state machine that defines the allowed transitions
  • the desired transitions associated with specific triggers

Let's continue to iterate and nail down these concepts ASAP.

[lauwers]

Here is another attempt at comparing Tal's proposal above with my proposal for extending interface type definitions to accomplish many of the same goals. The following table lists Tal's proposal, my understanding of corresponding v1.3 features and/or extensions in my proposal, and my comments. Hopefully this will stimulate further discussion:

Tal's Proposal	Version 1.x Extensions	Discussion
events	operations and notifications	Events do away with the distinction between operations and notifications. Operations were intended to be API calls, whereas notifications were intended to be callbacks. Is there value in keeping this distinction?
handlers	operation implementations	v1.3 supports a list of operation implementations, although this list is (unnecessarily) split into a primary implementation and dependent implementations. Wheras implementations in v1.3 are limited to artifacts, Tal's proposal allows implementations that are not artifacts (e.g. setting an attribute value).
event handler type	artifact type	Event handler types are a generalization of artifact types. In v1.3, artifact types uniquely identify an artifact processors used to "execute" the artifact. How does an orchestrator know how to "execute" a handler based on handler type?
triggers		Tal's triggers specify the events that need to be sent after an event has been "handled". The use of the word "triggers" might also be confusing since v1.3 uses triggers differently in the context of policies
	on_success	Defines a set of actions that need to be taken after an operation has been executed. The set of actions is borrowed from workflow and policy syntax in v1.3. Tal's proposal specifies some of those actions as event handlers instead, and limits "triggers" to sending events (i.e. calling operations). In Chris' proposal, event handlers just execute scripts, whereas everything else (e.g. setting attributes) is done using actions specified in on_success.
	preconditions	Chris' proposal associates preconditions with operations (events in Tal's proposal). I believe there is value in defining the state the interface must be in before an operation (or event) can be called, since this (explicitly) defines an interface state machine.

[tliron]

So, I've been thinking on how policy triggers can be unified with this. Some interesting thoughts. Here's an example of how I think it should look from a user perspective:

policy_types:

  Redundancy:
    properties:
      minimum: { type: integer }
    triggers:
    - condition:
        $lesser_than:
        - { $get_attribute: [ TARGET, count ] }
        - { $get_property: [ SELF, minimum ] }
      events:
      - target: $target
        interface_type: NodeLifecycle
        name: create

topology_template:

  policies:

    - type: Redundancy
      properties:
        minimum: 3
      targets:
      - node1
      - node2

What's interesting here is that we want to set a condition not on one node but on a collection of nodes. So, somehow we need condition expressions that could operate on all the targets of the policy, hence the TARGET special name for get_attribute in this example.

Basically, the main grammatical thrust of this proposal is that if we want a rich way to express trigger conditions we need to really beef up the TOSCA expression language for addressing entities in various function call contexts in which condition expressions can appear.

[pmjordan]

Another way would be to have a single target which is the name of a group; and then acknowledge that groups have attributes, one such attribute is the total number of leaf entities contained within the group.

[tliron]

I think that's possible, but then ... you would still need a way to iterate those leaf entities.

I'm also generally in favor of beefing of groups and make them more node-like: attributes, interfaces (we used to have them but they were removed), etc. I do find the group to be a useful entity for, well, group data and functionality.

Still, I think it's useful to have policies work on the topology nodes generally, not just user-defined groups, which are static.

[lauwers]

The discussion above may provide us with an opportunity to fix an
aspect of TOSCA that has bothered me for a long time: TOSCA does not
provide a formal way for service designers to define the actions
that can be performed on an entire service. The Version 1.3 spec
touches on this topic by suggesting at least three different actions
that can be performed:

1. Deploying a service: this involves instantiating a service
   representation from a service template, fulfilling requirements if
   necessary, performing substitutions if necessary, and then running
   the deploy workflow (which ideally should be created
   automatically).
2. Removing a service: this involves running the undeploy workflow
   (which ideally should be created automatically). The Version 1.3
   spec doesn't state whether substituting services should be removed,
   or whether requirements fulfilled by the orchestrator should be
   undone.
3. Running a workflow: this involved running an imperative
   workflow defined in the service template.

Aside from the lack of formalism in defining these actions, it is
clear that this list iscompletely inadequate:

• In practice, operators may want to perform dry-runs before
  deploying services. This means that the orchestrator may want to
  separate out a provisioning phase from an activation phase and
  perform activation phases using noop operations. How are these
  two different phases invoked and/or executed?
• There is no discussion about updating or upgrading a running
  service.
• Version 1.3 includes a Scalable capability type but provides no
  mechanism to invoke scaling of components.
• There is no accommodation for administratively taking components out
  of service.

I believe we should add grammar to formally specify the set of actions
that can be performed on a service. Such grammar would then allow
service designers to define any type of lifecycle management
operations they deem appropriate for their services.

This grammar would also allow us to formalize other implicit
assumptions in the spec. For example:

• The Version 1.3 spec suggests that an operation to deploy a
  service returns a set of values as defined in the outputs section
  of a service template. This should be formally specified.
• It should be possible for orchestrators to asynchronously generate
  service-related events to external systems. This could support
  escalation scenarios where policies defined for a service first
  attempt to recover from failures automatically but escalate to an
  external system (or operator) if the recovery process is
  unsuccesful.
• How should partial failure scnenarios be handled? Retry? Rollback?
  Since there is likely no single approach that works best for all
  application domains, TOSCA should support defining the selected
  approach.

As we continue to think about unifying the event handling model for
interfaces, we should also explore whether the same unified model can
be used at the service-template level.

[lauwers]

After today's TOSCA Language Ad-Hoc meeting, I believe a picture is starting to emerge of how our various proposals might converge on a significantly improved TOSCA interface model. The following table expands on my previous attempt to reconcile what has been proposed so far with what's currently in Version 1.3. Hopefully this comparison will provide additional context to drive this discussion towards conclusion.

Version 1.x	Version 2.0	Discussion
	interface properties and attributes	Version 1.x interfaces define operations and notifications only. V2.0 adds support for properties and attributes in interfaces to allow for interface-specific state variables.
	INTERFACE	Interface properties or artifacts are identified using the INTERFACE keyword in ToscaPath expressions.
operations and notifications	events	Operations were intended to be API calls, whereas notifications were intended to be callbacks. Events do away with this distinction since it was never clearly specified how these were intended to be implemented. TODO: decide whether to adopt the events keyword, revert back to supporting operations only, or pick something else entirely such as actions
operation implementations	handlers	In Version 1.x, implementations of operations and notifications are specified using artifacts (that are expected to be processed by a corresponding artifact processor). Version 2.0 handlers allow implementations to be defined using artifacts, and also using other mechanisms (e.g. by setting attribute values)
primary and dependent implementations		v1.3 supports a single primary implementation artifact and an ordered list of dependent artifacts. Version 2.0 eliminates this distinction and defines a single list of handlers.
artifact type	event handler type	Event handler types are a generalization of artifact types. In v1.3, artifact types uniquely identify the artifact processor used to execute the artifact. How does an orchestrator know how to execute a handler based on handler type?
on_success and on_failure	triggers	Version 2.0 triggers generalize the on_success and on_failure keywords of the V1.x policy grammar. They define the events that must (conditionally) be sent after an event has been handled. Triggers can be specified in interface types or in interface definitions.
actions	handlers and triggers	V1.x workflows and policies can define actions that need to be taken after an operation has been executed. Version 2.x splits separates actions into handlers that are part of operation implementations (such as setting attribute values) and triggers that define how events can be propagated to other nodes in the topology.
preconditions	trigger conditions	Version 1.x preconditions are evaluated in the context of executing workflow steps. It is unclear what should happen if the precondition evaluates to False. Version 2.0 associates conditions with triggers only. Conditions are evaluated after an event has been handled rather than before, and they specify whether execution of an event should result in other events being triggered.
	service-level events and triggers	Version 2.0 allows events and triggers to be defined at the service (template) level. This addresses a gap in the v1.x specification where no mechanism exists to specify the types of actions that can be taken on a service as a whole.

[pmjordan]

Some minor clarifications on the above table:

Under handlers, delete

Version 2.0 handlers allow implementations to be defined using mechanisms other than artifacts (e.g. by setting attribute values)

insert
Version 2.0 handlers allow implementations to be defined using artifacts, and also using other mechanisms (e.g. by setting attribute values

Tal's propsal above doesn't appear to use artifacts but instead has a command. I suggest that artifact type defintions should include the command string to execute them.

Be consistent with naming - either they are 'handlers' or they are 'event handlers'

delete

and triggers that define how actions can be propagated to other nodes in the topology.

Insert
and triggers that define how events can be propagated to other nodes in the topology

[lauwers]

Thanks Paul. I made minor edits based on your feedback.

[lauwers]

Can this new proposed syntax be used as a replacement for Version 1.3 imperative workflows?

[tliron]

Update -- I edited the original post with a small modification. Instead of the events keyname in triggers, which was used to send events based on the condition, I put a handlers keyname and specified new SendEvent handlers. Because that's what it really was, just ad hoc handlers that send events.

Note that this is slightly different in meaning from the handlers keyname at the interface event level. Those are registered to handle that specific named event, but here we mean "ad hoc" handlers that are directly tied to the condition. The implication here is that you don't actually need interface types at all anywhere in your TOSCA, you can handle everything with triggers + handlers. Interface types of course are more powerful because they are reusable and extensible building blocks.

[lauwers]

Based on the discussion in the TOSCA Language Ad-Hoc meeting of Aug. 23, 2023, it appears that we need to make a distinction between low-level and high-level concepts here:

1. The low-level interface descriptions describe the event mechanisms that describe the interface state machine (i.e., the various interface states, the events that cause transitions between these events, and the conditions that must be satisfied before events can be handled or before events can be propagated across the topology graph.
2. The "high-level" statements that dictate how these low-level mechanisms are used to accomplish intended behavior (i.e., intent statements).

Tal's example above seems to overload these two concepts. For example, it defines a create event that transitions the interface into the created state, but then it also hardcodes logic to further transition the interface from the created into the configured state, when that intent was never expressed by the consumer of the interface. If the consumer of the interface were to invoke the configure event to express the intent to transition the interface to the configured state, then there is no logic to handle a create event first. I believe this shows that we can't use the same set of events to perform individual state transitions as to express desired state.

I very much like the concepts introduced by Tal, but I believe we must extend them as follows:

• the unified event handling grammar should be used to express low-level mechanisms only
• it needs to be complemented with higher-level support for expressing intent

We might be able to express intent by introducing additional interface attributes (e.g., a desired_state attribute), but this raises the question whether just setting an attribute is sufficient to trigger state transitions. In Tal's proposal, it seems like all transitions are initiated by events, which makes for a very nice and clean model. Should interfaces distinguish between events that express changes in desired state vs. events that notify the interface of actual state transitions? If so, that starts to look like the distinction between operations and notifications again.

Thoughts?

[calincurescu]

What I would like is to be able as an option to connect also a graph traversal mechanism that triggers doing stuff. That would be for me something that would be "high level". For instance:

simple_vnflcm_startup:
    node_types: [tosca.nodes.nfv.VNF]
    follow_relationships: [tosca.relationships.Root]
    start_from: leafs
    direction: target-to-source
    trigger: 
        #e.g. Tal's definition of trigger

Then for instance, one could define an event-based triggering inside a node type, that is better understood already when defining the node type, with a representation-graph traversal that is better understood when working with the template. I would support both options, some problems are more suited to be fully-event based triggerd (e.g. event-assembled) while others would benefit to be more specified on the template level.

[pmbruun]

I do not like this.

In an ideal nirvana of everything being cloud-native, declaratively deployed applications, there is no need for sequencing at all. That should be the default for TOSCA, I agree with @tliron about that. In practice, however, things tend to have sequencing dependencies, and the objective is to find a nice way of controlling the order of actions that is more abstract than explicit BPEL workflows and is easy for template designers and template users to understand.

Yes, every workflow or declarative workflow can be reduced to some kind of event-passing and waiting mechanism in some implementations, but that means that event handling is a lower level construct than other mechanisms, such as arrows in an explicit workflow or declarative workflows derived from the topology.

Arguing that all dependencies should be specified using the event mechanism because everything can be done using events is like arguing that loops and recursion are redundant in programming languages because they can be mapped to GOTOs or jumps in lower level machine code.

Yes, events can be a useful language construct, just like some high-level programming languages have retained the GOTO construct. And by the way, in explicit workflows with boxes-and-arrows are exactly a GOTO mechanism.

TOSCA introduces a lot of complexity with a node having potentially multiple interfaces with different state-models. This means that it is hard to express a "direction" in simple terms like "deploy the topology" or "undeploy the topology", and for day 2 scenarios this gets even harder.

Still, at some level of abstraction, it should be possible to specify the "desired state" of a topology in some simple terms, if nothing else then at least because that is in our official TOSCA charter: "The TOSCA TC also aims to make the TOSCA language easy to use ..."

In HPE SD we chose a different strategy: We standardized on a singe state model for all nodes; it happens to be the MTOSI TMF 518_SA_2 standard because we are primarily working with Telco services. Instead of interfaces, we defined that a node may have "child component nodes" and each of these have their own state following the same standard. In TOSCA, this would amount to a node-substitution with a "component"-type relationship between a main node in the substitution and a variable number of other nodes. It meets the same objective, that the abstract node ends up with a multiple states, but it simplifies the specification because all nodes have a well-defined direction from undeployed to deployed and back.

Assuming, that somehow, we solve the problem of undeploy-deploy desired state in TOSCA, then relationship types should be able to define dependency rules. If an orchestrator chooses to implement that using an event-mechanism, a workflow engine or something else should not be mandated by TOSCA.

I remember that @tliron had the objection deployment sequencing on relationship types, that this can be very hard to visualize and debug in case the dependencies are not as expected. I need to point out here, that an event-based mechanism is even harder to debug because both the template designers and the template users will have no good overview of the logic.

Of course HPE SD also has events and triggers. So this does not mean that events should not be supported by the language. My point is only that, as with GOTOs, they should only be used in rare, exceptional cases.

We analyzed the strength of declarative workflows based on the topology of relationship types, and with a low set of types, we can model everything that a standard workflow system can do - and a lot more. To have full functional coverage, the designer needs to be able to express:

• Linear flow: First A, then B
• Concurrent flows: First A, then B and C concurrently (fork), possibly all followed by D (join)
• Conditional flow: First A, then if condition c, then B, followed by C
• Choice of flows: First A, then if c then B else C, followed by D (extensible to multiple choices)
• Multiple concurrent flows: First A, then concurrently B1, B2, …, Bn, followed by D
• Multiple sequential flows (loop): First A, then B1, B2, …, Bn in sequence, followed by D
• Sub-flows: Any A and B above may be a single task or a workflow.
• Backtracking: If c, then re-execute some previous step in the workflow.

Only the last one - backtracking - requires an imperative mechanism, possibly using an event-mechanism.

Importantly, in a declarative context, backtracking is not the same as a loop in a classical workflow. Rather it amounts to a reevaluation of the execution strategy under modified conditions/assumptions.

[pmbruun]

Oh, I forgot.
A typical case is that undeployment has the opposite dependencies of deployment, and one of the benefits of declarative workflows is that you shouldn't have to write one workflow for deployment and another one for undeployment.

With an event based model, if setup requires A to be deployed before B, then B would specify it needs to wait for A, and A needs to publish an event when B should proceed. This requires some specification work in both node (or nodetype) A and B.

Now for the reverse during undeployment, A needs to wait and B needs to publish an event. This requires further work in A and B and it needs to be kept consistent with the work done for the deployment case.

This is a hugely complex and error-prone task for the template designer, and it defies the purpose of having to do everything differently for deployment and undeployment.

... and then come the day 2 scenarios.

All of the above ought to be derivable from relationship types, specified once and for all per relationship type. But I don't think such behaviors are presently definable in-language. Perhaps this is one place, where we would need to refer out to capabilities of the orchestrator?

[lauwers]

I don't believe events as proposal by Tal are a low-level construct at all, and they are definitely not similar to GOTOs. In fact, they are a generic construct for passing control between entities, similar to how objects in an object-oriented system interact. Events make the creation of reusable components possible, where the logic for managing the lifecycle of a component can largely be defined locally within that component based on knowledge (defined in the node type) about possible incoming and outgoing relationships of that component. The emergent behavior that results from combining reusable components (and their localized event handling) into overall service topology graphs is a generalization of declarative workflows, in my opinion, that is much superior to any of the alternatives.

[tliron]

Really great feedback. I wrote some notes to myself and will try to propose something that can tie this all together to make Peter's vision work (I think Calin is thinking in the same direction). I agree with the requirement, we do need a way to assemble these event-handlers into a comprehensive flow of some sort, and traversing the node-relationship graph is part of it. The key to my thinking, as I mentioned in the meeting, is interface types (on nodes and relationships), assembling them together into a "graph" which is different from (but related to) the node-relationship graph. More to come when I have time to sit on this!

[pmbruun]

@lauwers I was not being precise - low/high level could be defined in several dimensions that are not entirely orthogonal.

• Amount of code/boiler-plate you need to write to implement a given functionality - large amount=low leve, small amount=high level
• Abstraction - the ability to encapsulate low-level functionality and hide implementation details
• Locality of functionality - one functionality should be managed in one place, it should not dissipate into changes in many places
• Expressiveness - Levels: low=How; medium=what; why=high. The concept of "intent based" is, for me, the ability to express "why" things should be done instead of just how or what. From the why, the concrete cases of how and what should be derivable.

Alone these dimensions are insufficient. Examples:

• A macro-processor is usually considered low-level because it does not respect concepts of scope and structure of the underlying language - but for an assembly language, a macro-assembler makes it higher level.
• Modula2 had abstraction and encapsulation but was utterly verbose and did not have the concept of objects - multiple state-instances from the same class definition
• Locality - The GOSUB of Basic was a terribly example of non-locality. Since there were no call stack, only global parameters, in order to re-use a subroutine, the caller needed to transfer parameters into global variables and back out after the call. So any change of parameters required a lot of boiler-plate code both inside the subroutine and in all places where it was called.
• Expressiveness - a very expressive language can mean that a small amount of typing can produce huge and complex functionality, but that can also mean that an very small typo can create huge, complex and wrong functionality. Some rule-based systems are highly expressive, but fail on locality (or denotationality) - adding one rule to a rule-base could alter the meaning of other rules already in the system.

Events are clearly very expressive, but they are very much expressing "how" and not "why". For me, intent-based, is something that we should strive for with TOSCA, and I think events fail in that dimension. Also locality is a problem, as I tried to explain - expressing the intent that node B can only exist if node A exists, implies setting up events and waits in both A and B, because events are uni-directional.

[lauwers]

Hi Peter, I agree that events are very much a mechanism, but what I like is the fact that this event mechanism can be used at a high level as well as a low level, and for defining global behavior as well as for local behavior. In fact, the proposed unified event handling functionality may actually enable your requirement for more of an intent paradigm in TOSCA:

• In TOSCA v1.3, there was no mechanism for external management systems to signal intent to the TOSCA orchestrator. TOSCA assumed that a service would be deployed ("orchestrated") based on the service template, but beyond that there was no TOSCA grammar to specify which other lifecycle management operations were supported on the service (e.g., modify/update, upgrade, move, scale, etc.). In Tal's proposal, events are supported at the service level and provide the mechanism for external systems to signal intended behavior changes to the Orchestrator.
• In theory, the Orchestrator should then be able to translate high-level intent statements into operations on individual nodes and relationships (i.e., translate the WHY/WHAT into the HOW). However, I'm building an extremely dumb orchestrator that has no built-in knowledge beyond the semantics of the TOSCA language. Specifically:
  • It has no built-in knowledge of the state machines associated with interfaces defined in TOSCA. Specifically, it does not know how operations in an interface must be sequenced to achieve a desired ("intended") outcome. New grammar is required to allow profile designers to define state machines associated with their interface types.
  • It has no built-in knowledge of how to interleave operations on one node with operations on relationships in and out of that node, or with operations of other nodes with which that node has relationships. New grammar is required to allow component designers to specify these local behaviors based on intended outcomes.

My expectation is that the "unified event handling" mechanism can be used to:

• define the state machines associated with interface types
• define the required state transitions to achieve a desired interface state
• specify local behaviors (in node type definitions) for interleaving node operations with relationship operations
• allow end-to-end service behaviors to emerge by combining local behaviors defined in node types based on dependencies in the topology graph
• use events defined at the service template level to allow external systems to signal intent
• use the event/trigger mechanism to translate global service-level intent into local component-level operations by triggering lower-level events.

I'm trying to document this in a lot more detail. Let's continue to iterate off-line and discuss during our next meeting.

[tliron]

Hi all, I want to try to streamline and clarify the original proposal (which has since evolved) and attempt to solve the workflow challenge.

I recognize that I had a bit of a concept soup with the addition of "events" vs. "triggers". Let's simplify it and remove "triggers" from the proposal. And also let's go back to the idea that interface types contain "operations" (nice continuity with TOSCA 1.3!), but we will remove "notifications" because the feature is handled differently in this proposal (nice continuity with TOSCA 1.2!).

What is proposed is four new entities:

1) Events

These are generic entry points into the event model.

All events can have an optional condition expression associated with them. The event is triggered only when the condition evaluates to "true". Leaving the condition out means that it will always be "true". Each event can have zero or more handlers via the handle keyname (see below).

There are three types:

1. Global events: These are continuously polled and triggered when the condition expression evaluates to "true". They are declared under the global events keyname (a list). They can be included in service templates and profiles, thus importing a profile can install global events required for it to function correctly.

2. Global external events: These are named and exposed by the service template and are triggered by an external orchestrator or user command. The optional condition expression can be specified in order to additionally limit when the event will trigger. They are declared the same as above with the addition of a external keyname, which specifies an externally exposed name. The presence of this keyname thus disables continuous polling for the event. The external events can be listed for any service template, just as with inputs and outputs. For imported profiles these external names will be prefixed with the namespace, as with imported types.

3. Handler events: These are not continuously polled, rather the condition expression is evaluated only after handlers are called. They are declared under the succeeded and failed keynames, with the same grammar as the handle keyname. All handlers under handle must succeed to activate succeeded and any one handler must fail in order to activate failed (though note that all handlers are still guaranteed to be called), so that one and only one of these keynames is ever activated. You can also attach succeeded and failed to individual handlers, and so the grammar can be nested ad infinitum.

Putting these together:

events:
- condition: { $equals: [ { $get_attribute: [ { $node: [ my-node ] }, state ] }, "Deployed" ] }
  handle:
  - type: Call
    properties: { targets: { $node: [ my-node ] }, interface_type: MyInterface, operation: my-operation }
    succeeded: # for just this handler
    - type: Log
      properties: { message: "my-operation succeeded!" }
  succeeded: # for all `handle` handlers
  - type: Log
    properties: { message: "all handlers succeeded!" }
  failed: # for all `handle` handlers
  - type: Log
    properties: { message: "a handler failed!" }

2) Operations

These are named, typed (they have an inputs signature), and graph-located entry points into the event model. They are declared by name on interface types and registered individually on all the node and relationship representations. This is no different from operations in TOSCA 1.X.

Unlike events, which are triggered based on conditions, operations can only be triggered when explicitly called by a Call handler (see below), thus there is no continuous polling involved. Implementations may very well implement events and operation calls under the same message-calling mechanism (e.g. a message bus or broker).

Like an event, you can add a handle keyname to an operation. A handler here can be Call, useful for composing operations together into a workflow. It can call an operation on the current node and interface type, on other interface types, or propagate to incoming and outgoing relationships and related nodes. Also as with events a condition keyname is available to optionally filter triggering when called, i.e. to disable the operation under certain conditions.

Refinement of handle, succeeded, and failed lists (from interface type to node/relationship type to node template) is additive: additional handlers are appended to the list. This is so that inherited behavior cannot be changed.

3) Handlers

Zero or more of these can be attached to any event or operation via the handle, succeeded, and failed keynames.

The handlers are called when the event or operation is activated. Whether they are called sequentially (and in what order) or in parallel is up to the implementation. Handlers do not return a value per se, but rather can "succeed" or "fail". The orchestrator is in charge of collecting that success state and activating succeeded or failed handlers accordingly.

4) Built-in handler types

• Call: This calls an operation, by interface type and name, on selected nodes or relationships (one or many).
• Workflow: This is a bit like multiple Call handlers but on steroids (see below).
• SetAttribute: This sets an attribute on zero or more selected nodes, capabilities, or relationships. (The notifications feature of TOSCA 1.3.)
• Execute: Executes a command, which can be an artifact. (The implementation + dependencies feature of TOSCA 1.X.)
• Log: Implementation-specific structured logging: log level (defaults to INFO), message, optional log namespace, optional additional properties.

I do recognize that there is a problem here in that the these types are meant to be a "standard" part of TOSCA 2.0, and when we removed the Simple Profile from TOSCA we specifically did not want to include any built-in types into TOSCA. However, I will point out that all the primitive data types (string, integer, scalars, etc.) are built-in. Likewise, we have built-in functions and also user-declared functions. So perhaps these can be an additional exception to our rule? Open to discussion.

The Workflow handler type

Let's say we have a DependsOn relationship type and we want to trigger a deploy operation on the Lifecycle interface of our nodes, starting at the outmost dependencies in the DependsOn graph. Let's also assume that that our Lifecycle interface has various other operations for the phases (create, created, configure, configured) and finally ends in an a deployed operation call. Let's finally assume that the DependsOn relationship has a DependencyLifecycle interface type, and that it should have "instantiate" called only after both source and target have been deployed succesfully. That seems like a common enough workflow. Achieving this, however, is impossible with just events and Call handlers. We need an imperative (programmatic) way to traverse the graph, make calls, and wait on results. In other words: a workflow.

Before delving into this I'm going to insist that every node and relationship type should "know" how to handle its own lifecycle operations itself and these should not be expressed in our workflow definition. This allows workflows to be generic and not have to support any known node types. They only have to know about the interface types and relationship types in order to traverse a graph. So, once we call the deploy operation we can string together the "phases" as in my initial proposal, finally called deployed at the end.

Also I see no way around the need for a potentially controversial feature: variables. For any imperative construct we need to support variables, even in functional languages. I will be prefixing variables with @ and using uppercase as a naming convention.

The keynames introduced are as follows:

• with: a closure where a variable is set to a selection, e.g. with @N = $nodes
• call: this is identical to the Call handler type (as above)
• when: the reverse of call, as it waits for an operation to be called

Note that both call and when can work on selections, so we can call an operation on a bunch of nodes or relationships or wait on all operations in that selection to be called.

Also note that variables can work like the path-traversal selection functions I introduced in #154. So: { @N: [ TARGET ] } works like { $self: [ TARGET ] }.

Below is the full workflow described above. Note that I am using pseudo-YAML here, because I started to get swamped in curly brackets. If this is clear enough we can then work it into real YAML:

interface_types:

  Lifecycle:
    operations:
      deploy:
        handle:
        - type: Workflow
          properties:
            do:
            - with @NODE = $self # closure just to preserve $self as a variable
              - with @RELS = { $outgoing_relationships: [ { $is_a: [ DependsOn ] } ] } # *all* relationships
                - with @DEPS = { @RELS: [ TARGET ] } # *all* dependencies

                  # Deploy all our dependencies (recursively!)
                  - call @DEPS, Lifecycle, deploy

                  # Wait for all dependencies have been deployed
                  - when @DEPS, Lifecycle, deployed

                    # Now finally create ourselves (starting the internal lifecycle of the interface)
                    - call @NODE, Lifecycle, create

                    # Wait for the last phase of our lifecycle
                    - when @NODE, Lifecycle, configured

                      # Now instantiate the relationships
                      - call @RELS, DependencyLifecycle, instantiate

                      # Wait for all relationships to be instantiated before we can finally say we're done
                      - when @RELS, DependencyLifecycle, instantiated
                        - call @NODE, Lifecycle, deployed

      create:
        succeeded:
        - type: Call
          properties:
            operation: created

      created:
        succeeded:
        - type: Call
          properties:
            operation: configure

      configure:
        succeeded:
        - type: Call
          properties:
            operation: configured

      configured:
       succeeded:
        - type: Call
          properties:
            operation: deployed

      deployed: {}

  DependencyLifecycle:
    operations:
      instantiate:
        succeeded:
        - type: Call
          properties:
            operation: deployed

      instantiated: {}

Note that there is no proper failure handling in this short example. What if a node fails to be deployed? Do we need to unravel all the deployments? (Most workflow engines I know just get stuck, anyway. Workflows are terrible.) Also, the workflow hangs if a certain node or relationship never calls its deployed operation. This could potentially be circumvented with timeouts, but it's rather unavoidable when you depend on an external system, whether synchronously or asynchronously. (Did I mention that workflows are terrible?) Of course even if the whole workflow succeeds, congratulation: you achieved Day 1. What if a dependency fails? Should the node that depends on them do something? What should it do? WORKFLOWS ARE TERRIBLE.

This workflow is also non-idempotent as is, but I think it's not hard to fix that by saving deployment state as an attribute. Thus a deployed handler in the interface type could SetAttribute state to "deployed", and we can check on that with $get_attribute before calling deploy again.

[pmbruun]

I have no problem with triggers and events, and the ideas are fine. What I am saying is:

• The TOSCA specification should not mandate that declarative workflows must be implemented using an event mechanism
• I am with @calincurescu and @tliron that it should be possible to declare generic information from which an actual flow can be derived in Relationships instead of in nodes or interfaces. There are complications
  • The state of a node representation is really the node state plus the combined state of its interfaces, where each interface potentially has a custom state model
  • Relationships are between nodes, there is no way to specify relationships between interfaces of different nodes or relationships between interfaces of the same node.
    • Do we need a new InterfaceRelationship type?
    • Do we need standardized or at least named state models, so that generic interface relationships can make assumptions about the state models?

Why do I mention relationships between interfaces of the same node? I am not convinced the interface state models are always completely orthogonal? I would expect that they are often linked - that interface A cannot be in state X unless interface B is in states Y or Z...

[pmbruun]

For those who don't remember the TMF MTOSI state machine by heart, this is what it looks like:

It is a mess, but the point is that it can be cleaned up to a total ordering of states and that makes it useful. If these are the states TOSCA wants is not the question.

[pmjordan]

I think the scenario @tliron intorduces under the heading of Workflow handling is one which we could usefully use to examine proposed syntax options. If you agree then perhaps it could be represented as a UML diagram.
I am generally in favour of re-using the efforts of other SDOs where we can, but as I recall from work in TMF, the MTOSI model refered to by @pmbruun has not been used in recent times because people want to define thier own state models (for good or ill). I don't think TOSCA should impose any particular state model on template designers.

[pmbruun]

@pmjordan That is fine. This decision was made long ago.
I am just pointing out that the fact that each interface might have its very own state model is going to be a problem for declarative workflows because it prevents defining the logic in relationship types.

[tliron]

@lauwers thinks that re-introducing workflows takes us a step back. :) I would argue that as always workflows are an optional feature, profiles do not have to use them. I know I never would. :)