Flogo Stream Action
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.github/ISSUE_TEMPLATE Update issue templates Sep 10, 2018
examples update imports Sep 4, 2018
pipeline update imports Sep 4, 2018
.gitignore update channel code Aug 23, 2018
LICENSE Update LICENSE Aug 1, 2018
README.md Update README.md Sep 5, 2018
action.go update imports Sep 4, 2018
action.json update imports Sep 4, 2018
action_test.go update imports Sep 4, 2018

README.md

Streams is a pipeline based, stream processing action for the Project Flogo Ecosystem

Flogo Stream

Edge devices have the potential for producing millions or even billions of events at rapid intervals, often times the events on their own are meaningless, hence the need to provide basic streaming operations against the slew of events.

A native streaming action as part of the Project Flogo Ecosystem accomplishes the following primary objectives:

  • Enables apps to implement basic streaming constructs in a simple pipeline fashion
  • Provides non-persistent state for streaming operations
    • Streams are persisted in memory until the end of the pipeline
  • Serves as a pre-process pipeline for raw data to perform basic mathematical and logical operations. Ideal for feeding ML models

Some of the key highlights include:

😀 Simple pipeline construct enables a clean, easy way of dealing with streams of data
Stream aggregation across streams using time or event tumbling & sliding windows
🙌 Join streams from multiple event sources
🌪 Filter out the noise with stream filtering capabilities

Getting Started

We’ve made building powerful streaming pipelines as easy as possible. Develop your pipelines using:

  • A simple, clean JSON-based DSL
  • Golang API

See the sample below of an aggregation pipeline (for brevity, the triggers and metadata of the resource has been omitted). Also don’t forget to check out the examples in the project-flogo/stream repo.

  "stages": [
    {
      "ref": "github.com/TIBCOSoftware/flogo-contrib/activity/aggregate",
      "settings": {
        "function": "sum",
        "windowType": "timeTumbling",
        "windowSize": "5000"
      },
      "input": {
        "value": "=$.input"
      }
    },
    {
      "ref": "github.com/TIBCOSoftware/flogo-contrib/activity/log",
      "input": {
        "message": "=$.result"
      }
    }
  ]