Switch branches/tags
Nothing to show
Find file Copy path
83ebef5 Oct 15, 2018
1 contributor

Users who have contributed to this file

122 lines (90 sloc) 4.88 KB

TaskLauncher Data Flow Sink

This application launches a registered task application using the Data Flow Server REST API.


Launch request args including:

  • the task name (required and registered as a task with the target Data Flow Server)

  • deployment properties (key value pairs, optional).

  • program arguments for the task (a list, optional).


  • Content-Type: application/json


A JSON document:

  "deploymentProps": {"key1":"val1","key2":"val2"},
  "args":["--debug", "--foo", "bar"]




N/A (launches task to the local system).


The tasklauncher-dataflow sink supports the following configuration properties:

The login password. (String, default: <none>)

The login username. (String, default: <none>)

Enable Data Flow DSL access. (Boolean, default: false)

The Data Flow server URI. (String, default: http://localhost:9393)

Skip Ssl validation. (Boolean, default: true)


The initial delay in milliseconds. (Integer, default: 1000)


The maximum polling period in milliseconds. Will be set to period if period > maxPeriod. (Integer, default: 30000)


The polling period in milliseconds. (Integer, default: 1000)

Using the TaskLauncher

A tasklauncher is a sink that consumes LaunchRequest messages, as described above, and launches a task using the configured Spring Cloud Data Flow server (given by --dataflow.uri). The task launcher periodically polls its input for launch requests but will pause when the SCDF server’s concurrent task execution limit given by .dataflow.task .maximum-concurrent-tasks is reached (see the reference docs for more details).

When the number of running tasks drops below this limit message polling resumes. This is intended to prevent the SCDF deployer’s deployment platform from running out of resources under heavy task load. The poller is scheduled using a DynamicPeriodicTrigger. By default the polling rate is 1 second, but may be configured to any duration. When paused, or if there are no launch requests, the trigger period will increase, applying exponential backoff, up to a configured maximum (30 seconds by default).

When the poller is paused it puts pressure on the message broker so some tuning will be necessary in extreme cases to balance resource utilization.


$ ./mvnw clean install -PgenerateApps
$ cd apps

You can find the corresponding binder based projects here. You can then cd into one one of the folders and build it:

$ ./mvnw clean package


Register a task app and create a task, the timestamp sample provides a simple demonstration.

dataflow:>app register --name timestamp --type task --uri ...
dataflow:>stream create http | task-launcher-dataflow-sink --deploy

Send a launch request,

$curl http://localhost:<port> -H"Content-Type:application/json" -d '{"name":"timestamp"}'
dataflow:>task execution list
║Task Name│ID│         Start Time         │          End Time          │Exit Code║
║timestamp│1 │Fri Aug 10 08:48:05 EDT 2018│Fri Aug 10 08:48:05 EDT 2018│0        ║