Running test-workflow from test-harness

[blueelephants]

I could compile conductor with gradle and also could start "main_workflow" with the Swagger UI.

I also could change the state of task_1 to "in progress" and "complete" but have problems to start dyn-task_2 ... not sure what API calls I need to start task_2 to feed parameter "taskToExecute".

Is there a way to execute the tasks of "main_workflow" task-after-task and see the API call interactions to learn more about its internal workings? There are some test classes within test-harness like "End2EndTests.java", but I couldn't figure out whether these classes will do the job and I couldn't even figure out how to start those test classes.

I appreciate any help.

[v1r3n]

Hi,

Here is the snippet for the second dynamic task in the main workflow:

{
      "name": "dyntask",
      "taskReferenceName": "task_2",
      "inputParameters": {
        "taskToExecute": "workflow.input.task2Name"
      },
      "type": "DYNAMIC",
      "dynamicTaskNameParam": "taskToExecute"
}

The taskToExecute comes as workflow input - so when you start the workflow pass the following input if you want the task_2 to be say "task_3":

{
   "task2Name": "task_3"
}

Conductor will use the input as the name of the task and schedule task_3.

[blueelephants]

Thanks for your help - it now works.

I have some other follow-up questions where I would like to ask you for some more help & information.

What is the typical "API call interaction" when running a workflow?

What I have figured out so far is this

1. Start a new workflow using /workflow/{name}.
   Then conductor puts the first task (task_1) into the task queue with status "SCHEDULED"
2. Worker 1 microservice instances (responsible for task_1) periodically poll the task queue using /tasks/poll/task_1 to get new work assigned
   This polling changes the status of task_1 to "IN PROGRESS"
3. Once worker 1 microservice instances have finished the work, they update the status of task_1 using /tasks to e.g. "COMPLETED"
   After that, conductor puts the next task(s) into the task queue with status "SCHEDULED" which can be polled by worker microservices again.

=> Is this right so far..? Can you possibly share with me a state machine diagram?

Some other questions came up during my tests:

• Polling some tasks:
  • what is the meaning of workerid in this polling API call?
  • I noticed that after some time (1 to 2 minutes), it is possible to re-poll a task of the task queue (and the poll count increases).
    Which parameter is controlling this behaviour? It seems to be a "time-out" simulation to re-assign work to new worker microservices?
• What is the purpose of /tasks/{taskId}/ack?
  What happens to a taskid when running this API call?
• Within the UI in the tab "Task Details" there is a field "Callback After" - what is the purpose of this field and how is it related to tasks?

Many thanks for your help.

[v1r3n]

The state machine you described is correct. I will publish the state machine diagram on the documentation soon.

To answer your questions in the order:

1. Worker Id is an informational identifier, we use it with hostname to identify the host on which the worker was running. That helps in troubleshooting issues when worker fails a task.
2. Once the task is polled, it needs to be acknowledged by the worker - this is to ensure worker has not lost the polled the task due to network issue etc. If the task is not ack'ed - it will be queued again in 60 seconds. (Configurable but currently fixed)
3. When the worker updates the status of the task as IN_PROGRESS, it can optionally provide a time delay after which the task should be made available. This is useful for long running tasks where the worker should be periodically checking the status of the task completion (by another process for example) at a fixed interval (say 1 hour). Callback after value can be set by worker as part of the Task's status update.

[blueelephants]

Thanks a lot. I am looking forward to taking a look at the state machine.