How conductor server behaves if its down while workflow execution

[rskTech]

Team,

I have recently started exploring the conductor framework.

I have two questions.
Q1. Lets say we started the workflow having 3 tasks, 1 task is done and while task 2 being executed on worker, suddenly conductor server is down, We keep all task details in persistent layer but we will loose the context, so what will happen once conductor server is back online? will that workflow will be triggered again or what?

Q2: What happens if conductor server come back before the task task 2 completion, will that workflow continue execution as is ?

Any help will be appreciated.

Thanks,
Rajendra

[v1r3n]

Hi @rskTech

Conductor server manages the state outside in the persistence layer. This means in the above example, the workflow will resume where it left off. However, the task 2 worker when it tries to update the server will receive the failure if the server is down. There are two choices here:

1. Don't do anything, in that case the server will see the task as in progress and if there is a timeout set on the task, will reschedule the task. On the worker side, you will see that the retryCount for the task is 2 and you can implement the logic to skip running the actual work and just update it to COMPLETED status.
2. When the worker tries to update the task it will receive the failure, persist the update somewhere in the worker system (database, in memory etc) until the server is available again and re-try the update. Please note, worker already has logic to do the retries, so this will ONLY happen if the server is down for an extended period of time.

In summary, Conductor server takes care of failures and provides right framework choices. The delivery guarantees for Conductor here is at-least-once