Observability of workflow lifetime

[kpko]

Hi,

first of all, thank you for your relentless work on the library and keeping up with issues and pull requests. 👍

I've already commented on #162, but this is really a separate issue, so for the sake of order I would like to describe the other feature request I mentioned in the comment.

I'm currently working on a project that would benefit from a workflow engine like WorkflowCore. Our users should have the possibility to design workflows using a graphical designer and watch the progress while executing them similar to Microsofts Flow user interface, either executing a long-running workflow over time or a shorter workflow that runs synchronously in a single HTTP call.

To integrate WorkflowCore we would need a method to subscribe to workflow lifetime events like WorkflowStarted, WorkflowCompleted, StepStarted, StepCompleted, StepFailed to show the workflow progress in our UI.

In #162 @Kahbazi provided a pull request that added an event bus to the workflow host. Something like this would solve this issue but I don't know the internals of the library well enough to evaluate the implementation.

In #18 there was a similar request where you recommended the ExecutionPointer table. I would argue for a solution that is independent of the storage as we're using MongoDB in our project and I think high intensity polling wouldn't be that great.

What are your thoughts on this? If you want to pursue the implementation of an event bus to observe running workflow instances, I would love to help.

[danielgerlag]

See PR #215
Let me know your initial thoughts, still a work in progress

[kpko]

Hi Daniel,

this looks great. I can't wait to dig into the pull request throughout the next days. Is there any chance we can include events regarding workflow steps like StepStarted, StepCompleted and StepError?

As soon as this PR is merged I will look into a RabbitMQ-based EventBus to support multi-node clusters.

[danielgerlag]

There is an event for StepError, and adding StepStarted and StepCompleted is trivial... I was just hesitant because this will generate a lot of events... perhaps it should be configurable?
Also, I will need to make the calling of whatever implementation of Publish to be async, as this could have the effect on slowing everything down.

[kpko]

That being configurable sounds great.

Could we decouple the publishing of the events from producing them? I'm thinking of a queue that gets consumed in a separate thread, grabs events and publishes them.

[kpko]

Daniel resolved this with the release of 1.6.9 - I'm thrilled to start 2019 with this :) Thanks again!