Is it possible to create long running workflow with Ocelot?

[VenkateshSrini]

Hi,

Is it possible to create long running workflow with Ocelot something like Azure Durable Function

Thanks and regards
Venkatesh

[raman-m]

Impossible, sure!
Will you contribute to implement such feature(s)?

[VenkateshSrini]

This is a good question. Apart from being an API gateway, if we would facilitate orchestration then it will be nice. I am missing an equivalent of WCFWorflowFunctions. These WCFWF functions extension of WCF durable function. These functions are used for storing the state of transaction based on id. However the primary payload processing will be delegate to a business component. Similarly, if Ocelot can offer some kind of framework with simple abstraction a commit and a rollback. Then with help if some back-end support it should be able to commit or role back transactions by calling a business component service. With all the goodness of API Gateway we if we implement this also it will be very poweful

[VenkateshSrini]

Most of my pet projects especially on generative AI I use Azure durable functions. In most of the solutioning projects when workflow in Azure or .NET is required this becomes a sort of Defacto choice for hardwired workflow. Ocelot have been of use in few internal projects. We are trying to position it in few RFP where a good API gateway solution with monetization and Resilience is required. We will be using in future projects

[raman-m]

Venkatesh,
Do you want to clone Azure Durable Functions feature or just integrate Ocelot to the Azure service with a new package?

Similarly, if Ocelot can offer some kind of framework with simple abstraction a commit and a rollback. Then with help if some back-end support it should be able to commit or role back transactions by calling a business component service.

I don't understand why Ocelot should manage workflow states? Keep in mind that if you will save some states (caching objects, workflow function calculations, writing/reading to/from persistent storage etc) by Ocelot instance then Ocelot will consume more CPU & OM resources which impact to performance of the gateway.

You need to build a separate client app which will make Http requests through Ocelot instance. Any Microservices architecture software can use any gateway including Ocelot.

Show us and upload to GitHub your POC projects with this Ocelot + workflow design research please!
We will get better understanding what you need, and what you cannot do with Ocelot now.

[ggnaegi]

@raman-m probably a stupid idea, but we could extend the request aggregation concept?

[raman-m]

@ggnaegi We have tons of requests with extension of Aggregation feature!

• We can enhance parallel "workflow" making them really parallel
• We can extend parallel "workflow" to support more HTTP verbs in addition to current GET only. We have some issues in backlog
• We can implement very simple serial workflow for Aggregation. Still makes sense.

But that doesn't mean we will integrate to some external workflow processors because our Release Process states:

The main Ocelot package must not have taken on any non MS dependencies.

But the integration to Azure workflow service can be implemented by contributors as a package. I'm fine to add a package to Ocelot repos and setup CI-CD. But we should not make effort to develop that pack.

[VenkateshSrini]

raman-m wrote on Jan 10

There could be client SDK that will abstract this.

POC is not yet fully shaped. Now I'm trying something with ELSA + Ocelot.

[raman-m]

@VenkateshSrini Hi!
But you still haven't answered, why should Ocelot process workflow steps? Why Ocelot should encapsulate some workflows at all?
Any concrete user scenario with microservices?

[VenkateshSrini]

If we are envisaging that we do not want to step up Ocelot to be not going to be more than an API gateway then we may not need this. May be this is a wanted feature that goes hand in hand in with various orchestrations. So it is better to be in on product is my point

[raman-m]

So, as a team, we will wait for your ready POC, and we will continue our discussion.

[raman-m]

Also, as a team, we could brainstorm a new Serial "workflows" in Aggregation feature, because Aggregation is semi-parallel now, and Aggregation is closely related to workflows, but not exactly the same...
This will be our custom implementation, Mr. Srini. We are not going to integrate to Azure Durable Functions as a workflow processor. We just cannot have references to non-Microsoft libs, see Development Process please.
But we are totally OK to create separate Ocelot NuGet package as Ocelot module which will have any references with integration to Azure Durable Functions service.
We have only 2 sr. developers in the team and we are overloaded having no chance to help you with your Ocelot+ADF researches. Sorry about that!
So, waiting for your POC to review...

[ggnaegi]

@VenkateshSrini You have to keep in mind, that we are 3 software engineers maintaining this project at the minute. We are doing it on the side, we don't get any money for this. We are open for new ideas, but any help is very welcome, since we need to prioritize. As for now, the overall application performance, some features about quality of service and service discovery have the priority.