Does this framework work in distributed multi-node cluster of an orchestration service?

[c-ankur]

Hi,

Does this framework support multi-node cluster for an orchestration micro-service?

• Does the framework ensure that the caseId is unique across a cluster, or is this the responsibility of the application itself to ensure this on a single/multi-node cluster?
• Can the caseId be a UUID or it has to be an integer?
• There is no clear explanation provided for some key interfaces, for example, the FlowretDAO. What are the methods in this DAO supposed to do and what's the meaning of the parameters passed?
  • What is the job of the incrCounter(String s) method which is taking a string as input, for example?
  • What are the write and read methods supposed to do exactly? Is the "key" parameter supposed to be a caseId or processId?
  • Similarly, for the other key interfaces provided, there are no clear java docs inline, or elsewhere.

Suggestion:
I think there could be a lot more adoption of this library if better/clear documentation with examples is provided. I did check the JUnits like you have mentioned in docs/tickets and could understand a few things a little better, but I wouldn't say those are a replacement for good API documentation. (readme file needs more documentation or need examples. Inline java docs would be great)

[deepakarora3]

Hi,

Thanks for taking out the time to look at Flowret and raise your questions. Below are the responses inline.

Does this framework support multi-node cluster for an orchestration micro-service?
Flowret is an embedded orchestrator which means that it runs independently on a pod. For example, an application running on the pod can receive an event (REST API or messaging etc.) and use Flowret to run the orchestration. The orchestration in this case will run completely on the pod where the request was received.

Does the framework ensure that the caseId is unique across a cluster, or is this the responsibility of the application itself to ensure this on a single/multi-node cluster?
Flowret leaves it to the application to create a case id.

Can the caseId be a UUID or it has to be an integer?
The case id has to be a string and can be a uuid.

There is no clear explanation provided for some key interfaces, for example, the FlowretDAO. What are the methods in this DAO supposed to do and what's the meaning of the parameters passed?
The documentation talks about this. You can look up the section "Get runtime service of Flowret". This DAO is an interface and the implementation needs to be provided by the application. It is used to read and write the Flowret specific information.

What is the job of the incrCounter(String s) method which is taking a string as input, for example?
This is a method that increments a counter identified by s. The implementation again is left to the application.

What are the write and read methods supposed to do exactly? Is the "key" parameter supposed to be a caseId or processId?
The read and write methods are used to read and write Flowret specific information to a database. The Flowret specific information is in the form of a Document (refer unify-jdocs for more info). The application can store the info in any data store of their choice and so the implementation of the DAO is data store specific. It is like Flowret saying to the application that hey here is a document that I need to store, please store the same in the data store of your choice.

Similarly, for the other key interfaces provided, there are no clear java docs inline, or elsewhere.
Suggestion:
I think there could be a lot more adoption of this library if better/clear documentation with examples is provided. I did check the JUnits like you have mentioned in docs/tickets and could understand a few things a little better, but I wouldn't say those are a replacement for good API documentation. (readme file needs more documentation or need examples. Inline java docs would be great)
Thanks - will try and improve the documentation to make it more clear.

[c-ankur]

Thanks, but I still have the following doubts:

• what’s the meaning of incrCounter method? What is its purpose? How does it help this library? I could only understand that it is incrementing a string which contains a number. But what’s the use of this to an application, I did not understand. Is this something related to the caseId or its uniqueness? Or just a way to help segregate the process logs/audit logs based on the counter (thinking of caseId here)?
• could you also share some details about the ProcessComponentFactory and what does the implementation of this interface have to do? Just create an object of either InvokableStep or InvokableRoute? Any examples here?
• ON_PERSIST event: When is this event triggered specifically? Does the application have to trigger it? Or this automatically triggered?

Appreciate the inputs!

[deepakarora3]

Responses inline:

what’s the meaning of incrCounter method? What is its purpose? How does it help this library? I could only understand that it is incrementing a string which contains a number. But what’s the use of this to an application, I did not understand. Is this something related to the caseId or its uniqueness? Or just a way to help segregate the process logs/audit logs based on the counter (thinking of caseId here)?
While processing an application, Flowret writes audit logs. The audit logs are many for one case id. Hence, Flowret expects an implementation from the application to return the next sequence number of a named counter. For example if I have a case id as c1, then the application could maintain a counter entity by the name of "counter-c1". When Flowret calls incrCounter, the existing value of the counter will be returned and the counter incremented. Reason for such an implementation is that Flowret is a lightweight implementation which does not have any data store of its own and expects the application to provide for means to store data. Usually most application in a cloud scenario have these things already in place.

could you also share some details about the ProcessComponentFactory and what does the implementation of this interface have to do? Just create an object of either InvokableStep or InvokableRoute? Any examples here?
This is a factory interface which when called by Flowret (using the interface), creates an object which implements invokableStep or invokableRoute inteface and gives this object to Flowret. This is the object that is actually called by Flowret to do the work of a particular step or route. There is basic documentation for this already present and you can also find examples of this in the test folder.

ON_PERSIST event: When is this event triggered specifically? Does the application have to trigger it? Or this automatically triggered?
Refer flow block documentation to know what persist is. It is a special step that tells Flowret to tell the application to persist data via the ON_PERSIST event. It is up to the application to know how to store the data. Example is that which executing steps in orchestration, an application could be collecting data. The application could save collected data when application pends. In certain cases, it may be required to save the data before the application pends. In such cases a persist step could be inserted and the application will be notified via the ON_PERSIST event. Note that there is nothing that stops the application from storing the data after each step is executed.

Hope this clarifies. Thanks.

[c-ankur]

Thank you so much!

• Could you elaborate a bit on the state machine of this engine may be? Do you mean a persist step will trigger an ON_PERSIST event which an EventHandler can process, and then the application can decide to store data in the database as per need? Couldn't we do this through a normal "step" as well? What's the advantage of this type of step being added?
• ON_PROCESS_START: why does this event not set the user_data as well? I assume user_data is application-specific data that needs to be processed and starting a process/case would involve needing such data.

[deepakarora3]

Could you elaborate a bit on the state machine of this engine may be?
The state machine is custom implemented based on the journey definition.

Do you mean a persist step will trigger an ON_PERSIST event which an EventHandler can process, and then the application can decide to store data in the database as per need?
Yes

Couldn't we do this through a normal "step" as well? What's the advantage of this type of step being added?
Yes you could have done it via a normal step. We just thought that it may be a good feature to have out of box.

ON_PROCESS_START: why does this event not set the user_data as well? I assume user_data is application-specific data that needs to be processed and starting a process/case would involve needing such data.
Does it not? I will check this. User data should be returned. If not, I will include in the next version in a few days. Thanks for letting me know.

If it is OK I will now close the issue. If you have more questions like this, request you to use the discussions tab rather than the issues tab. Thanks for your understanding.

[deepakarora3]

I checked the code. The reason why user data is not passed when the case is started is because user data is specific to a step being executed and is not at the case level.