Running actor in different modes

[hogaur]

Allow to only run in worker or server mode, as well as current default mode. Also moving actor management APIs from http routes. Advantages
Isolation between different goals of Kafka consumption and HTTP serving
Easier orchestration, as stream upgrade process can be significantly different from HTTP

[RajatVaryani]

@hogaur Can I pick this up?

[kartik7153]

@RajatVaryani Yeah sure, Go ahead. Assigning it to you.
Let's discuss the approach here.

[RajatVaryani]

Sure. What do you have in mind?

[kartik7153]

I was thinking to have three optional options passed while starting the app with Ziggurat such as server, worker(consumes from RabbitMQ), stream-worker. Then these options can be passed to ziggurat so that it figures out what states to start.

[RajatVaryani]

That will work. We will also require a config variable right? To control the mode in which the actor runs? We will use the value of that config in ziggurat's main and then pass on to start function?

[kartik7153]

Actually, I think config variable approach is better. Yeah, we can go ahead with that. wdyt @theanirudhvyas ?

[theanirudhvyas]

What I had in mind was to pass the mode in a command line argument. This way you don't need separate configs for different modes. And the mode in which ziggurat runs is decided by the arguments passed at startup.
I'm not sure if one approach is better than the other. I propose we do an analysis of both the approaches first (or any other approach that we can come up with). @RajatVaryani @kartik7153

[RajatVaryani]

@theanirudhvyas Do we want the users of the library to quickly change the mode the actor runs in? If that is the case then config would be easier. Wdyt?

[theanirudhvyas]

Even if you go the config way, you will still have to restart the service after a config change, how would it be easier?

[theanirudhvyas]

I had a conversation with @RajatVaryani where we discussed that going with the config approach will not require a change in the way ziggurat applications are deployed.
But it turns out that is not true. With different modes, we want different instances of our application to run in different modes simultaneously. This will require us to send different config-map to each instance and thus will require a change in the way we deploy.

[theanirudhvyas]

@RajatVaryani we should properly analyze as to how each approach will be implemented and what all changes will be required so that we don't make the same mistake in further discussions.
We can split the work if you want.

[RajatVaryani]

@theanirudhvyas Then it's clear to ahead with the command line argument approach right?

[kartik7153]

@RajatVaryani Yeah. Please go ahead with the command line approach.
Also please share what would init fn would look like

[theanirudhvyas]

@RajatVaryani yes, command line approach makes sense now. How are you going about implementing it?

[RajatVaryani]

@theanirudhvyas Let me analyze first and then get back to you. I have few questions as well.

1. Should we make the command line argument a mandatory parameter?
2. If not, what would be the default value of the argument?

[theanirudhvyas]

@RajatVaryani the argument does not have to be mandatory. By default, we can run all 3 procs.
In the command line arguments then we can pass the set of procs that we wish to run, if we don't want to run all of them.

The 3 procs are:

• http server
• kafka worker
• rabbitmq worker

Also could you analyse whether the command line options will except a set of procs or only 1 proc?
For e.g. (kafka-worker + rabbitmq worker). Or should we support only 1 proc at a time?

[RajatVaryani]

@theanirudhvyas As far as my understanding goes, rabbitmq worker is coupled with kafka streams to add retry, dead letter queue capability. Would it be a good design approach to give access to start different procs? (We might start the kafka worker with retry enabled and not pass rabbitmq worker when ziggurat starts). An alternative approach would be to give only http mode, worker mode and default mode. Wdyt?

[theanirudhvyas]

@RajatVaryani Kafka-streams would always start with rabbitmq. But that is not true the other way round, where you would only want to run rabbitmq consumers (and not kafka-streams), in the case where we are flushing rabbitmq queues (while rabbit migration for e.g.). HTTP-server can be its own proc.

We should also consider that

• should we give the functionality to run is streams mode with rabbitmq consumption (it will only push messages to rabbitmq). I can't think of a usecase where this will be needed though.
• should actors running only in consumer mode or stream mode have basic apis (like health-check)?

[theanirudhvyas]

@RajatVaryani could not continue with this, due to some personal responsibilities. So, I will take this up.

[kartik7153]

So there are three procs.
stream worker
retry worker
HTTP API

I was thinking that ping API would be present in HTTP API mode (no breaking change).
we can make new health check API for retry worker because of management APIs.

[kartik7153]

API changes
Currently, the main fn takes 4 arguments with actor routes as an optional argument.
Now, to make things backward compatible we have added one more arity in accepts map as an argument

(defn main [{:keys [start-fn stop-fn stream-routes actor-routes modes]}])