New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrent, Isolated commanded apps #117

Open
bamorim opened this Issue Dec 10, 2017 · 5 comments

Comments

Projects
None yet
4 participants
@bamorim
Copy link
Contributor

bamorim commented Dec 10, 2017

This is a very hard thing to implement, but lets discuss it
Also, I wrote this while rushing a little bit, I'll refine it later

Problem

This is somehow related to #113, but this is a little bit more.
I think that this approach will solve #113 and some other usecases.

One of the biggest problems of commanded right now (to me at least) is the "global" nature of it. Right now we configure an app using configs and that means that we can have only one instance per app (including umbrella apps).

This also makes testing very hard since we cannot run tests concurrently and we have to always reset the event store to ensure tests doesn't interfere with one another.

Another thing, as pointed by #113 is that we may want to have different EventStores per app.

Right now, the only way of having "different applications" is to use multiple routers, however, they all use the same eventstore, then the applications are not isolated (although it may be a good idea to have them connected through the same eventstore so we can have shared subscriptions to allow event-collaboration workflows, otherwise people would need some kind of integration middleware to propagate events)

Goals

Summing up, my goals are (in no specific order):

  • Allow isolated instances to be started at test (create and destroy applications on the go)
  • Allow multiple instances to run for different apps in the same umbrella/cluster
  • Be backwards compatible (as much as it is possible) with our current way of doing things
  • Start the applications via configuration by default (something like Ecto does with the repos, by specifying an otp_app)
  • Allow dispatching to a specific instance and directly using module (using a registration as it is right now)

Simplified overview

Just a fast overview so we know where we need to make changes

@slashdotdash please correct me where I'm wrong, which is probably in a lot of places since this is my fast overview over the code

A commanded application can be said to be composed by (and some other pieces):

  • Router (runs on caller process)
  • Dispatcher (runs on caller process)
  • Pipeline/Middleware Layer (runs on caller process)
  • Aggregate Supervisor
  • Aggregate processes
  • Event Store
  • Event Handlers
  • Registration
  • Subscriptions

The processes involved are generaly

  1. A router maps given commands to aggregates and then asks the dispatcher to dispatch it (via dispatcher module)
  2. The dispatcher
    1. Executes the pipeline
    2. Opens the aggregate (through the aggregate supervisor, which starts a new process and rehydrates with events from the eventstore or just returns the pid)
    3. Starts a supervised task that calls the aggregate genserver (via name: Aggregate Name + Aggregate uuid)
  3. The aggregate process
    1. Given the execution context, runs the handle function in the handler module or the execute on the aggregate module
    2. Tries to persist the events (calling EventStore directly)
  4. EventStore work (let's not get into this here)
    1. In background and async, the event handlers will catch up with event_store
  5. Dispatcher executes the pipeline according to the result
    1. There is one middleware that if consistency: :strong is passed, it waits on all event handlers using Subscriptions.wait_for

Proposal

Right now, the entrypoint for the application is the router.
I suggest including a new concept: An Application.

defmodule MyApplication
  use Commanded,
    router: MyRouter,
    event_store: MyEventStore
end

There would be three modus operandis:

Complete isolation

my_app = MyApp.start_link
MyApp.dispatch(cmd, application: my_app)

Per Application

MyApp.dispatch(cmd)

As it is

MyRouter.dispatch(cmd)

The way it works is as follows:
We'll introduce a new option for dispatch, which it is the pid of the application

MyRouter.dispatch(cmd, application: pid)
If the pid is not found, we'll lookup for a global application (which right now is started by :commanded otp app)

MyApp.dispatch is the same thing as the router, however, it looks up for the global application scoping by application module name (MyApp, MyOtherApp, etc) but also accepts the myapp pid.
It will need to lookup the router as well (which was given via the use Commanded opts)

Challanges

There a lot of places in the codebase where we use global registries. We may need to allow scoping these registries using Module names and pids. This includes:

  • Aggregates.Supervisor.open_aggregate
  • EventStore.*
  • Subscriptions.*

There is also a require a change on how the commanded application is started.

Inspiration

We may want to have a look at how Ecto does that

@slashdotdash

This comment has been minimized.

Copy link
Member

slashdotdash commented Dec 10, 2017

Thanks for the detailed write up @bamorim.

@drozzy

This comment has been minimized.

Copy link

drozzy commented Feb 16, 2018

I think this is additional complexity - so it should satisfy a need for majority.
FYI I don't run event-store on the same node as my app. I have multiple nodes (distributed) sharing the same event-store.

@bamorim

This comment has been minimized.

Copy link
Contributor Author

bamorim commented Mar 6, 2018

@drozzy how the nodes find the event store in your case? (Trying to understand how the registry should behave)

@drozzy

This comment has been minimized.

Copy link

drozzy commented Mar 6, 2018

@bamorim Just want to clarify that I run http://eventstore.org - which provides tcp or http endpoints. The language I use has an adapter for it. I use docker-compose for configuration. Manually hooking things up was a nightmare.

Something you said didn't sit right with me - different EventStores per app. I'm not sure I understand how this would work in the case of, for example, absolute event ordering requirements. If no such requirement exists - sounds like you just have different systems altogether, that should not even be part of the same codebase/repo. Sorry, maybe I misunderstood something.

@andreapavoni

This comment has been minimized.

Copy link

andreapavoni commented Nov 11, 2018

+1. I would like to be able to setup an umbrella made of separate apps, connected to a single event_store instance, but I still can't figure it if it's possible right now. The proposal here looks interesting.

Is there any roadmap for this? I'd be glad to help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment