New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQLStreamStore vs NEventStore vs EventStore #108

Closed
MNF opened this Issue Nov 7, 2017 · 4 comments

Comments

Projects
None yet
3 participants
@MNF

MNF commented Nov 7, 2017

Your project is relatively new and exist in the same area as more mature Event Store and NEventStore (and a few others).

Can you explain the reasons, why this project was created? What are the benefits ( and disadvantages) to use your library? Why I (as new in Event Store area) should choose your library?

It will be good to add the answer to wiki.
P.s. @damianh, I found your post http://dhickey.ie/2015/04/stepping-down-from-neventstore/, but it also doesn’t explain, why “NEventStore's current design doesnt work” for you.

@damianh

This comment has been minimized.

Member

damianh commented Nov 30, 2017

Good Q. Was meaning to blog about this, but here will do.

  1. NEventStore and SQLStreamStore are libraries. Event Store is a full blow database server + CEP projection engine. Entirely different beasts alltogether and not really comparable. The question comes down to performance and operations requirements. Event Store is far faster than NES and SSS and will forever be so. If you are in environment where you can deploy and operate Event Store, then go for it. My personal situation is such that I need to deploy on-premise with customers whose enterprise IT is a bit more "traditional". That is "RDBMS only".

    ES is much more suited to handling streams with high volume of data, especially such that is sourced from systems. I will never recommend SSS or NES for such scenarios (in same way I wouldn't recommend a relational db). SSS/NES is really for human orientated collaborative applications.

    Having said that, I've highly aligned SSS's API and it's internal behavior with Event Store's such that a migration to ES later, if desirable, isn't a major rewrite. There are API differences and SSS will never have all the functionality of ES.

  2. Yes I used to maintain NEventStore and successful solutions were developed with it by many people. About the time I was considering version 5 I realized that I wanted to make such significant and breaking changes that it was easier to start over than upset the community. What sort of changes...

  3. The addition of ambient transaction support to NES (v3 iirc) was a big mistake imho. It was added so one could transactionally read an NServiceBus message and append to a stream (if I recall correctly). However I've seen it (ab)used to span writes to multiple streams AND transactionally write to projections traversing the dispatch code. And even reading from a projection in between too. It facilitated bad design where the transaction boundary of a stream / aggregate was easily breached. I wanted to remove it but that would break a lot of people. SSS does not, and never will, support ambient transaction scopes.

  4. The dispatcher concept is broken. Pull based is far more reliable and simpler. Especially for replays and projections that move at different speeds. "Dispatching" can still be achieved by pulling and pushing to a queue or whatnot. NES does have a polling client. SSS uses a similar mechanism for its subscriptions.

  5. I didn't like NES commit concept. It made seeking for specific stream version event really complex. I wanted 1:1::Row:Message(event). It is my understanding that the commit concept only came to be as it wasn't possible to insert multiple documents into Mongo in a transaction. Thus the commit message contained a bundle of events. I am also of the opinion that if one needs to correlate multiple events that were related to a single operation / command, a correlation / causation id on the metadata is a better approach.

  6. The support-every-db under the sun was an ambitious goal. The reality was quite a bit different - I simply didn't want to support NoSQL stores (ravendb and mongo) any further. If you are able to deploy and operate a nosql db, use ES. Hence SQLStreamStore.

  7. I also wanted to be able to walk stream backwards, delete messages and number of other things. Can't do that with NES's API and would be difficult to implement with NES's underlying schemas.

  8. NES's object approach, via generics, encourages treating data (messages / events) as types & classes. This makes things complex for event versioning. SSS API is string based forcing the developer to think about messages in a primitive way. I deliberately chose not to use byte array to make it easier to splunk the data using standard RDMBS tooling.

  9. NES's up-conversion was an idea that sort of works if used sparingly and conversion is 1:1. However I feel that should be outside the library and closer to either a domain concern or an operations (stream rewriting) concern.

  10. NES's support for all sorts of serialization, compression and encryption while interesting academically, was in practice is unnecessary and added complexity. Operationally, spelunking the streams and commits using standard DB tooling and getting back binary was a sub-optimal experience. Compression didn't add anything and if you need to encrypt, you'll probably should be looking leveraging your underling infra to handle that. In the end, most people used JSON. SSS's payload is a string but is named as JsonData. Nothing else will be supported (there is an eye on leveraging json support in the various relational databases at some point in the future).

That's it really. Hope it helps. Closing but follow up with any further questions if needed.

edit: fixed some typos

@damianh damianh closed this Nov 30, 2017

@bruth

This comment has been minimized.

bruth commented Jan 3, 2018

I randomly stumbled across this and appreciate the pragmatism.. and I am asking without ever using SSS.

It is my understanding that the commit concept only came to be as it wasn't possible to insert multiple documents into Mongo in a transaction. Thus the commit message contained a bundle of events.

It seems they had to formally add the notion of a commit to make up for the lack of multi-doc atomicity in Mongo. I presume SSS supports atomic multi-event writes? If so, is the issue that in NES a commit is public API vs a transparent wrapper to make up for the lack of atomicity in Mongo?

SSS API is string based forcing the developer to think about messages in a primitive way.

So SSS puts the emphasis on simple data structures rather than types? Could you elaborate on this a bit more?

@damianh

This comment has been minimized.

Member

damianh commented Mar 26, 2018

@bruth apologies for late reply (I get many messages and this fell through the cracks)

I presume SSS supports atomic multi-event writes?

Correct.

If so, is the issue that in NES a commit is public API vs a transparent wrapper to make up for the lack of atomicity in Mongo?

I didn't design NES but yes, that is an issue for me. It also makes walking streams (forwards, backwards, from arbitrary version / position) very complicated and sub-optimal perf wise.

So SSS puts the emphasis on simple data structures rather than types? Could you elaborate on this a bit more?

Point 10 addresses this I believe. Is there something more to expand on?

@bruth

This comment has been minimized.

bruth commented Mar 26, 2018

Thanks @damianh. That does clarify things. At this point I no longer remember the context for my last question, but point 10 is very clear!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment