Consider global deterministic ordering of commits across streams

[damianh]

Originally discussed in #159

[erothFEI]

Just by adding more precision of the CommitStamp in the db significantly improves the odds of not having commits with the same stamp however this also restricted by the resolution of the DateTime.UtcNow property of .Net. This is only updated anywhere from 10-15 milliseconds. You can get around this issue by utilizing the Stopwatch class in .Net, this will give you practically down to the Tick level of precision. In my fork that can be seen here: erothFEI@aab703f, I have added a unit test that demonstrates what I am thinking. This is obviously not a real unit test it was just a quick way for me to demonstrate what I am thinking.

By doing it like that it virtually makes it not possible to have a duplicate CommitStamp on the same machine and would make it very rarely happen on different machines. And honestly if it were to happen, then really the two commits did happen at the same exact time and does it matter which one was first if you are dealing with different streams. You would still probably want to sort by commitsequence after sorting by commitstamp to guarantee the correct order within a stream.

Introducing some kind of global sequence across machines seems like way to much of a hassle and makes it much more difficult to scale freely.

What do you guys think?

[damianh]

I don't think having commits with the same commit stamp is an actual problem - the library is designed with multiple writers (same or different machines) in mind, so it's perfectly viable to have commits with the same commit stamp. The first problem is that the queries were sorting by commit stamp and resulting incorrect order within a stream. This has been fixed in PR #169, where we are now sorting by CommitSequence and thus guarantee ordering within a stream.

The second problem is that people are using CommitStamp to infer global ordering. This is bad. It looks like it works... until it doesn't. Also, the client is responsible for setting the CommitStamp value so there is no central authority ensuring that they are correct.

Really, CommitStamp should just be treated at this point as something informational and not guaranteeing anything.

Introducing some kind of global sequence across machines seems like way to much of a hassle and makes it much more difficult to scale freely.

I agree, but it seems others don't and I'd like to understand their use cases more.

[mynkow]

http://stackoverflow.com/questions/17483273/neventstore-issue-with-replaying-events

Currently when replaying from the store events can come in really messy order. For example events recorded today come before events recorded 1 month ago. This is huge. That means we have to design the read model without relations because everything is blowing right now. :( After all this is history, events (stream revisions) should be replayed in correct order somehow. We do not use shards or multiple writers and that is why I cannot help here much. We were forced to remove all hard relations (NHibernate read model) and replace them with simple Guids. The joins are done in code. If we are going to shard it will be ok, but we are not.

[gregoryyoung]

There is a geteventstore adapter coming for persistence. This is already
supported there and will come for free with the adapter.

On Monday, July 8, 2013, mynkow wrote:

http://stackoverflow.com/questions/17483273/neventstore-issue-with-replaying-events

Currently when replaying from the store events can come in really messy
order. For example events recorded today come before events recorded 1
month ago. This is huge. That means we have to design the read model
without relations because everything is blowing right now. :( After all
this is history, events (stream revisions) should be replayed in correct
order somehow. We do not use shards or multiple writers and that is why I
cannot help here much. We were forced to remove all hard relations
(NHibernate read model) and replace them with simple Guids. The joins are
done in code. If we are going to shard it will be ok, but we are not.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20597789
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

I've responded to the SO question. Feel free to continue discussion here
(or the DDD/CQRS group which has more members).

Yes, there is a GetEventStore adapter in the works, and while it will
probably expose 'node ordering'* amongst other features to you, it's very
unlikely to become a supported NEventStore feature. So if you do rely on
it, you will be tying your app to the persistence engine implementation.
This may or may not be of concern to you.

If you need to stay with SQL Server, and you must *have "global
ordering"*, your other choice is to fork NEventStore and extract/adapt it
to your needs. It is, after all, only 2 tables.

• Greg, if I've used the wrong term, please correct. You may want to
  mention what restrictions such ordering entails architecturally?
  ** Huge DDD smell

Regards,

Damian

On 8 July 2013 11:47, mynkow notifications@github.com wrote:

http://stackoverflow.com/questions/17483273/neventstore-issue-with-replaying-events

Currently when replaying from the store events can come in really messy
order. For example events recorded today come before events recorded 1
month ago. This is huge. That means we have to design the read model
without relations because everything is blowing right now. :( After all
this is history, events (stream revisions) should be replayed in correct
order somehow. We do not use shards or multiple writers and that is why I
cannot help here much. We were forced to remove all hard relations
(NHibernate read model) and replace them with simple Guids. The joins are
done in code. If we are going to shard it will be ok, but we are not.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20597789
.

[gregoryyoung]

None actually global ordering is quite simple in a single partition. In
multiple partitions you end up with a reordering delay that can add to
latency and some detection code for when it's not right.

Most can run in a single partition thus perfect ordering (let's say 10k tps)

On Monday, July 8, 2013, Damian Hickey wrote:

I've responded to the SO question. Feel free to continue discussion here
(or the DDD/CQRS group which has more members).

Yes, there is a GetEventStore adapter in the works, and while it will
probably expose 'node ordering'* amongst other features to you, it's very
unlikely to become a supported NEventStore feature. So if you do rely on
it, you will be tying your app to the persistence engine implementation.
This may or may not be of concern to you.

If you need to stay with SQL Server, and you must *have "global
ordering"*, your other choice is to fork NEventStore and extract/adapt it
to your needs. It is, after all, only 2 tables.

• Greg, if I've used the wrong term, please correct. You may want to
  mention what restrictions such ordering entails architecturally?
  ** Huge DDD smell

Regards,

Damian

On 8 July 2013 11:47, mynkow <notifications@github.com <javascript:_e({},
'cvml', 'notifications@github.com');>> wrote:

http://stackoverflow.com/questions/17483273/neventstore-issue-with-replaying-events

Currently when replaying from the store events can come in really messy
order. For example events recorded today come before events recorded 1
month ago. This is huge. That means we have to design the read model
without relations because everything is blowing right now. :( After all
this is history, events (stream revisions) should be replayed in correct
order somehow. We do not use shards or multiple writers and that is why
I
cannot help here much. We were forced to remove all hard relations
(NHibernate read model) and replace them with simple Guids. The joins
are
done in code. If we are going to shard it will be ok, but we are not.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20597789>

.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20605117
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

When using multiple partitions (aka shards), do you have a single sequence number somewhere that guarantees ordering across all the partitions?

[gregoryyoung]

Vector clocks.

On Monday, July 8, 2013, Damian Hickey wrote:

When using multiple partitions (aka shards), do you have a single sequence
number somewhere that guarantees ordering across all the partitions?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20606258
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[mynkow]

I do not get it. I cannot imagine a model without any relations. When replaying events I get some event for user before the event for creating that user. The events are persisted in 5 months difference. Events are thrown from different aggregates.

[damianh]

You should consider not doing DDD/CQRS/ES so. It's not a silver bullet. If
you do wish to continue down this route, go to the DDD/CQRS group to
further discuss and learn the concepts
https://groups.google.com/forum/#!forum/dddcqrs

On 8 July 2013 15:24, mynkow notifications@github.com wrote:

I do not get it. I cannot imagine a model without any relations. When
replaying events I get some event for user before the event for creating
that user. The events are persisted in 5 months difference. Events are
thrown from different aggregates.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20608729
.

[gregoryyoung]

Sorry but what?! I don't normally claim authority but here is a place I
think I have some.

What is wrong with expecting ordering in an event stream outside aggregate
boundaries? It's really really complex to write projections otherwise.

Greg

On Monday, July 8, 2013, Damian Hickey wrote:

You should consider not doing DDD/CQRS/ES so. It's not a silver bullet. If
you do wish to continue down this route, go to the DDD/CQRS group to
further discuss and learn the concepts
https://groups.google.com/forum/#!forum/dddcqrs

On 8 July 2013 15:24, mynkow <notifications@github.com <javascript:_e({},
'cvml', 'notifications@github.com');>> wrote:

I do not get it. I cannot imagine a model without any relations. When
replaying events I get some event for user before the event for creating
that user. The events are persisted in 5 months difference. Events are
thrown from different aggregates.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20608729>

.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20609575
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

Firstly, I suspect @mynkow has some domain modeling probs. I don't have
enough context, but here is not the place to discuss it. A more open
discussion, with context, at the DDD/CQRS group would be more appropriate.

What is wrong with expecting ordering in an event stream outside aggregate
boundaries?

Em... NES does guarantee ordering in an event stream. The typical NES
usage scenario, though, is 1:1::Stream:AR. If you have 1:N::Stream:AR, then
yeah, you have ordering for multiple AR's as they are in the same stream. I
don't know of anyone doing this with NES though.

[gregoryyoung]

Ok so streams per aggregate work very well in the domain.

However many (most) projections cross aggregate boundaries in which case
they need some form of ordering (does not need to be perfect but needs to
be reproducible). If not they become horribly complex.

Greg

On Monday, July 8, 2013, Damian Hickey wrote:

Firstly, I suspect @mynkow has some domain modeling probs. I don't have
enough context, but here is not the place to discuss it. A more open
discussion, with context, on DDD/CQRS would be more appropriate.

What is wrong with expecting ordering in an event stream outside
aggregate
boundaries?

Em... NES *does *guarantee ordering in an event stream. The typical NES
usage scenario, though, is 1:1::Stream:AR. If you have 1:N::Stream:AR,
then
yeah, you have ordering for multiple AR's as they are in the same stream.
I
don't know of anyone doing this with NES though.

On 8 July 2013 15:41, Greg Young <notifications@github.com<javascript:_e({}, 'cvml', 'notifications@github.com');>>
wrote:

Sorry but what?! I don't normally claim authority but here is a place I
think I have some.

What is wrong with expecting ordering in an event stream outside
aggregate
boundaries? It's really really complex to write projections otherwise.

Greg

On Monday, July 8, 2013, Damian Hickey wrote:

You should consider not doing DDD/CQRS/ES so. It's not a silver
bullet.
If
you do wish to continue down this route, go to the DDD/CQRS group to
further discuss and learn the concepts
https://groups.google.com/forum/#!forum/dddcqrs

On 8 July 2013 15:24, mynkow <notifications@github.com<javascript:_e({}, 'cvml', 'notifications@github.com');><javascript:_e({},

'cvml', 'notifications@github.com <javascript:_e({}, 'cvml',
'notifications@github.com');>');>> wrote:

I do not get it. I cannot imagine a model without any relations.
When
replaying events I get some event for user before the event for
creating
that user. The events are persisted in 5 months difference. Events
are
thrown from different aggregates.

—
Reply to this email directly or view it on GitHub<

https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20608729>

.

—
Reply to this email directly or view it on GitHub<

https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20609575>

.

Le doute n'est pas une condition agréable, mais la certitude est
absurde.

—

Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20609958>

.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20612458
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

Yep, and the aggregates may be in different domains and may not even be in the same store / partition / node / locality / connectivity.

[gregoryyoung]

And this is what you think about when deciding that. Ordering between
partitions is not that hard.

On Monday, July 8, 2013, Damian Hickey wrote:

Yep, and the aggregates may be in different domains and may not even be in
the same store / partition / node / locality / connectivity.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20614696
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

You mean ordering between partitions in GES, and not in a "persistence
agnostic" way, right? ;)

On 8 July 2013 16:48, Greg Young notifications@github.com wrote:

And this is what you think about when deciding that. Ordering between
partitions is not that hard.

On Monday, July 8, 2013, Damian Hickey wrote:

Yep, and the aggregates may be in different domains and may not even be
in
the same store / partition / node / locality / connectivity.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20614696>

.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20614951
.

[gregoryyoung]

In any case. The normal way of doing it is to provide a 100% deterministic
replay and probabilistic realtime with ability to detect failures. Event
sourced does this on mongo as example.

On Monday, July 8, 2013, Damian Hickey wrote:

You mean ordering between partitions in GES, and not in a "persistence
agnostic" way, right? ;)

On 8 July 2013 16:48, Greg Young <notifications@github.com<javascript:_e({}, 'cvml', 'notifications@github.com');>>
wrote:

And this is what you think about when deciding that. Ordering between
partitions is not that hard.

On Monday, July 8, 2013, Damian Hickey wrote:

Yep, and the aggregates may be in different domains and may not even
be
in
the same store / partition / node / locality / connectivity.

—
Reply to this email directly or view it on GitHub<

https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20614696>

.

Le doute n'est pas une condition agréable, mais la certitude est
absurde.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20614951>

.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20615303
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

Yeah, I think NES can be improved in this regard.

To clarify, when you say 100% deterministic do you mean:

1. Given a set of streams, when replayed a second time, and there have been
   0 appends since first replay, you should see the exact same series of event
   messages. Eg:
   Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
   Replay 2: A1, A2, A3, A4, B1, B2, B3, B4

2. Given a set of streams, when replayed a second time, and there have been
   1 or more appends since first replay, you should see the exact same series
   of event messages, with the new appends appearing at the end of the series?
   Eg: Where A5 is the 'latest' append:

Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
Replay 2: A1, A2, A3, A4, B1, B2, B3, B4, A5

At the moment, NES supports scenario 1. In scenario 2 for replay 2, NES
will generate A1, A2, A3, A4, A5, B1, B2, B3, B4. (I need to verify this)

[gregoryyoung]

2

On Monday, July 8, 2013, Damian Hickey wrote:

Yeah, I think NES can be improved in this regard.

To clarify, when you say 100% deterministic do you mean:

1. Given a set of streams, when replayed a second time, and there have been
   0 appends since first replay, you should see the exact same series of event
   messages. Eg:
   Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
   Replay 2: A1, A2, A3, A4, B1, B2, B3, B4

2. Given a set of streams, when replayed a second time, and there have been
   1 or more appends since first replay, you should see the exact same series
   of event messages, with the new appends appearing at the end of the series?
   Eg: Where A5 is the 'latest' append:

Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
Replay 2: A1, A2, A3, A4, B1, B2, B3, B4, A5

At the moment, NES supports scenario 1. In scenario 2 for replay 2, NES
will generate A1, A2, A3, A4, A5, B1, B2, B3, B4. (I need to verify
this)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20616606
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[erothFEI]

I think NES (at least in the SQL engine) is just basing ordering on Commit Sequence. So with this example:

C1 = A1, A2, A3
D1 = B1, B2, B3
C2 = A4
C3 = A5
D2 = B4

So the events happened like this: A1,A2,A3,B1,B2,B3,A4,A5,B4

So NES saves in commits: C1,D1,C2,C3,D2

But when getting all events it would order by Commit sequence: C1,D1,C2,D2,C3

Which would give you events in this order: A1,A2,A3,B1,B2,B3,A4,B4,A5

So if one aggregate grows faster in number of commits compared to other aggregates you start getting more and more out of order events across aggregates.

Eric Roth

[damianh]

Eric is correct
https://github.com/NEventStore/NEventStore/blob/master/src/proj/EventStore.Persistence.SqlPersistence/SqlDialects/CommonSqlStatements.resx#L148

But my main point is that it is still deterministic if there haven't been
any appends. Which may not be of use at all, heh.

Hmm, I wonder if that's why JO use CommitStamp there in the first place? In
any case that was broken when two commits had the same CommitStamp.
Ordering on a timestamp sucks. :|

On 8 July 2013 17:23, erothFEI notifications@github.com wrote:

I think NES (at least in the SQL engine) is just basing ordering on Commit
Sequence. So with this example:

C1 = A1, A2, A3
D1 = B1, B2, B3
C2 = A4
C3 = A5
D2 = B4

So the events happened like this: A1,A2,A3,B1,B2,B3,A4,A5,B4

So NES saves in commits: C1,D1,C2,C3,D2

But when getting all events it would order by Commit sequence:
C1,D1,C2,D2,C3

Which would give you events in this order: A1,A2,A3,B1,B2,B3,A4,B4,A5

So if one aggregate grows faster in number of commits compared to other
aggregates you start getting more and more out of order events across
aggregates.

Eric Roth

office | 443.270.5113
fax | 410.715.6538
cell | 732.779.8039
email | eric.roth@feisystems.commailto:eric.roth@feisystems.com

FEi Systems | 7175 Columbia Gateway Drive – Suite A | Columbia, Maryland
21046<
http://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=7175+Columbia+Gateway+Drive,+Columbia,+MD&sll=37.0625,-95.677068&sspn=43.393645,92.021484&ie=UTF8&hq=&hnear=7175+Columbia+Gateway+Dr,+Columbia,+Howard,+Maryland+21046&ll=39.179379,-76.806879&spn=0.04165,0.089865&z=14>
| www.FEIsystems.comhttp://www.FEIsystems.com

From: Greg Young [mailto:notifications@github.com]
Sent: Monday, July 08, 2013 12:13 PM
To: NEventStore/NEventStore
Cc: Eric Roth
Subject: Re: [NEventStore] Consider global deterministic ordering of
commits across streams (#170)

2

On Monday, July 8, 2013, Damian Hickey wrote:

Yeah, I think NES can be improved in this regard.

To clarify, when you say 100% deterministic do you mean:

1. Given a set of streams, when replayed a second time, and there have
   been
   0 appends since first replay, you should see the exact same series of
   event
   messages. Eg:
   Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
   Replay 2: A1, A2, A3, A4, B1, B2, B3, B4

2. Given a set of streams, when replayed a second time, and there have
   been
   1 or more appends since first replay, you should see the exact same
   series
   of event messages, with the new appends appearing at the end of the
   series?
   Eg: Where A5 is the 'latest' append:

Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
Replay 2: A1, A2, A3, A4, B1, B2, B3, B4, A5

At the moment, NES supports scenario 1. In scenario 2 for replay 2, NES
will generate A1, A2, A3, A4, A5, B1, B2, B3, B4. (I need to verify
this)

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20616606>

.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20616706>.

---

Important: This email is intended for the above named only and may be
confidential, proprietary, and/or legally privileged. If this email has
come to you in error, you must take no action on it, nor may you copy or
show it to anyone. Please contact the sender and delete the material from
any computer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20617495
.

[erothFEI]

Yea that is why I tried creating that fork that just significantly reduced the chance of duplicate CommitStamps by increasing the precision. You could practically eliminate the chance of duplicate CommitStamps on the same machine and significantly reduce the chance across machines because CommitStamps would be stored at the nanosecond level (of course depending if the DB can store that much precision).

Using vector clocks would also work but it has issues of its own. In GES, what are you using as the actor, the actual client or the servers? Do you do vector pruning?

Eric Roth

[gregoryyoung]

In single partition we use two longs (prepare and commit position logically
in partition). When distributed it will Vector clocks

On Monday, July 8, 2013, erothFEI wrote:

Yea that is why I tried creating that fork that just significantly reduced
the chance of duplicate CommitStamps by increasing the precision. You could
practically eliminate the chance of duplicate CommitStamps on the same
machine and significantly reduce the chance across machines because
CommitStamps would be stored at the nanosecond level (of course depending
if the DB can store that much precision).

Using vector clocks would also work but it has issues of its own. In GES,
what are you using as the actor, the actual client or the servers? Do you
do vector pruning?

Eric Roth

office | 443.270.5113
fax | 410.715.6538
cell | 732.779.8039
email | eric.roth@feisystems.com <javascript:_e({}, 'cvml',
'eric.roth@feisystems.com');><mailto:eric.roth@feisystems.com<javascript:_e({}, 'cvml', 'eric.roth@feisystems.com');>>

FEi Systems | 7175 Columbia Gateway Drive – Suite A | Columbia, Maryland
21046<7175 Columbia Gateway Drive, Columbia, MDhttp://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=7175+Columbia+Gateway+Drive,+Columbia,+MD&sll=37.0625,-95.677068&sspn=43.393645,92.021484&ie=UTF8&hq=&hnear=7175+Columbia+Gateway+Dr,+Columbia,+Howard,+Maryland+21046&ll=39.179379,-76.806879&spn=0.04165,0.089865&z=14>
| www.FEIsystems.comhttp://www.FEIsystems.com

From: Damian Hickey [mailto:notifications@github.com <javascript:_e({},
'cvml', 'notifications@github.com');>]
Sent: Monday, July 08, 2013 12:36 PM
To: NEventStore/NEventStore
Cc: Eric Roth
Subject: Re: [NEventStore] Consider global deterministic ordering of
commits across streams (#170)

Eric is correct

https://github.com/NEventStore/NEventStore/blob/master/src/proj/EventStore.Persistence.SqlPersistence/SqlDialects/CommonSqlStatements.resx#L148

But my main point is that it is still deterministic if there haven't been
any appends. Which may not be of use at all, heh.

Hmm, I wonder if that's why JO use CommitStamp there in the first place?
In
any case that was broken when two commits had the same CommitStamp.
Ordering on a timestamp sucks. :|

On 8 July 2013 17:23, erothFEI <notifications@github.com<javascript:_e({}, 'cvml', 'notifications@github.com');>
<mailto:notifications@github.com <javascript:_e({}, 'cvml',
'notifications@github.com');>>> wrote:

I think NES (at least in the SQL engine) is just basing ordering on
Commit
Sequence. So with this example:

C1 = A1, A2, A3
D1 = B1, B2, B3
C2 = A4
C3 = A5
D2 = B4

So the events happened like this: A1,A2,A3,B1,B2,B3,A4,A5,B4

So NES saves in commits: C1,D1,C2,C3,D2

But when getting all events it would order by Commit sequence:
C1,D1,C2,D2,C3

Which would give you events in this order: A1,A2,A3,B1,B2,B3,A4,B4,A5

So if one aggregate grows faster in number of commits compared to other
aggregates you start getting more and more out of order events across
aggregates.

Eric Roth

office | 443.270.5113
fax | 410.715.6538
cell | 732.779.8039
email | eric.roth@feisystems.com <javascript:_e({}, 'cvml',
'eric.roth@feisystems.com');><mailto:eric.roth@feisystems.com<javascript:_e({}, 'cvml', 'eric.roth@feisystems.com');>
<mailto:eric.roth@feisystems.com <javascript:_e({}, 'cvml',
'eric.roth@feisystems.com');>%3cmailto:eric.roth@feisystems.com<javascript:_e({}, 'cvml', '3cmailto%3Aeric.roth@feisystems.com');>>>

FEi Systems | 7175 Columbia Gateway Drive – Suite A | Columbia, Maryland
21046<
7175 Columbia Gateway Drive, Columbia, MDhttp://maps.google.com/maps?f=q&source=s_q&hl=en&geocode=&q=7175+Columbia+Gateway+Drive,+Columbia,+MD&sll=37.0625,-95.677068&sspn=43.393645,92.021484&ie=UTF8&hq=&hnear=7175+Columbia+Gateway+Dr,+Columbia,+Howard,+Maryland+21046&ll=39.179379,-76.806879&spn=0.04165,0.089865&z=14>

| www.FEIsystems.com<http://www.FEIsystems.com<http://www.FEIsystems.com
%3chttp:/www.FEIsystems.com>>

From: Greg Young [mailto:notifications@github.com <javascript:_e({},
'cvml', 'notifications@github.com');>]
Sent: Monday, July 08, 2013 12:13 PM
To: NEventStore/NEventStore
Cc: Eric Roth
Subject: Re: [NEventStore] Consider global deterministic ordering of
commits across streams (#170)

2

On Monday, July 8, 2013, Damian Hickey wrote:

Yeah, I think NES can be improved in this regard.

To clarify, when you say 100% deterministic do you mean:

1. Given a set of streams, when replayed a second time, and there have
   been
   0 appends since first replay, you should see the exact same series of
   event
   messages. Eg:
   Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
   Replay 2: A1, A2, A3, A4, B1, B2, B3, B4

2. Given a set of streams, when replayed a second time, and there have
   been
   1 or more appends since first replay, you should see the exact same
   series
   of event messages, with the new appends appearing at the end of the
   series?
   Eg: Where A5 is the 'latest' append:

Replay 1: A1, A2, A3, A4, B1, B2, B3, B4
Replay 2: A1, A2, A3, A4, B1, B2, B3, B4, A5

At the moment, NES supports scenario 1. In scenario 2 for replay 2,
NES
will generate A1, A2, A3, A4, A5, B1, B2, B3, B4. (I need to verify
this)

—
Reply to this email directly or view it on GitHub<

https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20616606>

.

Le doute n'est pas une condition agréable, mais la certitude est
absurde.

—
Reply to this email directly or view it on GitHub<

https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20616706>.

---

Important: This email is intended for the above named only and may be
confidential, proprietary, and/or legally privileged. If this email has
come to you in error, you must take no action on it, nor may you copy or
show it to anyone. Please contact the sender and delete the material
from
any computer.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20617495>

.

—
Reply to this email directly or view it on GitHub<
https://github.com/NEventStore/NEventStore/issues/170#issuecomment-20618350>.

---

Important: This email is intended for the above named only and may be
confidential, proprietary, and/or legally privileged. If this email has
come to you in error, you must take no action on it, nor may you copy or
show it to anyone. Please contact the sender and delete the material from
any computer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20618923
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[mynkow]

@erothFEI solution is far better than the current one. As already said "So if one aggregate grows faster in number of commits compared to other aggregates you start getting more and more out of order events across aggregates.". Of course you cannot get 100% correct order but you do not get 5-6 month older events replayed after the newest ones. On Monday I will try @erothFEI's code and ping back.

[gregoryyoung]

Just take a subscribe an introduce delay of say 5 seconds to reorder.
Detecting is simple if done right for failures. But when adapter is there
is 10x faster than closest and supports.

On Monday, July 8, 2013, mynkow wrote:

@erothFEI https://github.com/erothFEI solution is far better than the
current one. As already said "So if one aggregate grows faster in number of
commits compared to other aggregates you start getting more and more out of
order events across aggregates.". Of course you cannot get 100% correct
order but you do not get 5-6 month older events replayed after the newest
ones. On Monday I will try @erothFEI https://github.com/erothFEI's code
and ping back.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/170#issuecomment-20625847
.

Le doute n'est pas une condition agréable, mais la certitude est absurde.

[damianh]

@mynkow If you do this, as per #159, you introduce the possibility of events being out of order within a stream. While there is a slim chance of it occurring, it's a serious issue when it does.

[erothFEI]

you can always change the order by to be by commit stamp then commit sequence you will just want to make sure you have all your indexes correct so it doesn't get slow.

[mynkow]

@damianh I am thinking to load the events by commitstamp only when doing replay. This way loading aggregates will work normally. In theory this must work because when replaying the events there are no ARs involved at all. What do you think?

[damianh]

@mynkow Doesn't guarantee ordering. There could still be commit messages within the same stream that have the same stamp. Depending on your system, this may be very unlikely to occur, but when it does, it could be nasty.

We are currently considering tenant/partition level ordering though.

[damianh]

Fixed. See forthcoming announcement on mailing list.

[serra]

I assume you are referring to the v5 CI packages announcement (including PollingClient) on the mailing list?

[damianh]

@serra Yep.