Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Detect and recover projectors that are ahead of the event store. #96

Merged

Conversation

dennisdoomen
Copy link
Collaborator

@dennisdoomen dennisdoomen commented May 23, 2017

Adds a new options object that allows a subscriber to tell the dispatcher to restart the subscription at the first transaction available in the underlying event source, and allow the subscriber to clean-up just before that.

@dennisdoomen dennisdoomen changed the title [WIP} Detect and recover projectors that are ahead of the event store. [WIP] Detect and recover projectors that are ahead of the event store. May 23, 2017
@dennisdoomen dennisdoomen changed the title [WIP] Detect and recover projectors that are ahead of the event store. Detect and recover projectors that are ahead of the event store. May 23, 2017
{
yield return batch;
}
yield return batch;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we handle an empty batch?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because an implementation of an event store must provide at least one empty set of transactions, so that the dispatcher can detect subscribers that are ahead of the event store.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why you want to make that a responsibility of this method. Its name tells us that it should just split the sequence into batches of the given size.

Also the current implementation could lead to 3 different scenarios:

  1. It can return 1 empty batch
  2. It can return 1 or more non-empty batches
  3. It can return 1 or more non-empty batches followed by an empty batch

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True, but the MemoryEventSource needs this to be able to meet my new requirement that an event source should always send an empty set of transactions right after subscribing, if there are no transactions beyond the provided checkpoint

{
try
{
await handler(transactions);
if (options.RestartWhenAhead && !transactions.Any())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why absence of transactions here means being ahead of event store. Why is it called without transactions at all? This delegate is invoked multiple times. What if the second call has no transactions?

If we want the event store to notify us of being ahead, I would create a separate delegate parameter for that with a clear responsibility

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it's the other way around. By changing the subscription so that it requests the last transaction the subscriber has handled, and then verifying if it is still there, we can conclude that the subscriber is ahead. Otherwise we would have every event store signal us when we request a checkpoint that is beyond its last checkpoint. By going this route, we keep the complexity in the dispatcher rather than in each implementation of an event store.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have to modify all the event stores anyway. Otherwise they will not call this delegate with empty batch. So they better call another delegate in that case.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I tend to agree with you. Back to the drawing board....

@dennisdoomen
Copy link
Collaborator Author

Adding another delegate would mean the subscription delegate will start to look like this;

    public delegate IDisposable CreateSubscription(long? lastProcessedCheckpoint,
        Func<IReadOnlyList<Transaction>, SubscriptionInfo, Task> handler, Func<Task> noSuchCheckpoint,
        string subscriptionId);

Feels a bit awkward. What do you think?

@dennisdoomen dennisdoomen force-pushed the SupportAutorestart branch 3 times, most recently from e6e12b7 to 9adaed7 Compare May 26, 2017 07:21
@dennisdoomen
Copy link
Collaborator Author

@IharBury please review again. I've applied your proposal.

{
return Subscribe(checkpoint, handler, null);
BeforeRestarting = () => Task.FromResult(0);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Syntax suggestion: you can assign the initial value in the property declaration:
public Func<Task> BeforeRestarting { get; set; } = () => Task.FromResult(0);

@dennisdoomen dennisdoomen merged commit 6e68f84 into liquidprojections:master May 29, 2017
@dennisdoomen dennisdoomen deleted the SupportAutorestart branch May 29, 2017 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants