New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use leader's clock when calculating LTS cutoff timestamp #12998
Conversation
{State, Result}; | ||
apply( | ||
_RaftMeta, | ||
#{?tag := storage_event, ?payload := CustomEvent, ?now := Now}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tiny nit: for some reason, dialyzer didn't complain about this new tag in ra_command()
. 🙈
?tp(emqx_ds_storage_layer_store_batch, #{ | ||
shard => Shard, messages => Messages, options => Options | ||
}), | ||
case prepare_batch(Shard, Messages, Options) of |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Q: Should this piece of code eventually end up in what's now ra_store_batch/3
?
b6ecec8
to
9f7ef9f
Compare
Fixes EMQX-12198
Release version: v/e5.7
Summary
This PR fixes clock skew issues that could lead to the loss of messages during replay of the messages in the current epoch of LTS storage.
On the storage layer, this PR also separates preparation of the batch from committing it.
PR Checklist
Please convert it to a draft if any of the following conditions are not met. Reviewers may skip over until all the items are checked:
changes/(ce|ee)/(feat|perf|fix|breaking)-<PR-id>.en.md
filesChecklist for CI (.github/workflows) changes
changes/
dir for user-facing artifacts update