Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Journal should reject append if the sequence number is not greater than the previous record #6217

Closed
deepthidevaki opened this issue Jan 28, 2021 · 1 comment · Fixed by #10305 or #10348
Closed

Journal should reject append if the sequence number is not greater than the previous record #6217

deepthidevaki opened this issue Jan 28, 2021 · 1 comment · Fixed by #10305 or #10348
Assignees
@megglos
Labels
kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. scope/broker Marks an issue or PR to appear in the broker section of the changelog version:8.1.0 Marks an issue as being completely or in parts released in 8.1.0

Comments

@deepthidevaki
Copy link
Contributor

deepthidevaki commented Jan 28, 2021 •
edited
Loading

#6216 introduces a journal module with new api. The new api includes an application sequence number which is a number provided by the application. The application can use either this sequence number or the index to seek to a record. The journal requires that the sequence number is always increasing. The first version implemented in #6216 does not validate this. However it is required that journal implementation rejects the append when the sequence number is not greater than the previous record.

Scope:

  • Implement correct behaviour
  • Introduce a new exception/error to indicate the failure if necessary
  • Add tests
@deepthidevaki deepthidevaki added kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. scope/broker Marks an issue or PR to appear in the broker section of the changelog labels Jan 28, 2021
@deepthidevaki deepthidevaki added this to the Journal Refactor milestone Jan 28, 2021
@npepinpe npepinpe added this to Planned in Zeebe Mar 24, 2021
@deepthidevaki deepthidevaki removed this from the Journal Refactor milestone Mar 29, 2021
@npepinpe npepinpe removed this from Planned in Zeebe Oct 19, 2021
This was referenced Sep 8, 2022
Merged
Merged
@megglos megglos self-assigned this Sep 13, 2022
@megglos
Copy link
Contributor

megglos commented Sep 14, 2022

Reopening as in a benchmark we hit a bug that if a leader steps down the asqn is not reset properly, causing the partition to become unhealthy on that node if the new leader is replicating entries with a lower asqn.

@megglos megglos mentioned this issue Sep 14, 2022
Merged
10 tasks
zeebe-bors-camunda bot added a commit that referenced this issue Sep 14, 2022
@zeebe-bors-camunda @megglos
merge: #10348
15c1814 
10348: fix(journal): reset writer asqn on truncate r=megglos a=megglos

## Description

On truncate the current lastAsqn of the segment writer is reset to allow overwriting non-committed records afterwards.

## Related issues

<!-- Which issues are closed by this PR or are related -->

closes #6217



Co-authored-by: Meggle (Sebastian Bathke) <sebastian.bathke@camunda.com>
@Zelldon Zelldon added the version:8.1.0 Marks an issue as being completely or in parts released in 8.1.0 label Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@megglos megglos

Labels
kind/toil Categorizes an issue or PR as general maintenance, i.e. cleanup, refactoring, etc. scope/broker Marks an issue or PR to appear in the broker section of the changelog version:8.1.0 Marks an issue as being completely or in parts released in 8.1.0
Projects
None yet
Milestone
No milestone
5 participants
@npepinpe @megglos @menski @deepthidevaki @Zelldon