Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replicate only durable commits to replicas during recovery #1991

Merged
merged 1 commit into from
Apr 30, 2024

Conversation

as51340
Copy link
Contributor

@as51340 as51340 commented Apr 30, 2024

Description

Send only durable part of the buffer to replicas during recovery
Better checker for jepsen test for easier debugging.
Improved logs to more easily reason about applying deltas during recovery.

Before this PR, even with fsync set to 1, replicas could get non-durable data from main. If main would die and get back, replica could be in more advanced state than main = issue.

[master < Task] PR

  • Provide the full content or a guide for the final git message
    • Replica only durable commits to replicas during recovery

Documentation checklist

  • Add the documentation label tag
  • Add the bug / feature label tag
  • Add the milestone for which this feature is intended
    • If not known, set for a later milestone
  • Write a release note, including added/changed clauses
    • During recovery process, when replicas are behind main, they will receive only commits which are already fsynced on main.
  • Link the documentation PR here
  • Tag someone from docs team in the comments

@as51340 as51340 added bug bug Docs needed Docs needed CI -build=coverage -test=core Run coverage build and core tests on push CI -build=jepsen -test=core Run jepsen build and core tests on push labels Apr 30, 2024
@as51340 as51340 added this to the mg-v2.17.0 milestone Apr 30, 2024
@as51340 as51340 requested a review from Ignition April 30, 2024 10:05
@as51340 as51340 added the CI -build=release -test=e2e Run release build and e2e tests on push label Apr 30, 2024
@as51340 as51340 enabled auto-merge April 30, 2024 10:06
src/utils/file.cpp Outdated Show resolved Hide resolved
Copy link
Collaborator

@antoniofilipovic antoniofilipovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All good to me

@antoniofilipovic
Copy link
Collaborator

Maybe also comment about the PR name, we are sending only durable commits to replica in recovery, the buffer is not sent at all

@as51340 as51340 changed the title Send only durable part of the buffer to replicas during recovery Replica only durable commits to replicas during recovery Apr 30, 2024
@antoniofilipovic antoniofilipovic changed the title Replica only durable commits to replicas during recovery Replicate only durable commits to replicas during recovery Apr 30, 2024
Copy link

sonarcloud bot commented Apr 30, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@as51340 as51340 added this pull request to the merge queue Apr 30, 2024
Merged via the queue into master with commit da60621 Apr 30, 2024
11 checks passed
@as51340 as51340 deleted the fix-repl-log branch April 30, 2024 15:06
@as51340
Copy link
Contributor Author

as51340 commented May 2, 2024

@kgolubic RN above, docs written

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking bug bug CI -build=coverage -test=core Run coverage build and core tests on push CI -build=jepsen -test=core Run jepsen build and core tests on push CI -build=release -test=e2e Run release build and e2e tests on push Docs needed Docs needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants