Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Propagate apply_lsn from SK to PS to prevent GC from collecting objects which may be still requested by replica #7368

Merged
merged 4 commits into from
May 21, 2024
Merged

Propagate apply_lsn from SK to PS to prevent GC from collecting objects which may be still requested by replica #7368

merged 4 commits into from
May 21, 2024

Commits on May 21, 2024

  1. Propagate standby apply LSN to pageserver to hold off GC. 

    To avoid pageserver gc'ing data needed by standby, propagate standby apply LSN
through standby -> safekeeper -> broker -> pageserver flow and hold off GC for
it. Iteration of GC resets the value to remove the horizon when standby goes
away -- pushes are assumed to happen at least once between gc iterations. As a
safety guard max allowed lag compared to normal GC horizon is hardcoded as 10GB.
Add test for the feature.

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
    @arssher
    arssher and Konstantin Knizhnik committed May 21, 2024
    Configuration menu
    Copy the full SHA
    5578750 View commit details
    Browse the repository at this point in the history

  2. Fix bugs in hot standby feedback propagation and add test for it. 

    Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
    @arssher
    arssher and Konstantin Knizhnik committed May 21, 2024
    Configuration menu
    Copy the full SHA
    bb23edd View commit details
    Browse the repository at this point in the history

  3. Add metric for pageserver standby horizon. 

    Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
    @arssher
    arssher and Konstantin Knizhnik committed May 21, 2024
    Configuration menu
    Copy the full SHA
    ea52253 View commit details
    Browse the repository at this point in the history

  4. Minimize hot standby feedback xmins to next_xid. 

    Hot standby feedback xmins can be greater than next_xid due to sparse update of
nextXid on pageserver (to do less writes it advances next xid on
1024). ProcessStandbyHSFeedback ignores such xids from the future; to fix,
minimize received xmin to next_xid.

Co-authored-by: Konstantin Knizhnik <knizhnik@neon.tech>
    @arssher
    arssher and Konstantin Knizhnik committed May 21, 2024
    Configuration menu
    Copy the full SHA
    f02aeb5 View commit details
    Browse the repository at this point in the history