Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remember last written LSN when it is first requested #7343

Merged
merged 4 commits into from Apr 11, 2024

Conversation

knizhnik
Copy link
Contributor

@knizhnik knizhnik commented Apr 8, 2024

Problem

See https://neondb.slack.com/archives/C03QLRH7PPD/p1712529369520409

In case of statements CREATE TABLE AS SELECT... or INSERT FROM SELECT... we are fetching data from source table and storing it in destination table. It cause problems with prefetch last-written-lsn is known for the pages of source table
(which for example happens after compute restart). In this case we get get global value of last-written-lsn which is changed frequently as far as we are writing pages of destination table. As a result request-isn for the prefetch and request-let when this page is actually needed are different and we got exported prefetch request. So it actually disarms prefetch.

Summary of changes

Proposed simple patch stores last-written LSN for the page when it is not found. So next time we will request last-written LSN for this page, we will get the same value (certainly if the page was not changed).

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@knizhnik knizhnik requested a review from a team as a code owner April 8, 2024 12:34
@knizhnik knizhnik requested review from hlinnaka and MMeent April 8, 2024 12:34
Copy link

github-actions bot commented Apr 8, 2024

2754 tests run: 2636 passed, 0 failed, 118 skipped (full report)


Flaky tests (1)

Postgres 14

  • test_timeline_size_quota_on_startup: release

Code coverage* (full report)

  • functions: 28.1% (6429 of 22866 functions)
  • lines: 46.8% (45139 of 96400 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
ee6591f at 2024-04-10T19:01:19.074Z :recycle:

@MMeent
Copy link
Contributor

MMeent commented Apr 10, 2024

Please ping me after PG's PRs are merged & the references here are updated.

@knizhnik
Copy link
Contributor Author

Please ping me after PG's PRs are merged & the references here are updated.

Extra spaces are removed, PG versions are bumped

Copy link
Contributor

@MMeent MMeent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@knizhnik knizhnik merged commit d47e4a2 into main Apr 11, 2024
53 of 55 checks passed
@knizhnik knizhnik deleted the remember_last_written_lsn branch April 11, 2024 04:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants