Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: fix AUX key vectored get validation #7018

Merged
merged 2 commits into from Mar 5, 2024

Conversation

VladLazar
Copy link
Contributor

@VladLazar VladLazar commented Mar 5, 2024

Problem

The value reconstruct of AUX_FILES_KEY from records is not deterministic since it uses a hash map under the hood.
This caused vectored get validation failures when enabled in staging.

Summary of changes

Deserialise AUX_FILES_KEY blobs comparing. All other keys should reconstruct deterministically,
so we simply compare the blobs.

Testing

Tested this locally with data extracted from a tenant which ran into this problem in staging.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

The value reconstruct of AUX_FILES_KEY from records is not deterministic
since it uses a hash map under the hood. Hence, deserialise both results
before comparing. All other keys should reconstruct deterministically,
so we simply compare the blobs.
@VladLazar VladLazar marked this pull request as ready for review March 5, 2024 12:18
@VladLazar VladLazar requested a review from a team as a code owner March 5, 2024 12:18
Copy link
Contributor

@koivunej koivunej left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good after the removal of repeated left and right (repeated after assert_eq! macro has already done that).

Copy link

github-actions bot commented Mar 5, 2024

2484 tests run: 2361 passed, 0 failed, 123 skipped (full report)


Code coverage* (full report)

  • functions: 28.8% (6963 of 24204 functions)
  • lines: 47.3% (42649 of 90238 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
a9e0d09 at 2024-03-05T13:17:42.607Z :recycle:

@VladLazar VladLazar merged commit ae8468f into main Mar 5, 2024
53 checks passed
@VladLazar VladLazar deleted the vlad/vectored-get-validation-deser branch March 5, 2024 13:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants