-
Notifications
You must be signed in to change notification settings - Fork 360
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(pageserver): add metadata key range and aux key encoding #7401
Conversation
Signed-off-by: Alex Chi Z <chi@neon.tech>
I'm thinking about adding some test cases for key ordering, maybe in later pull requests. Ordering of keys are not important as we are using hashing for AUX keys. It might be important if at some point we transform to variable-sized keys or have some metadata keys requiring ordering when scanning. |
Signed-off-by: Alex Chi Z <chi@neon.tech>
2766 tests run: 2646 passed, 0 failed, 120 skipped (full report)Code coverage* (full report)
* collected from Rust tests only The comment gets automatically updated with the latest test results
e48f8d9 at 2024-04-23T15:23:06.848Z :recycle: |
checked by default non-rel-block key is on shard 0 as in key_is_shard0 |
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
Signed-off-by: Alex Chi Z <chi@neon.tech>
…h/metadata-key-encoding
reserving merge conflict with #7404 and not adding aux key range into |
Problem
Extracted from #7375. We assume everything >= 0x80 are metadata keys. AUX file keys are part of the metadata keys, and we use
0x90
as the prefix for AUX file keys.The AUX file encoding is described in the code comment. We use xxhash128 as the hash algorithm. It seems to be portable according to the introduction,
...though whether the Rust version follows the same convention is unknown and might need manual review of the library. Anyways, we can always change the hash algorithm before rolling it out in staging/end-user, and I made a quick decision to use xxhash here because it generates 128b hash + portable. We can save the discussion of which hash algorithm to use later.
Summary of changes
Checklist before requesting a review
Checklist before merging