Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: local layer path followups #7640

Merged
merged 15 commits into from
May 8, 2024
Merged

Conversation

jcsp
Copy link
Contributor

@jcsp jcsp commented May 7, 2024

  • Rename "filename" types which no longer map directly to a filename (LayerFileName -> LayerName)
  • Add a -v1- part to local layer paths to smooth the path to future updates (we anticipate a -v2- that uses checksums later)
  • Rename methods that refer to the string-ized version of a LayerName to no longer be called "filename"
  • Refactor reconcile() function to use a LocalLayerFileMetadata type that includes the local path, rather than carrying local path separately in a tuple and unwrap()'ing it later.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@jcsp jcsp added c/storage/pageserver Component: storage: pageserver a/tech_debt Area: related to tech debt labels May 7, 2024
@jcsp jcsp force-pushed the jcsp/issue-7504-local-paths-pt2 branch from 581c60d to f462807 Compare May 7, 2024 17:07
Copy link

github-actions bot commented May 7, 2024

3024 tests run: 2892 passed, 0 failed, 132 skipped (full report)


Flaky tests (3)

Postgres 16

  • test_vm_bit_clear_on_heap_lock: debug

Postgres 15

  • test_partial_evict_tenant[relative_equal]: release

Postgres 14

  • test_statvfs_pressure_usage: debug

Code coverage* (full report)

  • functions: 31.4% (6313 of 20124 functions)
  • lines: 47.3% (47580 of 100657 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
a1d43d8 at 2024-05-08T16:58:12.295Z :recycle:

@jcsp jcsp force-pushed the jcsp/issue-7504-local-paths-pt2 branch from f462807 to 16ba767 Compare May 8, 2024 09:48
@jcsp jcsp requested a review from problame May 8, 2024 09:49
@jcsp jcsp marked this pull request as ready for review May 8, 2024 09:52
@jcsp jcsp requested a review from a team as a code owner May 8, 2024 09:52
@jcsp jcsp force-pushed the jcsp/issue-7504-local-paths-pt2 branch from 16ba767 to eeb53b7 Compare May 8, 2024 11:25
pageserver/src/tenant/timeline/init.rs Show resolved Hide resolved
pageserver/src/tenant/timeline/init.rs Outdated Show resolved Hide resolved
pageserver/src/tenant/tasks.rs Outdated Show resolved Hide resolved
test_runner/regress/test_pageserver_generations.py Outdated Show resolved Hide resolved
@jcsp jcsp enabled auto-merge (squash) May 8, 2024 16:16
@jcsp jcsp merged commit ca154d9 into main May 8, 2024
49 checks passed
@jcsp jcsp deleted the jcsp/issue-7504-local-paths-pt2 branch May 8, 2024 16:50
jcsp added a commit that referenced this pull request May 14, 2024
We recently added support for local layer paths that contain a
generation number:
- #7609
- #7640

Now that we've cut a
[release](#7735) that includes
those changes, we can proceed to enable writing the new format without
breaking forward compatibility.
a-masterov pushed a commit that referenced this pull request May 20, 2024
- Rename "filename" types which no longer map directly to a filename
(LayerFileName -> LayerName)
- Add a -v1- part to local layer paths to smooth the path to future
updates (we anticipate a -v2- that uses checksums later)
- Rename methods that refer to the string-ized version of a LayerName to
no longer be called "filename"
- Refactor reconcile() function to use a LocalLayerFileMetadata type
that includes the local path, rather than carrying local path separately
in a tuple and unwrap()'ing it later.
a-masterov pushed a commit that referenced this pull request May 20, 2024
We recently added support for local layer paths that contain a
generation number:
- #7609
- #7640

Now that we've cut a
[release](#7735) that includes
those changes, we can proceed to enable writing the new format without
breaking forward compatibility.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a/tech_debt Area: related to tech debt c/storage/pageserver Component: storage: pageserver
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants