Repo history rewrite #1479
Merged
Repo history rewrite #1479
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
devinivy
reviewed
Aug 21, 2023
bnewbold
reviewed
Aug 21, 2023
devinivy
approved these changes
Aug 24, 2023
devinivy
reviewed
Aug 28, 2023
packages/pds/src/sql-repo-storage.ts
Outdated
Comment on lines
241
to
247
| qb | ||
| .where('repoRev', '>', cursor.rev) | ||
| .orWhere((inner) => | ||
| inner | ||
| .where('repoRev', '=', cursor.rev) | ||
| .where('cid', '>', cursor.cid.toString()), | ||
| ), |
There was a problem hiding this comment.
Would recommend using this pattern to ensure you hit the index: ("repoRev", "cid") > (${cursor.rev}, ${cursor.cid}). Shouldn't really be necessary, but we've seen it miss the index the other way for sure 🙃
There was a problem hiding this comment.
ah good looks 👍 simpler too
devinivy
approved these changes
Aug 28, 2023
devinivy
approved these changes
Aug 29, 2023
mloar
pushed a commit
to mloar/atproto
that referenced
this pull request
Sep 26, 2023
* logical changes in repo * tests * tweak commit data * building pds * patching up some tests * tidy + more tests * patch up bsky * clean up db * db migration * small patches * fix up another test * reinclude prevs * api & lex updates * add back in deprecated routes * backward compatibility for commit v2 * add upgrade repo version root * move namespace * migration test * patch up a few more tests * remove deprecated rebase routes * tweak api * sprinkle rev around a few more places * getCurrent -> getLatestCommit * fix test * pr feedback & tidy * fix up block pagination * tidy again * add in a tets * add cursor to listBlobs * getRepo rev -> since * clean up proofs test * dont change getHead * tweak migrate route * build branch * hit index in block range query * check for dupe record refs * tidy * set prev to null * dont build branch
mloar
pushed a commit
to mloar/atproto
that referenced
this pull request
Nov 15, 2023
* logical changes in repo * tests * tweak commit data * building pds * patching up some tests * tidy + more tests * patch up bsky * clean up db * db migration * small patches * fix up another test * reinclude prevs * api & lex updates * add back in deprecated routes * backward compatibility for commit v2 * add upgrade repo version root * move namespace * migration test * patch up a few more tests * remove deprecated rebase routes * tweak api * sprinkle rev around a few more places * getCurrent -> getLatestCommit * fix test * pr feedback & tidy * fix up block pagination * tidy again * add in a tets * add cursor to listBlobs * getRepo rev -> since * clean up proofs test * dont change getHead * tweak migrate route * build branch * hit index in block range query * check for dupe record refs * tidy * set prev to null * dont build branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Removes history from repos, introduces new sync semantics, and provides an upgrade path to repo v3.
Adds a temporary admin
com.atproto.temp.upgradeRepoVersionroute through which we can upgrade repos at our own pace.More discussion here: #1410
Blog post with discussion of changes here: https://atproto.com/blog/repo-sync-update
Note for reviewing: I would review this in in the following order