RFC: Reftable backend current version #1215
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
There are issues in commit ff80a24: |
|
There are issues in commit c6ec2d9: |
hanwen
force-pushed
the
reftable-rfc
branch
2 times, most recently
from
021c78b
to
d9bba41
Compare
|
There are issues in commit d9bba41: |
|
/submit |
|
Submitted as pull.1215.git.git.1644351400761.gitgitgadget@gmail.com To fetch this version into To fetch this version to local tag |
gitster
force-pushed
the
seen
branch
10 times, most recently
from
e21e376
to
df5906d
Compare
gitster
force-pushed
the
seen
branch
10 times, most recently
from
fda3590
to
6979b59
Compare
gitster
force-pushed
the
seen
branch
10 times, most recently
from
e4658f5
to
12f70ac
Compare
gitster
force-pushed
the
seen
branch
2 times, most recently
from
b644e66
to
fd5d4eb
Compare
hanwen
force-pushed
the
reftable-rfc
branch
from
4f29902
to
6e2bc0f
Compare
hanwen
force-pushed
the
reftable-rfc
branch
4 times, most recently
from
46ed454
to
17d751e
Compare
|
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
For background, see Documentation/technical/reftable.txt. This introduces the file refs/reftable-backend.c containing a reftable-powered ref storage backend. It can be activated by setting GIT_TEST_REFTABLE in the environment. When GIT_TEST_REFTABLE is set, the test prerequisite !REFFILES is set. There is no option to git-init for now, as the test suite still shows failures with GIT_TEST_REFTABLE=1. Example use: see t/t0031-reftable.sh Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Helped-by: Junio Hamano <gitster@pobox.com> Helped-by: Patrick Steinhardt <patrick.steinhardt@elego.de> Co-authored-by: Jeff King <peff@peff.net>
Documentation/glossary.txt defines precisely what pseudorefs are, but
does not define their syntax. Previous code suggests that they are all
uppercase at toplevel, but this is not true:
* HEAD is not a pseudo ref.
* various tests create symrefs called "FOO"
* refs under rebase-apply/ and rebase-merge/ are assumed to be in the
filesystem (so pending rebases can be aborted by deleting the tree
under .git/rebase-{apply,merge/).
Instead of trying to use a generic definition, setup a hardcoded list
of known pseudo-refs.
Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
hanwen
added a commit
to hanwen/git
that referenced
this pull request
Sep 19, 2023
This introduces the reftable backend as an alternative to the packed backend. This is an alternative to the approach which was explored in git#1215. This alternative is simpler the code, because we now longer have to worry about: - pseudorefs and the HEAD ref - worktrees - commands that blur together files and references (cherry-pick, rebase) This deviates from the spec that in Documentation/technical/reftable.txt. It might be possible to update the code such that all writes by default go to reftable directly. Then the result would be compatible with an implementation that writes only reftable (the reftable lock would still prevent races) relative to an implementation that disregards loose refs. Or, JGit could be adapted to follow this implementation. For this incremental path, the reftable format is arguably more complex than necessary, as - packed-refs doesn't support symrefs - reflogs aren't moved into reftable/ on the other hand, the code is already there, and it's well-structured and well-tested. This implementation is a prototype. To test, you need to do `export GIT_TEST_REFTABLE=1`. Doing so creates a handful of errors in the test-suite, most seemingly related to the new behavior of pack-refs (which creates a reftable/ dir and not a packed-refs file.), but it seems overseeable. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
hanwen
added a commit
to hanwen/git
that referenced
this pull request
Sep 19, 2023
This introduces the reftable backend as an alternative to the packed backend. This is an alternative to the approach which was explored in git#1215. This alternative is simpler the code, because we now longer have to worry about: - pseudorefs and the HEAD ref - worktrees - commands that blur together files and references (cherry-pick, rebase) This deviates from the spec that in Documentation/technical/reftable.txt. It might be possible to update the code such that all writes by default go to reftable directly. Then the result would be compatible with an implementation that writes only reftable (the reftable lock would still prevent races) relative to an implementation that disregards loose refs. Or, JGit could be adapted to follow this implementation. For this incremental path, the reftable format is arguably more complex than necessary, as - packed-refs doesn't support symrefs - reflogs aren't moved into reftable/ on the other hand, the code is already there, and it's well-structured and well-tested. refs/reftable-backend.c was created by cannibalizing the first version of reftable support (github PR 1215 as mentioned above). It supports reflogs and symrefs, even though those are never exercised. This implementation is a prototype, for the following reasons: - no considerations of backward compatibility and configuring an extension - no support for converting between packed-refs and reftable - no documentation - test failures when setting GIT_TEST_REFTABLE=1. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
hanwen
added a commit
to hanwen/git
that referenced
this pull request
Sep 19, 2023
This introduces the reftable backend as an alternative to the packed backend. This is an alternative to the approach which was explored in git#1215. This alternative is simpler the code, because we now longer have to worry about: - pseudorefs and the HEAD ref - worktrees - commands that blur together files and references (cherry-pick, rebase) This deviates from the spec that in Documentation/technical/reftable.txt. It might be possible to update the code such that all writes by default go to reftable directly. Then the result would be compatible with an implementation that writes only reftable (the reftable lock would still prevent races) relative to an implementation that disregards loose refs. Or, JGit could be adapted to follow this implementation. For this incremental path, the reftable format is arguably more complex than necessary, as - packed-refs doesn't support symrefs - reflogs aren't moved into reftable/ on the other hand, the code is already there, and it's well-structured and well-tested. refs/reftable-backend.c was created by cannibalizing the first version of reftable support (github PR 1215 as mentioned above). It supports reflogs and symrefs, even though those are never exercised. This implementation is a prototype, for the following reasons: - no considerations of backward compatibility and configuring an extension - no support for converting between packed-refs and reftable - no documentation - test failures when setting GIT_TEST_REFTABLE=1. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
hanwen
added a commit
to hanwen/git
that referenced
this pull request
Sep 19, 2023
This introduces the reftable backend as an alternative to the packed backend. This is an alternative to the approach which was explored in git#1215. This alternative is simpler the code, because we now longer have to worry about: - pseudorefs and the HEAD ref - worktrees - commands that blur together files and references (cherry-pick, rebase) This deviates from the spec that in Documentation/technical/reftable.txt. It might be possible to update the code such that all writes by default go to reftable directly. Then the result would be compatible with an implementation that writes only reftable (the reftable lock would still prevent races) relative to an implementation that disregards loose refs. Or, JGit could be adapted to follow this implementation. For this incremental path, the reftable format is arguably more complex than necessary, as - packed-refs doesn't support symrefs - reflogs aren't moved into reftable/ on the other hand, the code is already there, and it's well-structured and well-tested. refs/reftable-backend.c was created by cannibalizing the first version of reftable support (github PR 1215 as mentioned above). It supports reflogs and symrefs, even though those are never exercised. This implementation is a prototype, for the following reasons: - no considerations of backward compatibility and configuring an extension - no support for converting between packed-refs and reftable - no documentation - test failures when setting GIT_TEST_REFTABLE=1. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
hanwen
added a commit
to hanwen/git
that referenced
this pull request
Sep 20, 2023
This introduces the reftable backend as an alternative to the packed backend. This is an alternative to the approach which was explored in git#1215. This alternative is simpler the code, because we now longer have to worry about: - pseudorefs and the HEAD ref - worktrees - commands that blur together files and references (cherry-pick, rebase) This deviates from the spec that in Documentation/technical/reftable.txt. It might be possible to update the code such that all writes by default go to reftable directly. Then the result would be compatible with an implementation that writes only reftable (the reftable lock would still prevent races) relative to an implementation that disregards loose refs. Or, JGit could be adapted to follow this implementation. For this incremental path, the reftable format is arguably more complex than necessary, as - packed-refs doesn't support symrefs - reflogs aren't moved into reftable/ on the other hand, the code is already there, and it's well-structured and well-tested. refs/reftable-backend.c was created by cannibalizing the first version of reftable support (github PR 1215 as mentioned above). It supports reflogs and symrefs, even though those are never exercised. This implementation is a prototype, for the following reasons: - no considerations of backward compatibility and configuring an extension - no support for converting between packed-refs and reftable - no documentation - test failures when setting GIT_TEST_REFTABLE=1. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
hanwen
added a commit
to hanwen/git
that referenced
this pull request
Sep 20, 2023
This introduces the reftable backend as an alternative to the packed backend. This is an alternative to the approach which was explored in git#1215. This alternative is simpler, because we now longer have to worry about: - pseudorefs and the HEAD ref - worktrees - commands that blur together files and references (cherry-pick, rebase) This deviates from the spec that in Documentation/technical/reftable.txt. It might be possible to update the code such that all writes by default go to reftable directly. Then the result would be compatible with an implementation that writes only reftable (the reftable lock would still prevent races), but something must be done about the HEAD ref (which JGit keeps in reftable too.) Alternatively, JGit could be adapted to follow this implementation: despite the code being there for 4 years now, I haven't heard of anyone using it in production (exactly because CGit does not support it). For this incremental path, the reftable format is arguably more complex than necessary, as - packed-refs doesn't support symrefs - reflogs aren't moved into reftable on the other hand, the code is already there, and it's well-structured and well-tested. refs/reftable-backend.c was created by cannibalizing the first version of reftable support (github PR 1215 as mentioned above). It supports reflogs and symrefs, even though those features are never exercised. This implementation is a prototype, for the following reasons: - no considerations of backward compatibility and configuring an extension - no support for converting between packed-refs and reftable - no documentation - test failures when setting GIT_TEST_REFTABLE=1. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In https://lore.kernel.org/git/220203.867dab6dmp.gmgdl@evledraar.gmail.com/ Ævar considered the option of merging the reftable without 100% of tests passing. In my development branch, I am down to 26 test failures.
As input to this discussion, I'm proffering the latest version of the reftable support.