Introduce Refdb type #982

ddevault · 2020-02-28T16:43:26Z

TODO:

Future work:

Iterating over a refdb_backend from Python
First-class reflog support
Transactional refdb updates

src/refdb.c

src/refdb_backend.c

jdavid

Thanks for your work! It's quite a lot.

src/refdb_backend.c

jdavid · 2020-03-07T09:32:03Z

Looking at the API level.

I see there're a number of ways to get a backend:

refdb = repo.refdb       # wraps git_repository_refdb
refdb = Refdb(repo)      # wraps git_refdb_new
refdb = Refdb.open(repo) # wraps git_refdb_open

For consistency and to be explicit please change Refdb(repo) to Refdb.new(repo)

ddevault · 2020-03-09T15:27:16Z

For consistency and to be explicit please change Refdb(repo) to Refdb.new(repo)

Done.

ddevault · 2020-03-09T18:39:27Z

Related libgit2 bug:

libgit2/libgit2#5449

This prevents us from being able to create a RefdbFsBackend and use it to create a Refdb from scratch. Other than that, the functionality of this patch is complete. I'll add tests in a day or two.

jdavid

Only a couple of comments for the 1st commit. Thanks!

src/odb_backend.c

ddevault · 2020-03-17T15:01:42Z

Addressed comments on odb fixes. Still need to write tests and verify that everything works. I ran into some issues in a shallow test the other day.

ddevault · 2020-03-17T15:06:25Z

Note: Windows CI is failing due to lack of fnmatch support, which I used to implement globbing.

Options:

Drop in musl libc's implementation, ~300 LoC, MIT licensed
Add a code path for Windows which doesn't support globbing
Implement globbing some other way

ddevault · 2020-03-19T20:00:56Z

✓ Tests

Code-wise, this should be ready for review. The question of what to do about fnmatch on Windows remains, and I need to write up some docs and rebase everything.

ddevault · 2020-03-19T20:10:54Z

Oh, note that this requires the following libgit2 change before it'll work: libgit2/libgit2#5456

jdavid · 2020-03-21T07:58:23Z

Regarding fnmatch does libgit2 provide an implementation for this? Apparently they did, but replaced it by wildmatch, see libgit2/libgit2#5110 So I think we should use the same as libgit2.

ddevault · 2020-03-21T14:04:50Z

libgit2/libgit2#5458

I'll just vendor it in the meanwhile (on Monday).

ddevault · 2020-03-23T16:24:55Z

✓ fnmatch -> wildmatch

✓ Questions answered from libgit2 about the semantics of refdb_backend.write

Still needs docs but otherwise I think this is good to go.

jdavid · 2020-03-24T09:21:15Z

The test failures are because of libgit2/libgit2#5456 ?
I see this has been merged to master but not to maint/v0.99, then please update the CI files to use master, so the tests pass.

ddevault · 2020-03-24T17:27:19Z

Got all of the CI pointing at libgit2 master and the tests are passing.

ddevault · 2020-03-25T13:25:50Z

I have rigged up the documentation as well.

jdavid · 2020-03-26T16:06:30Z

Cool, now we have to wait for a libgit2 release with libgit2/libgit2#5456
Hopefully it won't take long.

Prevents freeing an uninitialized pointer later on.

Custom refdb backends will need to be able to construct References, and this change adds the necessary constructors. However, the resulting Reference objects are fragile, and should not be used for much else. This is noted in the docstring.

Currently this allows you to manipulate refdbs in a vacuum. A later commit will introduce RefdbBackend, then allow you to create a Repository given a refdb backend and an odb backend. The refdb is more or less a write-only interface from libgit2, so it's not really possible to verify that the functions added here have affected libgit2's state in the desired way. For this reason, tests are not included for this type.

This creates a git_repository with no odb and no refdb.

This is imported from libgit2

ddevault · 2020-03-30T20:13:55Z

I pushed a little bit of clean up in the course of implementing this:

https://git.sr.ht/~sircmpwn/git.sr.ht/commit/a3fe57b6844709eac64fdca60370e5c227979fd3

It works 😁

jdavid · 2020-04-01T18:00:36Z

src/refdb_backend.c

@@ -198,7 +198,6 @@ pygit2_refdb_backend_lookup(git_reference **out,

    *out = result->reference;
 out:
-    Py_DECREF(result);


Why this? Doesn't lookup(..) return a new reference?

Apparently not. This was causing double frees.

ddevault · 2020-04-02T18:46:02Z

Thanks!

jdavid · 2020-04-03T07:42:32Z

Thank you!

ddevault force-pushed the refdb branch 2 times, most recently from 7c93e2b to 4ff6185 Compare February 28, 2020 21:45

jdavid requested changes Feb 29, 2020

View reviewed changes

ddevault force-pushed the refdb branch from 4ff6185 to cec6867 Compare February 29, 2020 15:38

jdavid requested changes Mar 3, 2020

View reviewed changes

src/refdb_backend.c Outdated Show resolved Hide resolved

src/refdb_backend.c Show resolved Hide resolved

src/refdb_backend.c Outdated Show resolved Hide resolved

ddevault force-pushed the refdb branch 2 times, most recently from 4169c01 to a4cbb58 Compare March 9, 2020 15:26

ddevault force-pushed the refdb branch from a848ccb to f04d231 Compare March 9, 2020 18:07

jdavid requested changes Mar 16, 2020

View reviewed changes

src/odb_backend.c Outdated Show resolved Hide resolved

src/odb_backend.c Outdated Show resolved Hide resolved

src/odb_backend.c Outdated Show resolved Hide resolved

ddevault force-pushed the refdb branch from e12d742 to d43546a Compare March 17, 2020 14:57

ddevault force-pushed the refdb branch from d43546a to 098ca48 Compare March 19, 2020 20:00

ddevault force-pushed the refdb branch from 098ca48 to 9f1ed86 Compare March 23, 2020 16:23

ddevault changed the title ~~[WIP] Introduce Refdb type~~ Introduce Refdb type Mar 23, 2020

ddevault force-pushed the refdb branch 3 times, most recently from fdf0244 to 2406bf4 Compare March 24, 2020 13:54

jdavid approved these changes Mar 25, 2020

View reviewed changes

ddevault added 13 commits March 30, 2020 14:51

OdbBackend: fix memory leaks

0c93a62

wrap_reference: set repo even if NULL

de81110

Prevents freeing an uninitialized pointer later on.

[WIP] Introduce RefdbBackend, RefdbFsBackend

f22debb

Repository: add zero-arg constructor

c0f9ae5

This creates a git_repository with no odb and no refdb.

Repository: add set_odb, set_refdb

ec96d17

Refdb: set backend at runtime

6300169

RefdbBackend: Add tests and flush out a few bugs

57fd016

Repository: add test case for from-scratch repo

a531f13

refdb: vendor and use wildmatch over fnmatch

fcffd14

This is imported from libgit2

Switch CI to use libgit2 master

434c13f

Add documentation for custom refdb/odb backends

f445fe9

ddevault force-pushed the refdb branch from 258bf0e to f445fe9 Compare March 30, 2020 18:51

refdb_backend: fix reflog support shim

e012bb5

jdavid reviewed Apr 1, 2020

View reviewed changes

jdavid merged commit e012bb5 into libgit2:master Apr 2, 2020

This was referenced Apr 4, 2020

Custom Backends? #76

Closed

In-memory repository #678

Closed

How to use custom backends? #866

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Refdb type #982

Introduce Refdb type #982

ddevault commented Feb 28, 2020 •

edited

Loading

jdavid left a comment

jdavid commented Mar 7, 2020

ddevault commented Mar 9, 2020

ddevault commented Mar 9, 2020

jdavid left a comment

ddevault commented Mar 17, 2020

ddevault commented Mar 17, 2020

ddevault commented Mar 19, 2020 •

edited

Loading

ddevault commented Mar 19, 2020

jdavid commented Mar 21, 2020

ddevault commented Mar 21, 2020

ddevault commented Mar 23, 2020

jdavid commented Mar 24, 2020

ddevault commented Mar 24, 2020

ddevault commented Mar 25, 2020

jdavid commented Mar 26, 2020

ddevault commented Mar 30, 2020

jdavid Apr 1, 2020

ddevault Apr 1, 2020

ddevault commented Apr 2, 2020

jdavid commented Apr 3, 2020

Introduce Refdb type #982

Introduce Refdb type #982

Conversation

ddevault commented Feb 28, 2020 • edited Loading

jdavid left a comment

Choose a reason for hiding this comment

jdavid commented Mar 7, 2020

ddevault commented Mar 9, 2020

ddevault commented Mar 9, 2020

jdavid left a comment

Choose a reason for hiding this comment

ddevault commented Mar 17, 2020

ddevault commented Mar 17, 2020

ddevault commented Mar 19, 2020 • edited Loading

ddevault commented Mar 19, 2020

jdavid commented Mar 21, 2020

ddevault commented Mar 21, 2020

ddevault commented Mar 23, 2020

jdavid commented Mar 24, 2020

ddevault commented Mar 24, 2020

ddevault commented Mar 25, 2020

jdavid commented Mar 26, 2020

ddevault commented Mar 30, 2020

jdavid Apr 1, 2020

Choose a reason for hiding this comment

ddevault Apr 1, 2020

Choose a reason for hiding this comment

ddevault commented Apr 2, 2020

jdavid commented Apr 3, 2020

ddevault commented Feb 28, 2020 •

edited

Loading

ddevault commented Mar 19, 2020 •

edited

Loading