add initial support for Git protocol v2 #1244
Conversation
Implement Git protocol version negotiation and use Git protocol v2 for fetches if supported. For now, the observable behaviour of Dulwich is equivalent regardless of protocol version, except that two new features may be used if the server supports Git protocol v2. The first feature is a reference prefix list which may be used to filter refs server-side. This can be used to reduce the size of the initial reference listing announced by the server. Reducing the size of this announcement was a major design goal for protocol v2 to avoid sending of very large announcements when a repository contains a lot of references. This feature is intended as an optimization which servers are free to ignore depending on available server-side resources. Therefore, users of Dulwich should still be prepared to filter redundant refs manually (this limitation also applies to Git itself). A new --refspec porcelain option is provided in order to test this feature on the command line. The second feature is an object filter specification, which corresponds to the --filter option of 'git clone'. This can be used to omit objects while cloning repositories. For instance, the following command will clone a given repsitory without fetching any blob objects: dulwich clone --filter blob:none --bare REPO_URL (In this example the --bare option is used because creation of a work tree would fail without any blobs present.) The test suite now enables protocol v2 and keeps passing for me.
|
Not sure what the "unexpected success" failures on Windows are about. Are there more detailed test logs available? I don't have a Windows machine to run tests on myself. |
dulwich/cli.py
Outdated
| porcelain.pull(".", from_location) | ||
| parser = argparse.ArgumentParser() | ||
| parser.add_argument('--from_location', type=str) | ||
| parser.add_argument('--refspec', type=str, nargs='*') |
There was a problem hiding this comment.
| parser.add_argument('--refspec', type=str, nargs='*') | |
| parser.add_argument('refspec', type=str, nargs='*') |
for consistency with C Git
These are spurious - I've been meaning to look into them and possibly mark them as flapping. |
Reading Git's server code suggests that the server always uses the highest mutually supported version . See determine_protocol_version_server() in Git's protocol.c. It doesn't make much sense to run dulwich compat tests against ancient versions of Git so in practice the protocol version used during dulwich tests depends entirely on what the client is requesting. Therefore we should add a configuration knob to dulwich that controls the protocol version the client will request. Git clients already support such an option: protocol.version in git-config. I will try to add such an option to dulwich as a command-line parameter, and/or environment variable, and/or via the existing git config option -- any preference? Once this parameter exists, ideally the entire compat test suite would always run twice, once with the old protocol (v0/v1) and again with protocol v2. Any tests which only make sense in one case could be skipped in the other. |
Git prints a warning and proceeds with an unfiltered clone/fetch operation. Make dulwich behave the same way, for now.
|
I am not quite sure yet how to adjust the test suite to run compat tests across both protocol versions. I will look into this next. Meanwhile, feel free to merge this PR if you are happy with the changes so far. I can always open a new PR for the test suite changes later. |
Implement Git protocol version negotiation and use Git protocol v2 for fetches if supported. For now, the observable behaviour of Dulwich is equivalent regardless of protocol version, except that two new features may be used if the server supports Git protocol v2.
The first feature is a reference prefix list which may be used to filter refs server-side. This can be used to reduce the size of the initial reference listing announced by the server. Reducing the size of this announcement was a major design goal for protocol v2 to avoid sending of very large announcements when a repository contains a lot of references. This feature is intended as an optimization which servers are free to ignore depending on available server-side resources. Therefore, users of Dulwich should still be prepared to filter redundant refs manually (this limitation also applies to Git itself).
A new --refspec porcelain option is provided in order to test this feature on the command line.
The second feature is an object filter specification, which corresponds to the --filter option of 'git clone'. This can be used to omit objects while cloning repositories. For instance, the following command will clone a given repsitory without fetching any blob objects:
dulwich clone --filter blob:none --bare REPO_URL
(In this example the --bare option is used because creation of a work tree would fail without any blobs present.)
The test suite now enables protocol v2 and keeps passing for me.