replace git command use with dulwich

[abn]

This change introduces dulwich as the git backend, instead of system git executable. Together with an LRU cache when inspecting git package, this considerable improves performance for dependency solver and reuse of source when project has git dependencies.

In cases where dulwich fails with an HTTPUnauthorized error, Poetry falls back to system provided git client as a temporary measure. This will be replaced in the future once dulwich supports git credentials.

As an effect of this, Poetry no longer relies on the subprocess git client implemented in poetry-core. This can be removed as soon as core decouples its auto-excluded files detection from git CLI's ignored file listing.

In this change, we also introduce the experimental.system-git-client configuration for users needing a escape hatch.

poetry config experimental.system-git-client true

Resolves: #4629
Resolves: #5002
Resolves: #3392
Resolves: #2475
Resolves: #5188
Resolves: #5105
Resolves: #5493
Resolves: #4152
Resolves: #4678
Resolves: #2694

Example Improvement

When updating a project with git dependency things are much faster.

$ cat pyproject.toml | grep git
poetry = {git = "https://github.com/python-poetry/poetry.git"
$ time poetry@master update --quiet

real	0m38.240s
user	1m57.819s
sys	0m6.819s
$ time poetry@local update --quiet

real	0m7.095s
user	0m3.590s
sys	0m0.334s

Testing

Using pipx

pipx install --suffix=@5428 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/5428/head'

Using a container (podman | docker)

podman run --rm -i --entrypoint bash python:3.10 <<EOF
set -xe
python -m pip install -q git+https://github.com/python-poetry/poetry.git@refs/pull/5428/head
poetry new foobar
poetry add git+https://github.com/python-poetry/poetry.git#master
poetry update
EOF

To Do

• support repositories with submodules
• if revision and local head matches, short-circuit clone

Future Improvements

• (optional) currently source clones when solving uses {cache-dir}/src instead of {virtual-env}/src as provider does not have the environment available at that time
• (future) allow clearing of source cache via cache clear command

[alxgruU]

hi, any news for this issue ? we really need this fix

Applied applicable suggestions.

[abn]

Will need to manage conflict once #5510 is merged.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.