Skip to content

fetch_refspecs: Use dulwich for ssh urls. #196

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 27, 2023
Merged

fetch_refspecs: Use dulwich for ssh urls. #196

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions src/scmrepo/git/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -327,12 +327,18 @@ def fetch_refspecs(
progress: Optional[Callable[["GitProgressEvent"], None]] = None,
**kwargs,
) -> typing.Mapping[str, SyncStatus]:
from urllib.parse import urlparse

from .credentials import get_matching_helper_commands

if "dulwich" in kwargs.get("backends", self.backends.backends) and any(
get_matching_helper_commands(url, self.dulwich.repo.get_config_stack())
):
kwargs["backends"] = ["dulwich"]
if "dulwich" in kwargs.get("backends", self.backends.backends):
credentials_helper = any(
get_matching_helper_commands(url, self.dulwich.repo.get_config_stack())
Copy link
Collaborator

@skshetry skshetry Feb 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC credential-helper is not supposed to work for ssh, only for http urls. So this should be fixed in get_matching_helper_commands() or in upstream dulwich.

Regarding ssh not working in pygit2, let's investigate/fix it separately.

Copy link
Contributor Author

@daavoo daavoo Feb 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC credential-helper is not supposed to work for ssh, only for http urls. So this should be fixed in get_matching_helper_commands() or in upstream dulwich.

Not sure I understand what you mean. The previous logic is preserved but the change is to also force dulwich if the URL is ssh

Copy link
Collaborator

@skshetry skshetry Feb 27, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_matching_helper_commands should not match anything for ssh urls, so it should not enter this block.

Please try to address comments before merging.

Copy link
Contributor Author

@daavoo daavoo Feb 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_matching_helper_commands should not match anything for ssh urls, so it should not enter this block.

Sorry, I am still not sure I follow. You mean it should be something like:

if ssh or any(get_matching_helper_commands(url, self.dulwich.repo.get_config_stack()):

)
parsed = urlparse(url)
ssh = parsed.scheme in ("git", "git+ssh", "ssh") or url.startswith("git@")
Copy link
Contributor

@pmrowla pmrowla Feb 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should maybe be handled here: https://github.com/iterative/scmrepo/blob/e8de01e07edeacae33a9552a1005152b5a153b2a/src/scmrepo/git/backend/pygit2.py#L463-L464

But this may be a different problem entirely if pygit2+SSH is not working on linux/mac. libgit2/pygit2 are supposed to support SSH everywhere except windows.

Copy link
Contributor

@pmrowla pmrowla Feb 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the issue is that while it supports SSH, libgit2 doesn't parse openssh configs

https://libgit2.org/docs/guides/authentication/

So we probably need to be using pygit2.RemoteCallbacks and loading the keyfile or user/pass ourselves (we can use asyncssh for this)

if credentials_helper or ssh:
kwargs["backends"] = ["dulwich"]

return self._fetch_refspecs(
url,
Expand Down