dulwich: use subprocess ssh vendor if GIT_SSH or GIT_SSH_COMMAND env vars are set

[pmrowla]

We default to using asyncssh since we cannot rely on CLI ssh always being available. But if the user is setting GIT_SSH_COMMAND themselves we can assume the provided command is valid and we should prefer that over asyncssh (since it may have settings that are not in their ~/.ssh/config)

discourse context: https://discuss.dvc.org/t/use-different-ssh-credentials/1438

[pmrowla]

Semi-related to treeverse/dvc#8681

After this is fixed using GIT_SSH_COMMAND=ssh dvc ... will force DVC to use CLI ssh instead of asyncssh for git operations which will workaround issues where asyncssh does not support certain OpenSSH configurations. (Note that this is specifically for git over SSH, this does not affect cases where users are using git over HTTP)

[dberenbaum]

@pmrowla So the idea is to use the existing Git env var? Would a typical value for GIT_SSH be ssh for this purpose? Could you provide an example of how to use it? Do you want to document it in troubleshooting or somewhere?

[pmrowla]

This is just supporting the existing dulwich support for GIT_SSH_COMMAND for git over SSH operations. In most cases users will not need to use this option at all, because git defaults to using ssh and then OpenSSH will read whatever is in their ~/.ssh/config (which is what asyncssh does as well).

But in some cases there may be specific configuration options for the particular git server you are connecting to that you don't want to save in ~/.ssh/config (whether it's particular ssh keys or some obscure set of proxy related options), like:

$ GIT_SSH_COMMAND="/usr/bin/ssh -i /path/to/some/specific/private_key" git fetch ...

I'm not sure this needs to be documented anywhere, it is really only relevant for users that are already using GIT_SSH_COMMAND for their normal git usage, and just makes it so that DVC still works correctly when the user knows they need to pass non-standard SSH settings into git

(dulwich already respects GIT_SSH_COMMAND when it is set, the issue was that DVC would still force using asyncssh with the default ~/.ssh/config instead even when GIT_SSH_COMMAND was set)

[dberenbaum]

Semi-related to iterative/dvc#8681

In this case (or others where asyncssh does not match cli ssh), would a workaround would be to explicitly set GIT_SSH[_COMMAND]=ssh to force dvc to use cli ssh? Is that something we should document in troubleshooting?

[pmrowla]

We could consider documenting it in troubleshooting, but IMO it would be better to prefer fixing things in scmrepo/asyncssh whenever there is an actual incompatibility w/OpenSSH configs (like treeverse/dvc#8681), and just suggest this as a temporary workaround when necessary

[pmrowla]

Also, just note that when suggesting this as a workaround, users probably want GIT_SSH_COMMAND and not GIT_SSH (they behave similarly but are not actually the same).

GIT_SSH_COMMAND executes the command through the user's shell, so things like $PATH and shell aliases are respected (so you can use GIT_SSH_COMMAND=ssh, which will search $PATH for the location of ssh).

GIT_SSH executes the raw command and does not use the shell (so you have to use explicit paths to the ssh command like GIT_SSH=/usr/bin/ssh)