DVC get/import failed when Git behind a proxy

[luhuiguo]

Bug Report

get/import : Name or service not known

Description

I have a situation where my computer is behind a proxy, and needs to access a Git repository outside of the proxy network. When running dvc get/import behind my proxy, my file is not downloaded and I get the following error: [Errno -2] Name or service not known.

Configure Git to use a proxy

$ git config --global http.proxy http://proxy.address.com:port/
$ git config --global https.proxy http://proxy.address.com:port/

git clone , dvc pull ... Everything is OK

But when I want to download file tracked by DVC into other workspace

$ dvc get GIT_URL PATH
Cloning REPOSITORY.git|███████████████████████████████████████████████████████████████████████████████████████| 
Compressing |234/234 [00:20,    599obj/s]ERROR: unexpected error - HTTPSConnectionPool(host='GITLAB.ADDRESS.COM', port=443): 
Max retries exceeded with url: /REPOSITORY.git/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fd9e93e8190>: Failed to resolve 'GITLAB.ADDRESS.COM' ([Errno -2] Name or service not known)")): HTTPSConnectionPool(host='GITLAB.ADDRESS.COM', port=443): Max retries exceeded with url: REPOSITORY.git/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7fd9e93e8190>: Failed to resolve 'GITLAB.ADDRESS.COM' ([Errno -2] Name or service not known)")): <urllib3.connection.HTTPSConnection object at 0x7fd9e93e8190>: Failed to resolve 'GITLAB.ADDRESS.COM' ([Errno -2] Name or service not known): [Errno -2] Name or service not known

Reproduce

dvc get GIT_URL_BEHIND_A_PROXY PATH

Expected

dvc get/import use the git proxy config

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 3.55.1 (deb)
-------------------------
Platform: Python 3.10.8 on Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.39
Subprojects:

Supports:
        azure (adlfs = 2024.7.0, knack = 0.12.0, azure-identity = 1.17.1),
        gdrive (pydrive2 = 1.20.0),
        gs (gcsfs = 2024.6.1),
        hdfs (fsspec = 2024.6.1, pyarrow = 17.0.0),
        http (aiohttp = 3.10.5, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.10.5, aiohttp-retry = 2.8.3),
        oss (ossfs = 2023.12.0),
        s3 (s3fs = 2024.6.1, boto3 = 1.35.7),
        ssh (sshfs = 2024.6.0),
        webdav (webdav4 = 0.10.0),
        webdavs (webdav4 = 0.10.0),
        webhdfs (fsspec = 2024.6.1)
Config:
        Global: /home/luhg/.config/dvc
        System: /etc/xdg/dvc
Cache types: hardlink, symlink
Cache directory: xfs on /dev/mapper/nvme-data2
Caches: local
Remotes: ssh, local
Workspace directory: xfs on /dev/mapper/nvme-data2
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/c86ae03c82a8ffc47d6317a3b6788f2d

Additional Information (if any):

[shcheklein]

@luhuiguo could you please run it with -v and share the full stack trace. Thanks.

is it the same error if you run it w/o setting up the proxy in Git?

[luhuiguo]

@luhuiguo could you please run it with -v and share the full stack trace. Thanks.

is it the same error if you run it w/o setting up the proxy in Git?

Unset the Git proxy

$ git config unset --global http.proxy
$ git config unset --global https.proxy
$ git config list
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true

Git clone failed

$ git clone <GIT_REPOSITORY_URL>
Cloning into '<DIRECTORY>'...
fatal: unable to access '<GIT_REPOSITORY_URL>': Could not resolve host: <GITLAB_HOST>

DVC get failed

$ dvc get -v <GIT_REPOSITORY_URL> <PATH>
2024-09-23 10:45:53,687 DEBUG: v3.55.2 (deb), CPython 3.10.8 on Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.39
2024-09-23 10:45:53,687 DEBUG: command: get -v <GIT_REPOSITORY_URL> <PATH>
2024-09-23 10:45:53,946 DEBUG: Creating external repo <GIT_REPOSITORY_URL>@None
2024-09-23 10:45:53,946 DEBUG: erepo: git clone '<GIT_REPOSITORY_URL>' to a temporary dir
2024-09-23 10:46:18,913 ERROR: failed to get '<PATH>' - SCM error: Failed to clone repo '<GIT_REPOSITORY_URL>' to '/tmp/tmpibarss8odvc-clone': HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f0967118070>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)")): HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f0967118070>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)")): <urllib3.connection.HTTPSConnection object at 0x7f0967118070>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known): [Errno -2] Name or service not known
Traceback (most recent call last):
  File "urllib3/connection.py", line 196, in _new_conn
  File "urllib3/util/connection.py", line 60, in create_connection
  File "socket.py", line 955, in getaddrinfo
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 789, in urlopen
  File "urllib3/connectionpool.py", line 490, in _make_request
  File "urllib3/connectionpool.py", line 466, in _make_request
  File "urllib3/connectionpool.py", line 1095, in _validate_conn
  File "urllib3/connection.py", line 615, in connect
  File "urllib3/connection.py", line 203, in _new_conn
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7f0967118070>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dulwich/client.py", line 2290, in _http_request
  File "urllib3/_request_methods.py", line 136, in request
  File "urllib3/_request_methods.py", line 183, in request_encode_url
  File "urllib3/poolmanager.py", line 443, in urlopen
  File "urllib3/connectionpool.py", line 873, in urlopen
  File "urllib3/connectionpool.py", line 873, in urlopen
  File "urllib3/connectionpool.py", line 873, in urlopen
  File "urllib3/connectionpool.py", line 843, in urlopen
  File "urllib3/util/retry.py", line 519, in increment
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f0967118070>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)"))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "scmrepo/git/backend/dulwich/__init__.py", line 260, in clone
  File "dulwich/porcelain.py", line 546, in clone
  File "dulwich/client.py", line 752, in clone
  File "dulwich/client.py", line 840, in fetch
  File "dulwich/client.py", line 2157, in fetch_pack
  File "dulwich/client.py", line 2013, in _discover_references
  File "scmrepo/git/backend/dulwich/client.py", line 50, in _http_request
  File "dulwich/client.py", line 2298, in _http_request
dulwich.errors.GitProtocolError: HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7f0967118070>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)"))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dvc/scm.py", line 150, in clone
  File "scmrepo/git/__init__.py", line 154, in clone
  File "scmrepo/git/backend/dulwich/__init__.py", line 268, in clone
scmrepo.exceptions.CloneError: Failed to clone repo '<GIT_REPOSITORY_URL>' to '/tmp/tmpibarss8odvc-clone'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dvc/commands/get.py", line 37, in _get_file_from_repo
  File "dvc/repo/get.py", line 45, in get
  File "dvc/repo/__init__.py", line 302, in open
  File "dvc/repo/open_repo.py", line 60, in open_repo
  File "contextlib.py", line 79, in inner
  File "dvc/repo/open_repo.py", line 23, in _external_repo
  File "dvc/repo/open_repo.py", line 134, in _cached_clone
  File "funcy/decorators.py", line 47, in wrapper
  File "funcy/flow.py", line 246, in wrap_with
  File "funcy/decorators.py", line 68, in __call__
  File "dvc/repo/open_repo.py", line 198, in _clone_default_branch
  File "dvc/scm.py", line 155, in clone
dvc.scm.CloneError: SCM error

2024-09-23 10:46:18,950 DEBUG: Analytics is enabled.
2024-09-23 10:46:18,952 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmpggwvzttp', '-v']
2024-09-23 10:46:18,962 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmpggwvzttp', '-v'] with pid 201

Configure Git to use a proxy

$ git config --global http.proxy http://10.3.12.8:3128
$ git config --global https.proxy http://10.3.12.8:3128
$ git config list
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
http.proxy=http://10.3.12.8:3128
https.proxy=http://10.3.12.8:3128

GIT clone successful

$ git clone <GIT_REPOSITORY_URL>
Cloning into '<DIRECTORY>'...
remote: Enumerating objects: 166, done.
remote: Counting objects: 100% (133/133), done.
remote: Compressing objects: 100% (119/119), done.
remote: Total 166 (delta 42), reused 0 (delta 0), pack-reused 33
Receiving objects: 100% (166/166), 11.08 MiB | 874.00 KiB/s, done.
Resolving deltas: 100% (48/48), done.

DVC get failed

$ dvc get -v <GIT_REPOSITORY_URL> <PATH>
2024-09-23 10:40:27,336 DEBUG: v3.55.2 (deb), CPython 3.10.8 on Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.39
2024-09-23 10:40:27,336 DEBUG: command: get -v <GIT_REPOSITORY_URL> <PATH>
2024-09-23 10:40:27,486 DEBUG: Creating external repo <GIT_REPOSITORY_URL>@None
2024-09-23 10:40:27,486 DEBUG: erepo: git clone '<GIT_REPOSITORY_URL>' to a temporary dir
2024-09-23 10:41:06,026 ERROR: unexpected error - HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7ff140e4c040>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)")): HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7ff140e4c040>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)")): <urllib3.connection.HTTPSConnection object at 0x7ff140e4c040>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known): [Errno -2] Name or service not known
Traceback (most recent call last):
  File "urllib3/connection.py", line 196, in _new_conn
  File "urllib3/util/connection.py", line 60, in create_connection
  File "socket.py", line 955, in getaddrinfo
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 789, in urlopen
  File "urllib3/connectionpool.py", line 490, in _make_request
  File "urllib3/connectionpool.py", line 466, in _make_request
  File "urllib3/connectionpool.py", line 1095, in _validate_conn
  File "urllib3/connection.py", line 615, in connect
  File "urllib3/connection.py", line 203, in _new_conn
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7ff140e4c040>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dulwich/client.py", line 2290, in _http_request
  File "urllib3/_request_methods.py", line 136, in request
  File "urllib3/_request_methods.py", line 183, in request_encode_url
  File "urllib3/poolmanager.py", line 443, in urlopen
  File "urllib3/connectionpool.py", line 873, in urlopen
  File "urllib3/connectionpool.py", line 873, in urlopen
  File "urllib3/connectionpool.py", line 873, in urlopen
  File "urllib3/connectionpool.py", line 843, in urlopen
  File "urllib3/util/retry.py", line 519, in increment
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7ff140e4c040>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)"))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "dvc/cli/__init__.py", line 211, in main
  File "dvc/cli/command.py", line 41, in do_run
  File "dvc/commands/get.py", line 30, in run
  File "dvc/commands/get.py", line 37, in _get_file_from_repo
  File "dvc/repo/get.py", line 45, in get
  File "dvc/repo/__init__.py", line 302, in open
  File "dvc/repo/open_repo.py", line 60, in open_repo
  File "contextlib.py", line 79, in inner
  File "dvc/repo/open_repo.py", line 23, in _external_repo
  File "dvc/repo/open_repo.py", line 134, in _cached_clone
  File "funcy/decorators.py", line 47, in wrapper
  File "funcy/flow.py", line 246, in wrap_with
  File "funcy/decorators.py", line 68, in __call__
  File "dvc/repo/open_repo.py", line 198, in _clone_default_branch
  File "dvc/scm.py", line 152, in clone
  File "dvc/repo/experiments/utils.py", line 275, in fetch_all_exps
  File "dvc/repo/experiments/utils.py", line 275, in <listcomp>
  File "dvc/repo/experiments/utils.py", line 119, in iter_remote_refs
  File "scmrepo/git/backend/dulwich/__init__.py", line 590, in iter_remote_refs
  File "dulwich/client.py", line 2208, in get_refs
  File "dulwich/client.py", line 2013, in _discover_references
  File "scmrepo/git/backend/dulwich/client.py", line 50, in _http_request
  File "dulwich/client.py", line 2298, in _http_request
dulwich.errors.GitProtocolError: HTTPSConnectionPool(host='<GITLAB_HOST>', port=443): Max retries exceeded with url: /<REPOSITORY>/info/refs?service=git-upload-pack (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7ff140e4c040>: Failed to resolve '<GITLAB_HOST>' ([Errno -2] Name or service not known)"))

2024-09-23 10:41:06,128 DEBUG: Version info for developers:
DVC version: 3.55.2 (deb)
-------------------------
Platform: Python 3.10.8 on Linux-3.10.0-1160.el7.x86_64-x86_64-with-glibc2.39
Subprojects:

Supports:
        azure (adlfs = 2024.7.0, knack = 0.12.0, azure-identity = 1.17.1),
        gdrive (pydrive2 = 1.20.0),
        gs (gcsfs = 2024.6.1),
        hdfs (fsspec = 2024.6.1, pyarrow = 17.0.0),
        http (aiohttp = 3.10.5, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.10.5, aiohttp-retry = 2.8.3),
        oss (ossfs = 2023.12.0),
        s3 (s3fs = 2024.6.1, boto3 = 1.35.7),
        ssh (sshfs = 2024.6.0),
        webdav (webdav4 = 0.10.0),
        webdavs (webdav4 = 0.10.0),
        webhdfs (fsspec = 2024.6.1)
Config:
        Global: /home/luhg/.config/dvc
        System: /etc/xdg/dvc

Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2024-09-23 10:41:06,145 DEBUG: Analytics is enabled.
2024-09-23 10:41:06,147 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmp8y57xigq', '-v']
2024-09-23 10:41:06,155 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmp8y57xigq', '-v'] with pid 174

[shcheklein]

Okay, probably it should be fixed on the https://github.com/jelmer/dulwich side.

Is there a way for you to run with HTTP_PROXY and HTTPS_PROXY env vars set? I think dulwich supports those.

Probably you can create an alias for now for dvc get to include those env vars. Would that work for you for now?

[shcheklein]

I've created an issue upstream jelmer/dulwich#1368

[shcheklein]

Okay, seems it (the proxy via global Git config) should be supported. I've tried to do this:

(.venv) √ Projects/test-dvc-get % dvc import https://github.com/iterative/dataset-registry get-started/data.xml -o data/data.xml
ERROR: failed to import 'get-started/data.xml' from 'https://github.com/iterative/dataset-registry'. - stage working dir '/Users/ivan/Projects/test-dvc-get/data' does not exist
(.venv) ?1 Projects/test-dvc-get % mkdir data
(.venv) √ Projects/test-dvc-get % dvc import https://github.com/iterative/dataset-registry get-started/data.xml -o data/data.xml
Importing 'get-started/data.xml (https://github.com/iterative/dataset-registry)' -> 'data/data.xml'
ERROR: failed to import 'get-started/data.xml' - SCM error: Failed to clone repo 'https://github.com/iterative/dataset-registry' to '/var/folders/8f/fbysfztx1mb953_gpwl477p80000gn/T/tmphwwt1qxrdvc-clone': HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /iterative/dataset-registry/info/refs?service=git-upload-pack (Caused by ProxyError('Unable to connect to proxy', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x105ce6330>: Failed to establish a new connection: [Errno 51] Network is unreachable'))): HTTPSConnectionPool(host='github.com', port=443): Max retries exceeded with url: /iterative/dataset-registry/info/refs?service=git-upload-pack (Caused by ProxyError('Unable to connect to proxy', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x105ce6330>: Failed to establish a new connection: [Errno 51] Network is unreachable'))): ('Unable to connect to proxy', NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x105ce6330>: Failed to establish a new connection: [Errno 51] Network is unreachable')): <urllib3.connection.HTTPSConnection object at 0x105ce6330>: Failed to establish a new connection: [Errno 51] Network is unreachable: [Errno 51] Network is unreachable

after running:

$ git config --global http.proxy http://10.3.12.8:3128
$ git config --global https.proxy http://10.3.12.8:3128

So, it's trying to connect to proxy (and fails).

We need a simpler way to reproduce this to research - e.g. some way to run a local proxy to do some experiments.