dvc exp pull UnicodeDecodeError when using ssh git backend #16
Description
When running dvc exp pull with a ssh git remote, I get:
☧hnr γmainᵪ dvc exp pull -vvv -j1 -r litis-remote pianosa 'train_gae.num_layers=37'
2021-12-13 13:50:09,697 TRACE: Namespace(cprofile=False, yappi=False, cprofile_dump=None, pdb=False, instrument=False, instrument_open=False, quiet=0, verbose=3, version=None, cd='.', cmd='
pull', force=False, pull_cache=True, dvc_remote='litis-remote', jobs=1, run_cache=False, git_remote='pianosa', experiment='train_gae.num_layers=37', func=<class 'dvc.command.experiments.Cmd
ExperimentsPull'>)
2021-12-13 13:50:09,803 DEBUG: Adding '/home/kchoi/prog/hnr/.dvc/config.local' to gitignore file.
2021-12-13 13:50:09,805 DEBUG: Adding '/home/kchoi/prog/hnr/.dvc/tmp' to gitignore file.
2021-12-13 13:50:10,345 DEBUG: git pull experiment 'pianosa' -> 'refs/exps/80/e4443848e6a61775b67834054b7bbf3f0cb982/train_gae.num_layers=37:refs/exps/80/e4443848e6a61775b67834054b7bbf3f0cb
982/train_gae.num_layers=37'
2021-12-13 13:50:11,016 ERROR: unexpected error - 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
------------------------------------------------------------
Traceback (most recent call last):
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/main.py", line 55, in main
ret = cmd.do_run()
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/command/base.py", line 45, in do_run
return self.run()
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/command/experiments.py", line 758, in run
self.repo.experiments.pull(
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/repo/experiments/__init__.py", line 1015, in pull
return pull(self.repo, *args, **kwargs)
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/repo/__init__.py", line 49, in wrapper
return f(repo, *args, **kwargs)
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/repo/scm_context.py", line 152, in run
return method(repo, *args, **kw)
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dvc/repo/experiments/pull.py", line 38, in pull
repo.scm.fetch_refspecs(
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/scmrepo/git/__init__.py", line 253, in _backend_func
return func(*args, **kwargs)
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/scmrepo/git/backend/dulwich/__init__.py", line 526, in fetch_refspecs
fetch_result = client.fetch(
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dulwich/client.py", line 531, in fetch
result = self.fetch_pack(
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dulwich/client.py", line 1055, in fetch_pack
self._handle_upload_pack_tail(
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dulwich/client.py", line 841, in _handle_upload_pack_tail
self._read_side_band64k_data(
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/dulwich/client.py", line 604, in _read_side_band64k_data
cb(pkt)
File "/home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/scmrepo/progress.py", line 53, in __call__
msg.decode("ascii").strip() if isinstance(msg, bytes) else msg
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
------------------------------------------------------------
2021-12-13 13:50:11,093 DEBUG: Adding '/home/kchoi/prog/hnr/.dvc/config.local' to gitignore file.
2021-12-13 13:50:11,094 DEBUG: Adding '/home/kchoi/prog/hnr/.dvc/tmp' to gitignore file.
2021-12-13 13:50:11,094 DEBUG: Removing '/home/kchoi/prog/.eq9Lt6MFKbjqcroSTD2bPe.tmp'
2021-12-13 13:50:11,094 DEBUG: Removing '/home/kchoi/prog/.eq9Lt6MFKbjqcroSTD2bPe.tmp'
2021-12-13 13:50:11,094 DEBUG: Removing '/home/kchoi/prog/.eq9Lt6MFKbjqcroSTD2bPe.tmp'
2021-12-13 13:50:11,095 DEBUG: Removing '/home/kchoi/prog/hnr/.dvc/../../litis-dataset/.dvc/cache/.ZCVaiQvNkiz3kGe8dUALzQ.tmp'
2021-12-13 13:50:11,096 DEBUG: Version info for developers:
DVC version: 2.9.2 (pip)
---------------------------------
Platform: Python 3.9.7 on Linux-5.15.6-200.fc35.x86_64-x86_64-with-glibc2.34
Supports:
webhdfs (fsspec = 2021.11.1),
http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.5),
ssh (sshfs = 2021.11.2)
Cache types: reflink, hardlink, symlink
Cache directory: btrfs on /dev/mapper/luks-84945a3e-93c0-48b1-9cae-683b71c03215
Caches: local
Remotes: ssh
Workspace directory: btrfs on /dev/mapper/luks-84945a3e-93c0-48b1-9cae-683b71c03215
Repo: dvc, git
Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help!
2021-12-13 13:50:11,097 DEBUG: Analytics is enabled.
2021-12-13 13:50:11,127 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmpc7fypzoo']'
2021-12-13 13:50:11,128 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmpc7fypzoo']'
When manually printing msg I get:
☧hnr γmainᵪ dvc exp pull -vvv -j1 -r litis-remote pianosa 'train_gae.num_layers=37'
2021-12-13 13:51:21,263 TRACE: Namespace(cprofile=False, yappi=False, cprofile_dump=None, pdb=False, instrument=False, instrument_open=False, quiet=0, verbose=3, version=None, cd='.', cmd='
pull', force=False, pull_cache=True, dvc_remote='litis-remote', jobs=1, run_cache=False, git_remote='pianosa', experiment='train_gae.num_layers=37', func=<class 'dvc.command.experiments.Cmd
ExperimentsPull'>)
2021-12-13 13:51:21,348 DEBUG: Adding '/home/kchoi/prog/hnr/.dvc/config.local' to gitignore file.
2021-12-13 13:51:21,350 DEBUG: Adding '/home/kchoi/prog/hnr/.dvc/tmp' to gitignore file.
2021-12-13 13:51:21,894 DEBUG: git pull experiment 'pianosa' -> 'refs/exps/80/e4443848e6a61775b67834054b7bbf3f0cb982/train_gae.num_layers=37:refs/exps/80/e4443848e6a61775b67834054b7bbf3f0cb
982/train_gae.num_layers=37'
Fetching git refs |0.00 [00:00, ?obj/s]
> /home/kchoi/mambaforge/envs/hnr/lib/python3.9/site-packages/scmrepo/progress.py(53)__call__()
-> self._reporter._parse_progress_line(
(Pdb) msg
b'D\xc3\xa9compte des objets: 10, fait.\n'
(Pdb) msg.decode('ascii')
*** UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
(Pdb) msg.decode('utf-8')
'Décompte des objets: 10, fait.\n'
(Pdb)
I believe this is because my system is localized in french, and so I'm hitting a utf-8 decoding error.
I'll make a pull request later to change the decoding from ascii to utf-8.