Skip to content

ssh: error while import-url with special character #4628

@Gabrielllopes

Description

@Gabrielllopes

Bug Report

Problem while download files with special characters.

Please provide information about your setup

DVC version: 1.7.9 
---------------------------------
Platform: Python 3.8.3 on Linux-5.4.0-48-generic-x86_64-with-glibc2.10
Supports: gdrive, gs, http, https, ssh
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/sda5
Workspace directory: ext4 on /dev/sda5
Repo: dvc, git

Error log :

2020-09-28 15:53:35,511 ERROR: failed to import ssh://user@1.2.3.4/home/data/cana/ds30. You could also try downloading it manually, and adding it with `dvc add`. - ssh command 'md5sum /home/data/cana/ds30/cana-mucuna/class35_e2545053-f2c5-4108-9042-67244a94e267_p_['cana']_o_['cana', 'mucuna'].jpg' finished with non-zero return code 1': md5sum: '/home/data/cana/ds30/cana-mucuna/class35_e2545053-f2c5-4108-9042-67244a94e267_p_[cana]_o_[cana,': No such file or directory
md5sum: mucuna].jpg: No such file or directory

------------------------------------------------------------
Traceback (most recent call last):
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/command/imp_url.py", line 14, in run
    self.repo.imp_url(
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/repo/__init__.py", line 51, in wrapper
    return f(repo, *args, **kwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/repo/scm_context.py", line 4, in run
    result = method(repo, *args, **kw)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/repo/imp_url.py", line 54, in imp_url
    stage.run()
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/funcy/decorators.py", line 39, in wrapper
    return deco(call, *dargs, **dkwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/stage/decorators.py", line 36, in rwlocked
    return call()
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/funcy/decorators.py", line 60, in __call__
    return self._func(*self._args, **self._kwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/stage/__init__.py", line 429, in run
    sync_import(self, dry, force)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/stage/imports.py", line 29, in sync_import
    stage.save_deps()
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/stage/__init__.py", line 392, in save_deps
    dep.save()
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/output/base.py", line 268, in save
    self.hash_info = self.get_hash()
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/output/base.py", line 178, in get_hash
    return self.tree.get_hash(self.path_info)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/base.py", line 263, in get_hash
    hash_info = self.get_dir_hash(path_info, **kwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/base.py", line 330, in get_dir_hash
    dir_info = self._collect_dir(path_info, **kwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/base.py", line 310, in _collect_dir
    new_hashes = self._calculate_hashes(not_in_state)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/base.py", line 296, in _calculate_hashes
    return dict(zip(file_infos, hashes))
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/base.py", line 295, in <genexpr>
    hashes = (hi.value for hi in executor.map(worker, file_infos))
  File "/home/gabriel/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 611, in result_iterator
    yield fs.pop().result()
  File "/home/gabriel/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/home/gabriel/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 388, in __get_result
    raise self._exception
  File "/home/gabriel/anaconda3/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/progress.py", line 126, in wrapped
    res = fn(*args, **kwargs)
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/ssh/__init__.py", line 242, in get_file_hash
    return HashInfo(self.PARAM_CHECKSUM, ssh.md5(path_info.path))
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/ssh/connection.py", line 295, in md5
    md5 = self.execute("md5sum " + path).split()[0]
  File "/home/gabriel/anaconda3/lib/python3.8/site-packages/dvc/tree/ssh/connection.py", line 276, in execute
    raise RemoteCmdError("ssh", cmd, ret, err)
dvc.tree.base.RemoteCmdError: ssh command 'md5sum /home/data/cana/ds30/cana-mucuna/class35_e2545053-f2c5-4108-9042-67244a94e267_p_['cana']_o_['cana', 'mucuna'].jpg' finished with non-zero return code 1': md5sum: '/home/data/cana/ds30/cana-mucuna/class35_e2545053-f2c5-4108-9042-67244a94e267_p_[cana]_o_[cana,': No such file or directory
md5sum: mucuna].jpg: No such file or directory

------------------------------------------------------------
2020-09-28 15:53:35,520 DEBUG: Analytics is enabled.
2020-09-28 15:53:35,605 DEBUG: Trying to spawn '['daemon', '-q', 'analytics', '/tmp/tmp4x4p60hi']'
2020-09-28 15:53:35,608 DEBUG: Spawned '['daemon', '-q', 'analytics', '/tmp/tmp4x4p60hi']'

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugDid we break something?help wantedp2-mediumMedium priority, should be done, but less importanttriageNeeds to be triaged

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions