SFTP transfer crashes on paths with "%" characters #1348

Closed
natecode opened this Issue Jun 29, 2015 · 6 comments

Projects

None yet

3 participants

@natecode

When transferring files from a host with get(), an exception is triggered if any path has a "%" code in it.

Fatal error: get() encountered an exception while downloading
Underlying exception:
    %d format: a number is required, not dict

To reproduce, create a directory on the remote host with % in the file:

ssh remote 'mkdir -p top/%a; touch top/%a/%d'
mkdir out

The run the following script to try to retrieve this dir:

from fabric.api import settings, get
with settings:
   get('top/*', 'out')

The issue is in the % expansion within sftp, where it passes a dict of args before calling abspath() in sftp.py:

        local_path = os.path.abspath(escaped_path % path_vars )
@natecode

This happens in all versions of Fabric that I've tried. The only workaround I found was to disable % expansion in this code path. Rather than try to escape %'s in the input, it's probably better to move to str.format(), which is a little more easy to work with.

@bitprophet
Member

Is there a reason doing normal %-formatting escaping (using double-percents, %%, IIRC) won't suffice here?

@natecode
natecode commented Jul 4, 2015

Because when doing a recursive get() from a remote directory, you can't know what's under the directory without listing its contents first?

This can happen if there is any name with %d in it at an arbitrary depth from the root dir you're transferring. This is something Fabric should definitely handle for the user since recursively listing and then %-escaping each file entry and passing it to get() individually seems like a poor user experience.

@bspink bspink added a commit to bspink/fabric that referenced this issue Aug 5, 2015
@bspink bspink Fixes #1348: add more robust handling for % format chars in remote pa…
…th when using sftp get()
9c9e5b2
@bspink
Contributor
bspink commented Aug 5, 2015

The patch I wrote didn't change to using the format method for strings as mentioned above, as that would break functionality for anyone with any of the specially-handled formatted strings. I just updated the existing regex to be more specific and escape anything except for the aforementioned special strings.

@natecode
natecode commented Aug 5, 2015

@bspink Thank you, I tested this patch and it fixes the case I listed in the issue. I look forward to updating when this is next released.

Cheers!

@bitprophet
Member

Rolling into the PR! edit: which is #1361, no auto-link was created, thx GH

@bitprophet bitprophet closed this Apr 7, 2016
@bitprophet bitprophet added a commit that referenced this issue Apr 7, 2016
@bspink @bitprophet bspink + bitprophet Fixes #1348: add more robust handling for % format chars in remote pa…
…th when using sftp get()
e2ca3fd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment