SFTP transfer crashes on paths with "%" characters

[natecode]

When transferring files from a host with get(), an exception is triggered if any path has a "%" code in it.

Fatal error: get() encountered an exception while downloading
Underlying exception:
    %d format: a number is required, not dict

To reproduce, create a directory on the remote host with % in the file:

ssh remote 'mkdir -p top/%a; touch top/%a/%d'
mkdir out

The run the following script to try to retrieve this dir:

from fabric.api import settings, get
with settings:
   get('top/*', 'out')

The issue is in the % expansion within sftp, where it passes a dict of args before calling abspath() in sftp.py:

        local_path = os.path.abspath(escaped_path % path_vars )

[natecode]

This happens in all versions of Fabric that I've tried. The only workaround I found was to disable % expansion in this code path. Rather than try to escape %'s in the input, it's probably better to move to str.format(), which is a little more easy to work with.

[bitprophet]

Is there a reason doing normal %-formatting escaping (using double-percents, %%, IIRC) won't suffice here?

[natecode]

Because when doing a recursive get() from a remote directory, you can't know what's under the directory without listing its contents first?

This can happen if there is any name with %d in it at an arbitrary depth from the root dir you're transferring. This is something Fabric should definitely handle for the user since recursively listing and then %-escaping each file entry and passing it to get() individually seems like a poor user experience.

[bspink]

The patch I wrote didn't change to using the format method for strings as mentioned above, as that would break functionality for anyone with any of the specially-handled formatted strings. I just updated the existing regex to be more specific and escape anything except for the aforementioned special strings.

[natecode]

@bspink Thank you, I tested this patch and it fixes the case I listed in the issue. I look forward to updating when this is next released.

Cheers!

[bitprophet]

Rolling into the PR! edit: which is #1361, no auto-link was created, thx GH