Skip to content

add --files-from=FILE to allow transfer of select files only #116

Merged
merged 4 commits into from May 23, 2013

2 participants

@mdomsch
s3tools member
mdomsch commented Feb 19, 2013

This solves the change of behavior introduced by processing
excludes/includes during os.walk(), where previously:

s3cmd sync --exclude='' --include='.gpg'

would walk the whole tree and transfer only the files named *.gpg.

Since the change to os.walk(), the exclude '*' matches everything, and
nothing is transferred.

This patch introduces --files-from=FILE to match rsync behaviour,
where the list of files to transfer (local to remote) is taken not
from an os.walk(), but from the explicit list in FILE.

The equivalent for remote to local, and remote to remote, is not yet
implemented.

mdomsch added some commits Feb 19, 2013
@mdomsch mdomsch add --files-from=FILE to allow transfer of select files only
This solves the change of behavior introduced by processing
excludes/includes during os.walk(), where previously:

s3cmd sync --exclude='*' --include='*.gpg'

would walk the whole tree and transfer only the files named *.gpg.

Since the change to os.walk(), the exclude '*' matches everything, and
nothing is transferred.

This patch introduces --files-from=FILE to match rsync behaviour,
where the list of files to transfer (local to remote) is taken not
from an os.walk(), but from the explicit list in FILE.

The equivalent for remote to local, and remote to remote, is not yet
implemented.
3ce5e98
@mdomsch mdomsch accept --files-from=- to read from stdin
This allows shell syntax:

find . -name \*.gpg | s3cmd sync --files-from=- src dst

to take the list of files to transfer from stdin.

Be careful, as using with a --delete option will cause files on the
remote side not listed in stdin to be deleted too.
b76c5b3
@mdomsch
s3tools member
mdomsch commented Feb 20, 2013

I added a second patch to accept '-' meaning read from stdin by request.

mdomsch added some commits Feb 20, 2013
@mdomsch mdomsch add --files-from to manpage bc547d1
@mdomsch mdomsch use --files-from only on local source (not remote, not dest)
The restriction to use --files-from on the source and not the
destination list comes from the equivalent behavior in rsync.

The restriction on not using it for remote sources is only because I
haven't figured out the best way to handle that.  That may be added in
the future.
ba412e2
@mdomsch mdomsch added a commit to mdomsch/s3cmd that referenced this pull request May 11, 2013
@mdomsch mdomsch Merge branch 'files-from' into merge, pull request #116 6a82024
@mludvig mludvig merged commit ba412e2 into s3tools:master May 23, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.