History.txt

=== 1.1.42 (2018-08-21)

* Update gem dependencies: aws-sdk 1.67.0, nokogiri 1.8.4

=== 1.1.41 (2017-11-30)

* Fixed:  s3rm now incrementally deletes objects (per new --batch size parameter; defaults to 10,000)

=== 1.1.40 (2015-12-21)

* Fixed:  Don't load Ruby 1.8.x compatibility library for Ruby 2.1.8 (#18) [Chris Mear]

=== 1.1.39 (2015-03-02)

* Upgrade gem dependencies:  aws-sdk 1.63.0, nokogiri 1.6.6.2.

=== 1.1.38 (2014-06-11)

* Added new --storage-class option to `s3ls` that displays the backing
  storage class (STANDARD/GLACIER).

* Upgrade to aws-sdk ~> 1.36.2

=== 1.1.37 (2013-08-26)

* Fixed regression introduced in v1.1.33.

=== 1.1.36 (2013-08-26)

* Don't use; regression introduced in v1.1.33.

=== 1.1.35 (2013-08-26)

* Don't use; regression introduced in v1.1.33.

=== 1.1.34 (2013-08-26)

* Don't use; regression introduced in v1.1.33.

=== 1.1.33 (2013-08-26)

* Added:  New --mkdir option on s3cp allowing to recreate the source directory
          structure from any point in the source path(s).

          The --mkdir option accepts a regular expression to match against the
          source path(s) and will recreate the source directory structure from
          the match point.

          Assuming the following bucket content,

          s3://bucket/foo/bar/1
          s3://bucket/foo/bar/2
          s3://bucket/foo/baz/3
          s3://bucket/foo/baz/4

          then running:

          % s3cp --recursive --mkdir /foo/  s3://bucket/foo/bar .

          would create the following local directories and files:

          ./foo/bar/1
          ./foo/bar/2

          If the regular expression contains a capturing group, the destination
          path(s) will be relative to the group's starting position instead of
          the regular expression's matching position:

          % s3cp --recursive --mkdir "/foo/(.)"  s3://bucket/foo/ .

          would create the following local content:

          ./bar/1
          ./bar/2
          ./baz/3
          ./baz/4

          Note: The --mkdir option is currently only valid when copying from S3
          to the local filesystem.

=== 1.1.32 (2013-08-26)

* Bad release (released from dirty repository checkout). Do not use.

=== 1.1.31 (2013-05-16)

* Fixed:  Passing ACLs to s3cp using --header 'x-amz-acl: public-read' was broken
          since 1.1.24.

=== 1.1.30 (2013-05-15)

* Changed:  Improved error handling -- all commands now exit with non-zero exit
            code on error.  [Alexis Midon / alexism@github / Pull request #11]

* Changed:  s3cp properly fails   when source key is not specified or does not exist.
            [Alexis Midon / alexism@github / Pull request #11]

=== 1.1.29 (2013-05-15)

* Added:  s3buckets command may now be used to delete buckets and enable/suspend
          versioning on bucket.

          % s3buckets --delete my-bucket

          % s3buckets --enable-versioning my-bucket

          % s3buckets --suspend-versioning my-bucket

=== 1.1.28 (2013-05-07)

* Added:  s3buckets command may now be used to create buckets.

          % S3CP_REGION="eu-west-1" s3buckets --create super-bucket --acl public_read

=== 1.1.27 (2013-04-04)

* Fixed:  Fixed [NoMethodError] undefined method 'exist?' in s3cp when using
          --no-overwrite option (residual bug from RightAWS -> AWS SDK migration).
          s3cp now prints message on STDERR when skipping files that already exists.

=== 1.1.26 (2013-03-21)

* Fixed:  Prevent crash if $terminal.output_rows can't be determined in s3ls/s3tree.
          Depending on the platform, a warning such as:  "stty: standard input:
          Inappropriate ioctl for device" may be displayed instead.

=== 1.1.25 (2013-03-19)

* Changed:  Don't display progress bar during upload to S3 unless $stdout.isatty

=== 1.1.24 (2013-03-15)

* Changed:  Commands now print a succint error code + message instead of
            Ruby stacktrace.  (Stacktraces still available using --debug flag)

=== 1.1.23 (2013-03-15)

* Added:   s3cat now accepts --head, --tail and --range arguments.

              # Display only first 8KB of file
              % s3cat --head 8k bucket:path/to/file

              # Display only last 4096 bytes of file
              % s3cat --tail 4096 bucket:path/to/file

              # Download file content between positions 4096 - 12288 (exclusive)
              % s3cat --range 4k-12k bucket:path/to/file > file_fragment

              # Download content after position 12288 (inclusive)
              % s3cat --range 12k- bucket:path/to/file > file_fragment

              # Download content before position 12288 (exclusive)
              % s3cat --range -12k bucket:path/to/file > file_fragment

=== 1.1.22 (2013-03-07)

* Changed: s3cp now adds a new "md5" custom metadata to any uploaded file
           using multi-part protocol with --checksum option to allow for
           proper content integrity checking and synchronization (--sync)
           on (large / multi-part) files since the etag metadata is only a
           valid MD5 value for files uploaded as a single part.

           As a reminder, --checksum is on by default and multi-part upload
           kicks in by default for files > 16MB.  These defaults are
           configurable through the command-line, environment variables or
           $HOME/.s3cp.

           On the command line,

             # Turn things off
             % export S3CP_CHECKSUM=false
             % export S3CP_MULTIPART=false

             # Configure multipart threshold for files > 64MB
             % export S3CP_MULTIPART=67108864

           In your in $HOME/.s3cp,

             ENV["S3CP_MULTIPART"] = "false"

             # Use multi-part upload only for files > 128MB
             AWS.config.s3_multipart_threshold = (128 * 1024 * 1024)


=== 1.1.21 (2013-03-05)

* Fixed:  s3cp --sync would not correctly copy a file from S3 to local file
          system if the file didn't already exist *and* it had invalid
          MD5 checksum metadata in S3 (from a multi-part upload).


=== 1.1.20 (2013-02-26)

* Added:  "--header" command-line option to s3cp, s3up, s3mod.

          This option allows passing individual headers including ones with
          a comma in the value, e.g.:

            s3cp --header 'Cache-Control: max-age=604800,must-revalidate'

          That otherwise would have been interpreted as two headers by the
          "--headers" option.   You may use several "--header" arguments
          if needed.

          (Be warned that at the time of this writing, the 's3stat' command
          does not display all headers/metadata on the S3 object.  This is
          a bug in the aws-sdk gem.   You may use the AWS console to get an
          accurate list of all object headers and metadata.)

=== 1.1.19 (2013-02-06)

* Fixed:    Error in `s3lifecycle` if rule prefix is nil.

=== 1.1.18 (2013-02-05)

* Added:    Add new `s3lifecycle` command to manage object transtion rules.

    A few examples:

    # Show lifecycle rules for bucket
    % s3lifecycle s3://bucket

    # Add expiration rule (expire after 180 days)
    % s3lifecycle --name "Optional name" --expire 180 s3://bucket/below/some/path/

    # Transition objects to Glacier after 180 days
    % s3lifecycle --glacier 180 s3://bucket/below/some/path/

    # Transition objects to Glacier after 2013-03-29
    % s3lifecycle --glacier 2013-03-29 s3://bucket/below/some/path/

    # Enable rule by path (you can use --name to enable by name)
    % s3lifecycle --enable s3://bucket/below/some/path/

    # Disable rule by name
    % s3lifecycle --disable --name "Some rule" s3://bucket

=== 1.1.17 (2013-02-05)

* Fixed:    Speed-up `s3du` (same dealio as s3ls -l from version 1.1.14)

=== 1.1.16 (2013-01-31)

* Fixed:    s3ls paging (with > 1000 keys) was broken (bug introduced in s3cp 1.1.14)

=== 1.1.15 (2013-01-29)

* Fixed:    s3ls would raise undefined method '[]' for nil:NilClass if given
            path didn't exist (bug introduced in s3cp 1.1.14)

=== 1.1.14 (2013-01-27)

* Changed: s3cp now adds source file basename to destination path if destination
           path ends with '/'.

           e.g.  s3cp s3://bucket/path/to/file.ext s3://bucket/some/other/path/
                 will copy the file to s3://bucket/some/other/path/file.ext

* Changed: Speed-up s3ls -l (long-listing format with date + size). Performance
           has regressed since migrating to AWS SDK (from right-aws) since
           high-level AWS SDK API doesn't support memoization of S3 objects --
           back to using low-level API.

=== 1.1.13 (2013-01-27)

* Added:   s3stat --acl option to display associated access-control list XML document.

* Added:   s3mod now accepts multiple (wildcard) paths and supports setting both
           --headers and --acl. (Denis Arh).

* Fixed:   Beter metadata display in s3stat.

=== 1.1.12 (2013-01-22)

* Added:   Support for 'Content-Disposition' header (Denis Arh).

=== 1.1.11 (2012-12-13)

* Added:   Support for requests with 'requester-pays' header (experimental).
           To use, set environment variable S3CP_REQUESTER_PAYS to "1".
           Details @ http://docs.amazonwebservices.com/AmazonS3/latest/dev/RequesterPaysBuckets.html
           and http://docs.amazonwebservices.com/AmazonS3/latest/dev/ObjectsinRequesterPaysBuckets.html

=== 1.1.10 (2012-12-13)

* Added:   New `s3edit` command for easy editing of files.
           Basically s3cp file locally, launch $EDITOR and s3cp file back to S3.

=== 1.1.9 (2012-11-27)

* Added:   New `s3tree` command similar to Unix `tree` command.
* Changed: Correctly display 'Move' text instead of 'Copy' when moving files.

=== 1.1.8 (2012-10-29)

* Changed:  Relaxed highline dependency requirement from ~> 1.5.1 to >= 1.5.1
            (for better compatibility with other gems, e.g., capistrano)

=== 1.1.7 (2012-10-17)

* Fixed:  s3cp -r (recursive) would fail with "uninitialized constant FileUtils (NameError)"
          if destination directory didn't exist.

=== 1.1.6 (2012-10-09)

* Fixed:  Potential error: utils.rb:58:in `join': can't convert nil into String (TypeError)
          if HOME environment variable was nil.

=== 1.1.5 (2012-09-19)

* Added:  It's now possible to specify a bucket's AWS region using the S3CP_REGION
          environment variable, e.g.,

          % S3CP_REGION=eu-west-1 s3ls my-eu-bucket:prefix/

          or for a shell session,

          % export S3CP_REGION=eu-west-1
          % s3ls my-eu-bucket:prefix/

          Similarly, it is possible to specify a fully-qualified endpoint using
          the S3CP_ENDPOINT environment variable:

          % S3CP_ENDPOINT=s3-eu-west-1.amazonaws.com
          % s3ls my-eu-bucket:prefix/

=== 1.1.4 (2012-09-11)

* Fixed:  Command-line completion that was broken since 1.1.0 (after aws-sdk migration)

* Fixed:  Recursive s3-to-s3 copy that was broken since 1.1.0 (after aws-sdk migration)


=== 1.1.3 (2012-09-05)

* Fixed:   s3-to-s3 copy command failed with error message:
           `s3_to_s3': undefined local variable or method `s3_' for #<Object:0x7f333b239298> (NameError)


=== 1.1.2 / (2012-08-22)

* Added:   All commands now execute an init script ($HOME/.s3cp by default,
           or $S3CP_CONFIG if defined) allowing customization, such as
           setting up the AWS.config parameters such as S3 endpoints, max retries,
           credentials, etc.

           See http://docs.amazonwebservices.com/AWSRubySDK/latest/AWS.html#config-class_method
           for details.

=== 1.1.1 / (2012-08-22)

* Fixed: Invalid break (SyntaxError) in s3ls.

=== 1.1.0 / (2012-08-22)

* First stable release using the aws-sdk gem.  No changes from 1.1.0.pre.4.

=== 1.1.0.pre.4 / (2012-08-22)

* Fixed:    s3dir was not correctly displaying directories

=== 1.1.0.pre.3 / (2012-08-22)

* Fixed:    s3cp >5GB file uploads (Peter Tan)

* Changed:  Updated aws-sdk dependency to ~> 1.6.3

=== 1.1.0.pre.2 / (2012-08-08)

* Added:    's3cp --version' now reports version information.

* Changed:  S3CP uses the default credential provider from aws-sdk which makes
            a best effort to locate your AWS credentials.  It checks a variety of
            locations in the following order:

              * Static credentials from AWS.config (e.g. AWS.config.access_key_id,
                AWS.config.secret_access_key)

              * The environment (e.g. ENV['AWS_ACCESS_KEY_ID'] or
                ENV['AMAZON_ACCESS_KEY_ID'])

              * EC2 metadata service (checks for credentials provided by
                roles for instances).

=== 1.1.0.pre.1 / (2012-07-23)

* Changed:  Underlying AWS library has been changed from 'right-aws' to the official
            AWS SDK ('aws-sdk').   The main driver for this is to stay current on the
            latest features, including instance-specific roles, temporary credentials,
            etc.

* Added:    New 's3buckets' command displays all S3 buckets associated with the
            account.

* Added:    New '--acl PERMISSION' option to 's3cp' and 's3up' commands.

=== 1.0.6 / (2012-07-23)

* Fixed:  s3cp would exit with errorlevel 0 despite errors when running in
          interactive mode.

=== 1.0.5 / (2012-06-25)

* Added:   New command 's3mv' to move files; s3mv is an alias to s3cp --move.

=== 1.0.4 / (2012-06-25)

* Fixed:  Potential "undefined method `finish' for nil:NilClass" error in s3cp.

=== 1.0.3 / (2012-05-11)

* Added:    Exponential backoff for `s3cp` command based on the formula:

              delay = initial_retry_delay * (factor ^ retries)

            where:
                    - `initial_retry_delay` is 1 second by default.
                    - `factor` is 1.4142 by default
                      (retry delay doubles every two retries)
                    - max. number of retries is now 20 by default.

            (which means s3cp retries for roughly 58 minutes by default)

* Fixed:  `s3ls` and `s3du` now properly handle Errno::EPIPE

* Fixed:  `s3du` would not display last character of files

=== 1.0.2 / 2012-03-9

* Added:  `s3up` now outputs uploaded file size to STDERR, e.g.,

          % ls | s3up s3://bucket/path/to/file
          s3://bucket/path/to/file => 214B

=== 1.0.1 / 2012-03-9

* Added:  New command `s3up` to upload STDIN to S3
          e.g. some_cmd | s3up s3://bucket/path/to/destination

  Note: `s3up` does not yet directly stream data to S3 since right_aws gem
        requires the upload size to be known in advance. `s3up` currently
        persists STDIN into a temp file although this may change in the
        future.

=== 1.0.0 / 2012-03-9

Pop the champagne!  It's time we called this a one-oh!! :)

S3CP now offers the feature set originally intended and it's been used reliably
by several people for several weeks and so deemed to be robust enough for
everyday use -- with the usual disclaimer that there are probably undiscovered
bugs -- but at least all known bugs have been addressed.

Probably worth mentioning: there are two outstanding experimental features,

1) s3cp --sync:  Does not handle missing files.  It's basically
                 a faster copy; not a true "sync" at this point.

2) Bash command-line completion is still very rough.

=== 0.2.7 / 2012-03-09

* Fixed #3: s3cp adds extra slash when copying to the root of a bucket.
            e.g. s3://myBucket//someFile or s3://myBucket//someFolderToCopy/someFile

=== 0.2.6 / 2012-03-06

* Fixed:  Possible division-by-zero error in s3du if it encounters
          zero-length files.

=== 0.2.5 / 2012-03-02

* Added: "s3du" command to calculate disk usage.
         Supports --depth, --regex, --unit parameters and more!

* Changed: "s3ls -l" command now accepts --unit and --precision to configure
           file size display.  Uses "smart" unit by default.

* Changed: "s3ls -l" will now use S3CP_DATE_FORMAT environment if set.

=== 0.2.4 / 2012-02-27

* Fixed: Handling of relative paths in s3cp.

  % s3cp -r bucket:path/to/files /local/path

  will now produce local files:

     /local/path/files/A
     /local/path/files/B
     /local/path/files/C
     /local/path/files2/D
     /local/path/files2/E
     ...

  whereas,

  % s3cp -r bucket:path/to/files/ /local/path


    /local/path/A
    /local/path/B
    /local/path/C
    ...

  (note the trailing slash on "bucket:path/to/files")

=== 0.2.3 / (2012-02-24)

* Added: --include and --exclude REGEX support for s3cp

* Added: --sync mode for s3cp

=== 0.2.2 / (2012-02-23)

* Added: Progress bars during upload/download if $stdout.isatty

* Fixed: s3cat now handles broken pipes properly
         e.g. "s3cat bucket:some/file | head" will now terminate early.

=== 0.2.1 / (2012-02-20)

* Added: Bash completion now supports exclusions through S3CP_EXCLUDES
         and defaults to excluding keys containing "_$folder$".

* Changed: s3dir and s3ls --delimiter now display both directories and files

=== 0.2.0 / (2012-02-20)

* Added: s3stat command to display S3 object properties

* Added: s3dir as a shortcut for "s3ls --delimiter / ..."
         (default to character in S3CP_DELIMITER environment variable or "/" if not defined)

* Added: s3cp defaults can now be set using environment variables
         S3CP_OVERWRITE, S3CP_CHECKSUM, S3CP_RETRIES, S3CP_RETRY_DELAY

* Added: Support for Bash command-line completion of S3 URLs (see below).

* Fixed: Skip checksum verification for S3 objects with invalid MD5's
         (details @ https://forums.aws.amazon.com/message.jspa?messageID=234538)

To install Bash completion for S3 URLs, add the following to ~/.bashrc:

  for cmd in [ s3cat s3cp s3dir s3ls s3mod s3rm s3stat ]; do
    complete -C s3cp_complete $cmd
  done

=== 0.1.15 / (2012-02-17)

* Added: s3cp now automatically checks MD5 checksums during download/upload
         and retries up to 5 times by default if the checksum fails.
         The number of attempts may be configured using --max-attempts,
         the retry delay may be changed with --retry-delay and the check
         may be disabled completely using --no-checksum.

=== 0.1.14 / (2012-02-09)

* Fixed:  --date-format argument didn't work correctly

=== 0.1.13 / (2012-02-06)

* Long-listing (-l) now displays file size (in bytes).

=== 0.1.12 / (2012-02-03)

* Re-released 0.1.11 due to misfire.

=== 0.1.11 / (2012-02-03)

* Added:  s3cp --no-overwrite option to prevent overwriting existing files.

=== 0.1.10 / (2012-01-24)

* Fixed: --max-keys now works correctly with --delimiter
* Fixed: do not display any keys if there are no common-prefix
         delimiter-matching keys

=== 0.1.9 / (2012-01-23)

* Added:  s3ls now accepts --max-keys and --delimiter parameters.

=== 0.1.8 / (2011-12-29)

* Fixed:  Apparently, File.new(path, File::CREAT|File::WRONLY) does not
          truncate existing files; use File.new(path, "wb") instead.

=== 0.1.7 / (2011-12-29)

* Fixed:  s3cp would not truncate existing files when overwriting, possibly
          resulting in corrupted files.

=== 0.1.6 / (2011-12-16)

* Changed: s3rm now uses multi-object delete operation for faster deletes
* Changed: dependency on 'aboisvert-aws' instead of 'right-aws' until
           it supports multi-object delete.

=== 0.1.5 / 2011-10-17

* Fixed:  All commands now work properly on Ruby 1.8.x

=== 0.1.4 / 2011-10-17

* Added:  s3rm command

=== 0.1.3 / 2011-09-29

* Fixed:  s3cp --headers names are now converted to lowercase since underlying
          RightAWS gem expects lowercase header names.

=== 0.1.2 / 2011-09-29

* Added:   s3cp now supports passing --headers (Donnie Flood)
* Changed: s3ls now displays entries with s3://@bucket/ prefix

=== 0.1.1 / 2011-09-22

* Added:  s3mod command (Josh Carver)

=== 0.1 / 2011-04-05

* First release