Skip to content

Commit

Permalink
Expose copy_options (#663)
Browse files Browse the repository at this point in the history
* Disable tagging_directive for now

* Use copy_options at initialize s3 level

* Update .gitignore

---------

Co-authored-by: HK Dahal <hk@nebham.com>
  • Loading branch information
hkdahal and HK Dahal committed Apr 28, 2024
1 parent 7aa69d8 commit 9c1dc78
Showing 1 changed file with 10 additions and 3 deletions.
13 changes: 10 additions & 3 deletions lib/shrine/storage/s3.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,15 @@
class Shrine
module Storage
class S3
attr_reader :client, :bucket, :prefix, :upload_options, :signer, :public
attr_reader :client, :bucket, :prefix, :upload_options, :copy_options, :signer, :public

MAX_MULTIPART_PARTS = 10_000
MIN_PART_SIZE = 5*1024*1024

MULTIPART_THRESHOLD = { upload: 15*1024*1024, copy: 100*1024*1024 }

COPY_OPTIONS = { tagging_directive: "REPLACE" }

# Initializes a storage for uploading to S3. All options are forwarded to
# [`Aws::S3::Client#initialize`], except the following:
#
Expand All @@ -41,6 +43,10 @@ class S3
# be passed to [`Aws::S3::Object#put`], [`Aws::S3::Object#copy_from`]
# and [`Aws::S3::Bucket#presigned_post`].
#
# :copy_options
# : Additional options that will be used for copying files, they will
# be passed to [`Aws::S3::Object#copy_from`].
#
# :multipart_threshold
# : If the input file is larger than the specified size, a parallelized
# multipart will be used for the upload/copy. Defaults to
Expand All @@ -62,13 +68,14 @@ class S3
# [`Aws::S3::Bucket#presigned_post`]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Object.html#presigned_post-instance_method
# [`Aws::S3::Client#initialize`]: http://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/S3/Client.html#initialize-instance_method
# [configuring AWS SDK]: https://docs.aws.amazon.com/sdk-for-ruby/v3/developer-guide/setup-config.html
def initialize(bucket:, client: nil, prefix: nil, upload_options: {}, multipart_threshold: {}, max_multipart_parts: nil, signer: nil, public: nil, **s3_options)
def initialize(bucket:, client: nil, prefix: nil, upload_options: {}, multipart_threshold: {}, max_multipart_parts: nil, signer: nil, public: nil, copy_options: COPY_OPTIONS, **s3_options)
raise ArgumentError, "the :bucket option is nil" unless bucket

@client = client || Aws::S3::Client.new(**s3_options)
@bucket = Aws::S3::Bucket.new(name: bucket, client: @client)
@prefix = prefix
@upload_options = upload_options
@copy_options = copy_options
@multipart_threshold = MULTIPART_THRESHOLD.merge(multipart_threshold)
@max_multipart_parts = max_multipart_parts || MAX_MULTIPART_PARTS
@signer = signer
Expand Down Expand Up @@ -241,14 +248,14 @@ def copy(io, id, **copy_options)
# don't inherit source object metadata or AWS tags
options = {
metadata_directive: "REPLACE",
tagging_directive: "REPLACE"
}

if io.size && io.size >= @multipart_threshold[:copy]
# pass :content_length on multipart copy to avoid an additional HEAD request
options.merge!(multipart_copy: true, content_length: io.size)
end

options.merge!(@copy_options)
options.merge!(copy_options)

object(id).copy_from(io.storage.object(io.id), **options)
Expand Down

0 comments on commit 9c1dc78

Please sign in to comment.