Progress callback for S3Object

[amw]

Hi, I would like to indicate progress of a large file upload to my users. Currently it is not supported with the SDK. If I wanted to use a single-part upload only (I don't) I could use the block form of S3Object#write and report how many bytes I've fed to the SDK, but that's not an actual upload - at that point the data is still waiting to be sent. For multipart upload there's not even that.

What do you think about adding a callback to that method? Something like:

progress = -> bytes { ... }
object.write file: path, progress_callback: progress

[trevorrowe]

This is an interesting suggestion, but we are currently feature locked on the v1 SDK. This would be a great suggestion for the v2 SDK: https://github.com/aws/aws-sdk-core-ruby

[amw]

aws-sdk-core-ruby has been moved back to this repository, but the issues have not been migrated. Maybe it's enough to reopen this issue? The other issue I've opened is amazon-archives/aws-sdk-core-ruby#194.

[trevorrowe]

I added this to our public backlog here: https://github.com/aws/aws-sdk-ruby/blob/master/FEATURE_REQUESTS.md#progress-callbacks-for-amazon-s3-object-uploads

[bmedici]

Hi there, as I'm integrating an S3 target in rest-ftp-daemon, I was wondering if upload progress was available through a callback method or passing a block, now ?
Any news about this feature ?

[Doug-AWS]

We've implemented a wait_until method on the S3 client. The Dev Guide has a section that describes the mechanism. See the "Waiters" section of the "Programming with the AWS SDK for Ruby" topic, http://docs.aws.amazon.com/sdk-for-ruby/latest/DeveloperGuide/aws-ruby-sdk-programming.html.

[bmedici]

Thanks @Doug-AWS for the pointer. It may be useful to provide a clean and easy way to get a method called with progress information on a Aws::S3::Resource upload_file transfer (callback).

[mullermp]

Reopening - deprecating usage of Feature Requests backlog markdown file.

[sivagollapalli]

@mullermp Any guidelines so I could take this up?

[mullermp]

@sivagollapalli I'm not sure - it sounds like the original requestor wanted a callback function. Would a built in progress bar suffice?

[trevorrowe]

A callback function which yields the total object size in bytes and the number of bytes transmitted would be more flexible and then avoids sending data to standard out. Ideally this could be done generically for all streaming input or streaming output operations (which receive or return IO objects).

[siva-wal]

If s3 supports callback function which sends amount of bytes that has been written then callback would be better. If there is no such thing then @trevorrowe option would be better so developers can render output to some frontend apps using IO objects