Byte range support in S3 Transfer Manager

[stefanmoro]

Hi All!

I've been using the S3 Transfer Manager (http://boto3.readthedocs.io/en/latest/reference/customizations/s3.html) for quite some time and I'm really happy with the increased performance it gives when downloading/uploading large objects. The built in retry mechanism is also very helpful.

I just started a new project where I'll work with really large objects that I want to process in parts. The idea is to use byte ranges to fetch parts of an object using byte ranges, process it and then fetch the next part. These parts can be as large as several 100MB so multipart download would speed things up. I'm aware that byte range requests can be used with http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.get_object, but then I'd have to implement multipart and retry mechanisms myself (if I've understood things correctly).

Is there a way to use byte ranges with the S3 Transfer Manager that I haven't figured out yet? If not, are there any plans to support it in the future?

Thanks
Stefan

[dstufft]

There is not currently a way to do this, I'll go ahead and mark this as a feature request.