Boto3: Issue Copying metadata for large files from one S3 bucket to other s3 bucket

[sb8053035]

Describe the bug

import boto3
s3 = boto3.resource('s3')
source_bucket = {'Bucket':'s3bucket1','Key':'app/a.xml'}
s3.meta.client.copy(source_bucket,'targets3bucket','app/a.xml')

Metadata is not copied when file size is 10 MB or greater. When Copying object from one s3 bucket to other s3 bucket, file copy is successful but metadata is missing when file size is greater than 10 MB.

Expected Behavior

s3 metadata not copied to destination for large files

Current Behavior

S3 object is copied to destination bucket, but metadata is not .

Reproduction Steps

Upload file to source s3 bucket and add metadata.
use below code and verify destination bucket, if file is copied with metadata.

import boto3
s3 = boto3.resource('s3')
source_bucket = {'Bucket':'s3bucket1','Key':'app/a.xml'}
s3.meta.client.copy(source_bucket,'targets3bucket','app/a.xml')

Possible Solution

No response

Additional Information/Context

No response

SDK version used

1.24.44

Environment details (OS name and version, etc.)

AWS Lambda, Python 3.7

[tim-finnigan]

Hi @sb8053035 thanks for reaching out. Have you tried using the copy_from method to copy the metadata over as described here?

[sb8053035]

@tim-finnigan Thank you. It works as an alternative approach.
But s3.meta.client.copy still a bug where it works perfectly fine when object size is in kb and fails to copy metadata when object size is larger.

[tim-finnigan]

Thanks @sb8053035 for following up. I also tested the copy_object command and it copied the object metadata by default. However, there may be slight performance implications and trade-offs between these three commands. There was an older issue I found which discussed some of distinctions: #3051. I think that some further review is needed to better distinguish and document the specific trade-offs between these commands. To consolidate issues I'm going to close this and we can keep tracking issues related to copy, copy_from and copy_object in #3051.