S3ng include md5 checksum on put #4100
Conversation
We wrongly passed err to errors.Wrapf instead of err1, leading to a nil error to be returned.
wkloucek
requested review from
a team,
labkode,
ishank011 and
glpatcern
as code owners
| @@ -71,9 +71,9 @@ func (bs *Blobstore) Upload(node *node.Node, source string) error { | |||
| } | |||
| defer reader.Close() | |||
|
|
|||
| _, err1 := bs.client.PutObject(context.Background(), bs.bucket, bs.path(node), reader, node.Blobsize, minio.PutObjectOptions{ContentType: "application/octet-stream"}) | |||
| _, err = bs.client.PutObject(context.Background(), bs.bucket, bs.path(node), reader, node.Blobsize, minio.PutObjectOptions{ContentType: "application/octet-stream", SendContentMd5: true}) | |||
There was a problem hiding this comment.
This might be a performance problem. SendContentMd5: true causes minio-go to reread the whole file before making the actual putobject request as it needs to send a Content-MD5 header. Maybe a more recent version of minio-go supports sending that as a trailer.
There was a problem hiding this comment.
nah ... we can't send our MD5 with this lib ... the SendContentMd5 flag will always calculate the hash itself and there is no way to sneak in a Content-MD5 header.
There was a problem hiding this comment.
let's try to get a fix into upstream: minio/minio-go#1867
|
we cannot use the precalculated md5 hash when large files are sent using multipart upload. For now stick to the lib calculating the hash. We can work on minio/minio-go#1867 to make it use a user provided hash when it does not use multipart uploads. |
Bugfix: S3ng include md5 checksum on put
We've fixed the S3 put operation of the S3ng storage to include a md5 checksum.
This md5 checksum is needed when a bucket has a retention period configured (see https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html).