S3 PutObject is missing a Content-MD5 header when writing to a bucket with a default object lock configuration

[huubvdw]

When writing a new object to a bucket with a default object lock configuration, the request will result is a 400 bad request response. InvalidRequest: Content-MD5 HTTP header is required for Put Object requests with Object Lock parameters

I noticed in ApplyChecksumMiddleware the PutObject operation only sets a sha256 content hash header.

As you don't know upfront if a bucket has a object lock configuration, I'm thinking of setting both the sha256 content hash header and the content MD5 header for the PutObject operation.

Any other suggestions?

[diehlaws]

Hi @huubvdw, thanks for reaching out to us about this, and for submitting the associated PR #1695. We will review the PR shortly to ensure it is valid to merge.

[howardlopez]

@huubvdw, your investigation and PR are appreciated. Unfortunately, PutObject only requires the Content-MD5 header when there are object lock parameters, so we won't be able to accept your PR to change the general PutObject behavior in this way.

S3 and the SDK teams are aware of the issue and have discussed options internally, but for now, you'll have to supply your own ContentMD5 parameter to object lock requests with PutObject. For example:

$client->putObject([
    'Bucket' => 'your_bucket',
    'Key' => 'your_key',
    'Body' => 'your_body',
    'ObjectLockMode' => 'GOVERNANCE',
    'ObjectLockRetainUntilDate' => $someDate
    'ContentMD5' => base64_encode(md5('your_body', true))
]);

[mmomasters]

Hi everyone, I am having the same issue and was wondering where can I supply my own ContentMD5 parameter for s3fs to work correctly while using guidance ?

I tried s3fs with -o enable_content_md5 with no success

Thank you!

[harshadparekh]

This suggestion of passing contentMD5 with PutObject() request assumes that the data we want to write as an object in AWS is available as a file locally so that we can pre-compute the MD5 on the content. For cases where we're generating data on the fly and providing it as an input stream to PutObject() request, how do we get the MD5 to pass alongwith? Also, how does this work when using MultiPartUpload for large objects?

[TheJosh]

It appears that the ContentMD5 field is not listed in the SDK Docs:
https://docs.aws.amazon.com/aws-sdk-php/v3/api/api-s3-2006-03-01.html#putobject

Does the field still exist, even though it's not in the docs?

[dean-huczok]

Can't find it listed but in the PutObjectRequest class there is a MD5Digest property.
You just gotta MD5 hash your file, base64-encode the hash (string), then slap that string into the PutObjectRequest (C#) and it verks!

PutObjectRequest putRequest = new PutObjectRequest
            {
                BucketName = bucketName,
                Key = keyName,
                FilePath = filePath,
                ContentType = "text/plain",
                MD5Digest = base64encodedHashString  // <-- this will become the Content-MD5 header
            };

[burka]

I calculate the MD5 and set it as the header, file by file:

s3cmd put --add-header="Content-MD5: "`cat your-file | openssl dgst -md5 -binary | openssl enc -base64` your-file s3://bucket

Not nice, but works :-)