Checksum in object stat/get

[trollfot]

Hello

Bug Report

Describe the bug

As I understand from the S3 docs, when providing a "x-amz-checksum-X" header to put an object, the same header should be returned upon stat/get (HEAD/GET). It's currently not the case. Is it due to minio headers generation ? Reading the raw request in the stat does not yield any more result.

To Reproduce

upload a file via "put_object" with a "x-amz-checksum-X" header that is valid. Stat or Get the object.

Expected behavior

The header should be present

[hlf20010508]

I've tried this in official minio-py, and the result is the same, no such x-amz-checksum-X header.
I think you should open an issue there.

[trollfot]

Thank you. I just raised the concern there. I suspect it's minio's responsability and the header is just not present there.

[trollfot]

It seems the query functions do not allow for extra headers to be added and a special header is needed to get the checksum back. Currently, The only headers passed along are the server side encryption (ssec).

[trollfot]

minio/minio-py#1330

[trollfot]

get_object and fget_object have the ability to get an extra dict of request headers. Maybe this could be done for stat_object.

[hlf20010508]

I still can't get it even in get_object.
What's your parameters in put_object and get_object? I guess I missed something.

[trollfot]

It seems that, to get back a checksum header, you have to set a request header "x-amz-checksum-mode: ENABLED". (https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_RequestSyntax). You need to have set a "x-amz-checksum-X" header on the put_object (x-amz-checksum-sha256 for example).

[hlf20010508]

You mean like this?

import hashlib
import base64
import io

buffer = io.BytesIO('test')

sha256 = hashlib.sha256(buffer.getvalue()).digest()
b64_sha256 = base64.b64encode(sha256).decode()

client.put_object(
    "my-bucket",
    "my-object",
    buffer,
    len(buffer.getvalue()),
    metadata={
        'x-amz-checksum-mode': 'ENABLED',
        'x-amz-checksum-sha256': b64_sha256
    },
)

[trollfot]

As i understand it from the docs, 'x-amz-checksum-mode': 'ENABLED', is only required on get/head methods, to signal the server that you want the checksum back as a metadata with the rest of the info

[hlf20010508]

ok, i got it.

[hlf20010508]

Done.
Please update using pip install git+https://github.com/hlf20010508/miniopy-async.git to have a test.
If nothing wrong, I'll publish new version

[trollfot]

        stat = await storage.stat_object(
            userid, child.object_name, request_headers={
                "x-amz-checksum-mode": "ENABLED"
            })

<CIMultiDictProxy('Accept-Ranges': 'bytes', 'Content-Length': '66113', 'Content-Type': 'image/jpeg', 'Etag': '"aa183ad8c63be503136cf2cb6f364c9a"', 'Last-Modified': 'Fri, 17 Nov 2023 11:53:38 GMT', 'Server': 'MinIO', 'Strict-Transport-Security': 'max-age=31536000; includeSubDomains', 'Vary': 'Origin', 'Vary': 'Accept-Encoding', 'X-Amz-Checksum-Sha256': 'MGCj+pPNMgMINi1y/RL1qZtCIyZCBWyFPLwtDtlWtdU=', 'X-Amz-Id-2': 'dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8', 'X-Amz-Request-Id': '179866FF4F9AC0FE', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'x-amz-meta-filename': 'unnamed.jpg', 'Date': 'Fri, 17 Nov 2023 11:55:06 GMT')>

The checksum is there, as expected. I had a look at the code, it's now symmetrical with the other get methods. I think it's very nice and a special thank you to you for the reactivity and help.

[hlf20010508]

You're welcome.
New version published.