put_object with ChecksumAlgorithm="CRC32C" fails

[dukovac]

Describe the issue

put_object with ChecksumAlgorithm=CRC32C fails with botocore.exceptions.FlexibleChecksumError: Unsupported checksum algorithm: crc32c.

The documentation for put_object says that the request syntax is ChecksumAlgorithm='CRC32'|'CRC32C'|'SHA1'|'SHA256' meaning the caller can specify CRC32C for a SDK calculated checksum.

The botocore file httpchecksum.py shows that the above is not the case: https://github.com/boto/botocore/blob/develop/botocore/httpchecksum.py#L465.

Code to reproduce:

access_key = None
secret_key = None
endpoint_url = None
bucket_name = None

s3_params = {
    "addressing_style": "path",
    "payload_signing_enabled": True,
}

args = {
    "service_name": "s3",
    "aws_access_key_id": access_key,
    "aws_secret_access_key": secret_key,
    "endpoint_url": endpoint_url,
    "verify": False,
    "config": Config(
        s3=s3_params,
        signature_version="s3v4",
        region_name="US",
    ),
}

session = boto3.session.Session()
client = session.client(**args)


filename = "helloworld.bin"
payload = b'hello world'

put_kwargs = {
    "Bucket": bucket_name
    "Key": filename,
    "Body": payload,
    "ContentLength": 11,
    "ChecksumAlgorithm": "CRC32C"
}

response = client.put_object(**put_kwargs)

Stack trace:

>       response = client.put_object(**put_kwargs)

tests_py/test_checksums.py:1021: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/local/lib/python3.6/site-packages/botocore/client.py:508: in _api_call
    return self._make_api_call(operation_name, kwargs)
/usr/local/lib/python3.6/site-packages/botocore/client.py:877: in _make_api_call
    resolve_checksum_context(request_dict, operation_model, api_params)
/usr/local/lib/python3.6/site-packages/botocore/httpchecksum.py:234: in resolve_checksum_context
    resolve_request_checksum_algorithm(request, operation_model, params)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
operation_model = OperationModel(name=PutObject)
supported_algorithms = ['crc32', 'sha1', 'sha256']

    def resolve_request_checksum_algorithm(
        request,
        operation_model,
        params,
        supported_algorithms=None,
    ):
        http_checksum = operation_model.http_checksum
        algorithm_member = http_checksum.get("requestAlgorithmMember")
        if algorithm_member and algorithm_member in params:
            # If the client has opted into using flexible checksums and the
            # request supports it, use that instead of checksum required
            if supported_algorithms is None:
                supported_algorithms = _SUPPORTED_CHECKSUM_ALGORITHMS
    
            algorithm_name = params[algorithm_member].lower()
            if algorithm_name not in supported_algorithms:
                raise FlexibleChecksumError(
>                   error_msg="Unsupported checksum algorithm: %s" % algorithm_name
                )
E               botocore.exceptions.FlexibleChecksumError: Unsupported checksum algorithm: crc32c

/usr/local/lib/python3.6/site-packages/botocore/httpchecksum.py:255: FlexibleChecksumError
_______________________________________ 1 of 1 completed, 0 Pass, 1 Fail, 0 Skip, 0 XPass, 0 XFail, 0 Error, 0 ReRun ________________________________________

Links

https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3/client/put_object.html

[dukovac]

To use "CRC32C" you must install awscrt : pip install awscrt worked for me.

https://boto3.amazonaws.com/v1/documentation/api/latest/guide/quickstart.html