Skip to content

Support upload to S3 buckets with Object Lock enabled #15557

Description

@yoni-yad2

Is your feature request related to a problem? Please describe.
Currently when trying to upload a file into an S3 bucket which has Object Lock enabled, this error shows:

InvalidRequest: Content-MD5 HTTP header is required for Put Object requests with Object Lock parameters

Describe the solution you'd like
Calculate MD5 hash for the file and upload it in the request header along with the file.

Describe alternatives you've considered
Disabling Object Lock on the S3 bucket temporarily to be able to upload the file.

Additional context
Discussion about workarounds for this issue

Activity

  1. added
    s3AWS S3 Protocol Implementation
    on Jan 18, 2024
  2. self-assigned this
    on Jan 18, 2024
  3. dkocher commented on Jan 22, 2024

    @dkocher
    Contributor

    This would need to be set conditionally depending on the object lock configuration 1 of the bucket.

    Footnotes

    1. https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html ↩

  4. dkocher commented on Feb 8, 2024

    @dkocher
    Contributor

    This would need to be set conditionally depending on the object lock configuration 1 of the bucket.

    Footnotes

    1. https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObjectLockConfiguration.html ↩

    It would probably be easier to just set the Content-MD5 unconditionally for al uploads.

  5. dkocher commented on Feb 8, 2024

    @dkocher
    Contributor
  6. added a commit that references this issue on Feb 8, 2024
    9e73dba
  7. dkocher commented on Feb 8, 2024

    @dkocher
    Contributor

    Looks like instead we can use one of

    x-amz-checksum-crc32: ChecksumCRC32
    x-amz-checksum-crc32c: ChecksumCRC32C
    x-amz-checksum-sha1: ChecksumSHA1
    x-amz-checksum-sha256: ChecksumSHA256
    
  8. added this to the 8.7.4 milestone on Feb 8, 2024
  9. added a commit that references this issue on Feb 12, 2024
  10. dkocher commented on Mar 7, 2024

    @dkocher
    Contributor

    Fix does not apply for multipart uploads.

  11. reopened this on Mar 7, 2024
  12. dkocher commented on Mar 7, 2024

    @dkocher
    Contributor

    Fix does not apply for multipart uploads.

    We previously came to the conclusion in f7977d0 that the x-amz-checksum-sha256 must be omitted for parts.

  13. added a commit that references this issue on Mar 8, 2024
    230c28d
  14. linked a pull request that will close this issueFix #15557 for multipart uploads #15699on Mar 8, 2024
  15. added a commit that references this issue on Mar 8, 2024
    a4826c8
  16. added a commit that references this issue on Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

s3AWS S3 Protocol Implementation

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions