Coverage of presigned URLs/presigned POST requests

[fomojola]

Existing documentation URL(s)

https://developers.cloudflare.com/r2/platform/s3-compatibility/api/
https://developers.cloudflare.com/r2/examples/aws-sdk-js/

Section that requires update

The list of sample code showing the calls to listObjects() and listBuckets() doesn't provide any examples of presigned URLs
similar to

https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getSignedUrl-property

or presigned POSTs like

https://docs.aws.amazon.com/AmazonS3/latest/userguide/HTTPPOSTExamples.html#HTTPPOSTExamplesFileUpload.

Are these even supported operations? The compatibility matrix at https://developers.cloudflare.com/r2/platform/s3-compatibility/api/ doesn't say whether or not these are supported.

What needs to change?

Updates to clarify which operations are supported: the direct browser upload functionality is a staple of S3 usage, and as I (and probably many others) feverishly work to replace S3 usage with R2, this will speed up possible implementations. Browser presigned POSTs would be nice, but in a pinch presigned PUTs could be used as well. This is available for the Cloudflare Stream service, so it seems reasonable to expect it to be in the R2 implementation.

How should it change?

List if these functions are supported and (possibly) provide samples of the right way to call them.

[fomojola]

Comments: I have the basic s3 sdk configured and working: I can upload and download files via the API. It is specifically when I am trying to generate presigned urls (so I can upload/download using curl) and signed POSTs (so I can upload from a browser) that I'm having difficulty. I'm uncertain if its CORS related (there isn't any CORS configuration available), and given that its still in beta maybe I'm just ahead of the curve here. But after about 7 hours of torturing my code I figured if I ask here maybe answers will have emerged by the time I wake up.

[KianNH]

R2 doesn't currently support presigned URLs - they're on the roadmap.

[fomojola]

Thank you! Will table my migration now and resume once those become available.

[Flynsarmy]

Are multipart uploads impossible without presigned URLs?

[KianNH]

No, you can still do them - just not by using presigned URLs

Authenticating at the time of the request with any AWS SDK or utility like rclone will work.

[st1992]

Any ETA/timeline on when presigned URLs will be supported by R2?

[KianNH]

R2 has just launched presigned URL support.

[Flynsarmy]

Is there a latest product updates news feed somewhere?

[KianNH]

Currently for R2? Your best bet is our developer discord, https://discord.gg/cloudflaredev

Going forward, I have made a retroactive changelog. https://github.com/cloudflare/cloudflare-docs/pull/4853/files

[Flynsarmy]

Oh the changelog will be super useful. Tyvm!

[Flynsarmy]

After the changes I'm still getting CORS errors:

Access to XMLHttpRequest at 'https://mybucket.r2.cloudflarestorage.com/myfile.jpg?uploadId=some_id&partNumber=1&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=some_credential&X-Amz-SignedHeaders=host&X-Amz-Expires=3600&X-Amz-Signature=some_signature from origin 'https://mysite.com' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

Is there a way to set CORS policy for R2 or are client-side uploads still not possible?

[computerex]

Having same issue myself. I have managed to produce a pre-signed URL where I can put stuff with postman + requests:

response = cl.generate_presigned_url(
        ClientMethod = 'put_object',
        Params = {
            'Bucket': 'orbiter-mods',
            'Key': key,
        },
        ExpiresIn=3600
    )

file_size = os.path.getsize(filename)

    with open(filename, "rb") as f:
        with tqdm(total=file_size, unit="B", unit_scale=True, unit_divisor=1024) as t:
            wrapped_file = CallbackIOWrapper(t.update, f, "read")
            requests.put(response, data=wrapped_file)

This works flawlessly.

Trying to PUT to the generated URL via ajax results in CORS issue like above. Trying to use form to put like this also fails:

<form action="https://<account_id>.r2.cloudflarestorage.com/orbiter-mods/foo/test1.JPG?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=<credential>&X-Amz-Date=20220704T163733Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=<signature>" method="put" enctype="multipart/form-data">
      <!-- Copy the 'fields' key:values returned by S3Client.generate_presigned_post() -->
      <input type="hidden" name="X-Amz-Algorithm" value="AWS4-HMAC-SHA256" />
      <input type="hidden" name="X-Amz-Credential" value="<crednetial>" />
      <input type="hidden" name="X-Amz-Date" value="20220704T165618Z" />
      <input type="hidden" name="X-Amz-Expires" value="3600" />
      <input type="hidden" name="X-Amz-SignedHeaders" value="host" />
      <input type="hidden" name="X-Amz-Signature" value="<signature>" />
    File:
      <input type="file"   name="foo/test1.JPG" /> <br />
      <input type="submit" name="submit" value="Upload to Amazon S3" />
    </form>

Results in a 403:

<Error>
<Code>SignatureDoesNotMatch</Code>
<Message>The request signature we calculated does not match the signature you provided. Check your secret access key and signing method. </Message>
</Error>