Limit an upload filesize with a Pre-signed URL

[hequs]

I'm using S3 to store users' files on my website. In order to optimize server load, I've configured a direct file upload from browser for my users. The flow is the following:

1. user calls the server API method to retrieve a Pre-singed Upload Url
2. user uploads some file to AWS S3 directly with the url
3. user call another API method to notify server, that the upload was finished

I want to protect my S3 storage from obusing, so I need the way to limit the maximum file size. Unfortunately, I have not found a way to do this via .Net SDK.

I've read about restriction polices for HTTP POST (http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-post-example.html). Can it be applied on a Pre-signed Url flow.

Regards, Vadim.

[PavelSafronov]

I don't believe your use-case is supported for pre-signed urls, but take a look at Authenticating Requests in Browser-Based Uploads Using POST . This should cover your case and there is built-in support for limiting the upload size (along with other options) using a POST policy.

[tejasmanohar]

Is the top answer here correct/endorsed? Essentially, it suggests setting Content-Length to the "maximum" length.

https://stackoverflow.com/questions/25991275/limit-size-of-objects-while-uploading-to-amazon-s3-using-pre-signed-url

The V4 signing protocol offers the option to include arbitrary headers in the signature. See: http://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-query-string-auth.html So, if you know the exact Content-Length in advance, you can include that in the signed URL. Based on some experiments with CURL, S3 will truncate the file if you send more than specified in the Content-Length header. Here is an example V4 signature with multiple headers in the signature http://docs.aws.amazon.com/general/latest/gr/sigv4-add-signature-to-request.html

[janacm]

For any other wanderers that end up on this thread - if you set the Content-Length attribute when sending the request from your client, there a few possibilities:

1. The Content-Length is calculated automatically, and S3 will store up to 5GB per file
2. The Content-Length is manually set by your client
   2a - The Content-Length matches your actual file size and S3 stores it.
   2b - The Content-Length is less than your actual file size, so S3 will truncate your file to fit it.
   2c - The Content-Length is larger than your actual file size, and you will receive a 400 Bad Request

In any case, a malicious user can override your client and manually send a HTTP request with whatever headers they want, including a much larger Content-Length than you may be expecting. Signed URLs do not protect against this! The only way is to setup an POST policy. Official docs here: https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-HTTPPOSTConstructPolicy.html

More details here: https://janac.medium.com/sending-files-directly-from-client-to-amazon-s3-signed-urls-4bf2cb81ddc3?postPublishedType=initial

Alternatively, you can have a Lambda that automatically deletes files that are larger than expected.

[orekav]

For any other wanderers that end up on this thread - if you set the Content-Length attribute when sending the request from your client, there a few possibilities:

@janacm

Thanks for sharing the link to your post but it seems it is not public.

This one is free to read https://zaccharles.medium.com/s3-uploads-proxies-vs-presigned-urls-vs-presigned-posts-9661e2b37932

[janacm]

For any other wanderers that end up on this thread - if you set the Content-Length attribute when sending the request from your client, there a few possibilities:

@janacm

Thanks for sharing the link to your post but it seems it is not public.

This one is free to read https://zaccharles.medium.com/s3-uploads-proxies-vs-presigned-urls-vs-presigned-posts-9661e2b37932

Hey here's that link without paywall: https://towardsaws.com/sending-files-directly-from-client-to-amazon-s3-signed-urls-4bf2cb81ddc3?sk=88c25bcd8689b40a4fd4045d4cf93eff