Generating cache-friendly presigned URLs

[nybblr]

I have a server that generates presigned URLs on the fly so the frontend can display resources in an S3 bucket (images, videos).

resource = Aws::S3::Resource.new

resource.bucket("my_bucket").object("my_image.jpg")
  .presigned_url(:get, expires_in: 3600)

# => https://my_bucket.s3.amazonaws.com/my_image.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20160405T194002Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=abc123aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

This works fine, but caching will fail on most browsers because the query params change on every single request (updated expires_at); this makes for a pretty sad experience.

I'd like to take advantage of the Header-based authorization API so all my requests are for https://my_bucket.s3.amazonaws.com/my_image.jpg, but just the headers will change. The frontend would then manually use ajax to request the image with the correct headers generated by the backend.

Is there a Ruby API exposed for https://docs.aws.amazon.com/AmazonS3/latest/API/sigv4-auth-using-authorization-header.html? Or can the headers be derived from the presigned_url query params?

[awood45]

I believe that this authorization is fundamentally the same. The only signed header in that request is "host", so I don't think you would see signature errors within the expiration period if other headers changed.

The problem you're running in to is one of generating the presigned URL for each request, am I understanding that correctly?

[nybblr]

Correct—for each client request, we generate a new URL, so signing info needs to be submitted through Headers, but there doesn't seem to be an API exposed for that.

Are you suggesting I could perhaps derive the headers from the query params, and the signature would still be valid? I was thinking from Amazon's article that the entire request (including headers + params) gets signed, so the signature would differ based on using query params vs headers for authorization.

[awood45]

You can change headers that aren't part of the signature on the request, but the entire query string is signed - there's no way around this that I am aware of.

What are you trying to do?

[trevorrowe]

@nybblr Are you looking for an interface like this:

req = s3.presigned_request(:get_object, bucket:'...', key: '...')
req.uri #=> "https://..."
req.headers #=> { ... } authorization in here

If so, this is something we have strongly considered and is already on our back log. If not, could you provide a clarifying example of what you are looking for?

[nybblr]

Sorry for the slow response. @trevorrowe Ah yes exactly that!

In the meantime, I don't suppose it's possible to derive those headers from the query params since the whole Request (with headers + url) is used to generate the signature?

[nybblr]

@trevorrowe @awood45 any other clarifications needed? Or is this just waiting on the backlog to thin out a bit?

[awood45]

Added to feature request backlog.

[cjyclaire]

Soft ping here. PR #1477 just opened addressing this feature request : )
Feel free to chime in and add comments : )

[willsmanley]

Is there any solution to this? I am using PHP instead of Ruby but I'm experiencing the same issue with front end caching.

[willsmanley]

After looking for a few minutes I realized that using signed cookies was a better implementation for me.