Active Storage does not handle HEAD requests properly with AWS S3 backend

[soirju]

Steps to reproduce

A javascript HEAD XHR request to a resource on the web server is redirected to a HEAD request on the S3 server. The resulting redirect URL is calculated using the url method in rails/activestorage/lib/active_storage/service/s3_service.rb based on the :get method only, including the pre-signed key. This key allows only GET requests in S3, no HEAD request.

Expected behavior

Redirect works with 200 - OK

Actual behavior

Redirected HEAD request gives 403 - forbidden

System configuration

Rails version: 5.2.2

Ruby version: 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]

Example:

Initial header request

eedchme@DE-00000657:~$ curl --head "https://www.meyer-kohlscheid.com/weissensing/rails/active_storage/blobs/eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaHBHQT09IiwiZXhwIjpudWxsLCJwdXIiOiJibG9iX2lkIn19--93210b5d911fd57dd7a9fd5436e503b979f674aa/test.mp3"
HTTP/1.1 302 Found
Date: Thu, 18 Apr 2019 15:02:50 GMT
Server: Apache/2.4.29 (Ubuntu)
Cache-Control: max-age=300, private
Referrer-Policy: strict-origin-when-cross-origin
X-Permitted-Cross-Domain-Policies: none
X-XSS-Protection: 1; mode=block
X-Request-Id: 949557ce-9553-40ad-882c-f513932087b4
X-Download-Options: noopen
X-Runtime: 0.011524
X-Frame-Options: SAMEORIGIN
X-Content-Type-Options: nosniff
X-Powered-By: Phusion Passenger 5.0.30
Transfer-Encoding: chunked
Location: https://s3.eu-central-1.amazonaws.com/weissensing.meyer-kohlscheid.com/P681vxjhfjbWfRiFtrrc4w9c?response-content-disposition=attachment%3B%20filename%3D%22test.mp3%22%3B%20filename%2A%3DUTF-8%27%27test.mp3&response-content-type=audio%2Fmpeg&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI7CJAEZLGQS2X6OQ%2F20190418%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20190418T150250Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=15303b6e4fa6bb3bc6467a181efedab3ba90b789d6c632f3f9036296381725ce
Status: 302 Found
Content-Type: text/html; charset=utf-8

---

Follow re-direct (HEAD request) 403 - Forbidden

eedchme@DE-00000657:~$ curl --head "https://s3.eu-central-1.amazonaws.com/weissensing.meyer-kohlscheid.com/P681vxjhfjbWfRiFtrrc4w9c?response-content-disposition=attachment%3B%20filename%3D%22test.mp3%22%3B%20filename%2A%3DUTF-8%27%27test.mp3&response-content-type=audio%2Fmpeg&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI7CJAEZLGQS2X6OQ%2F20190418%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20190418T150545Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=3c40852c09b0d4dd5968703b8fa6b1296c28468cd5779bf43377c5c5e32e2bad"
HTTP/1.1 403 Forbidden
x-amz-request-id: E8878A449D6D70B5
x-amz-id-2: LUAJcAU4e6Mskjrrb5R6Nn03H6GuGyVurnMfJAcQGByEjdiZNmcXlVgcF6JxE5tmitAnkLNEwAs=
Content-Type: application/xml
Transfer-Encoding: chunked
Date: Thu, 18 Apr 2019 15:06:02 GMT
Server: AmazonS3

---

Follow same re-direct (GET request) - Working!

eedchme@DE-00000657:~$ curl "https://s3.eu-central-1.amazonaws.com/weissensing.meyer-kohlscheid.com/P681vxjhfjbWfRiFtrrc4w9c?response-content-disposition=attachment%3B%20filename%3D%22test.mp3%22%3B%20filename%2A%3DUTF-8%27%27test.mp3&response-content-type=audio%2Fmpeg&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI7CJAEZLGQS2X6OQ%2F20190418%2Feu-central-1%2Fs3%2Faws4_request&X-Amz-Date=20190418T150545Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host&X-Amz-Signature=3c40852c09b0d4dd5968703b8fa6b1296c28468cd5779bf43377c5c5e32e2bad" --output test.mp3
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 9296k 100 9296k 0 0 8729k 0 0:00:01 0:00:01 --:--:-- 8729k

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least three months.
The resources of the Rails team are limited, and so we are asking for your help.
If you can still reproduce this error on the 5-2-stable branch or on master, please reply with all of the information you have about it in order to keep the issue open.
Thank you for all your contributions.

[vkalach]

this one is still actual
Rails 6.0.3.5

[ledowong]

Having the same issue in rails 7. Ideal if there is an option for us to override the hard-code :get in here would be nice.

My application scenario:

1. frontend ask for a export file.
2. the file not ready yet, background job start generate the export file, frontend retry the same endpoint every 3 seconds, until the file is ready.
3. the export file is ready and uploaded to s3 in background job
4. endpoint return the signed s3 url to frontend
5. frontend JS trigger browser download the signed s3 url
6. Due to s3 cache delay (it is reasonable), s3 return 404 for the signed url.
7. browser saved the file to file system, however the file content is actually s3 404 XML...

What I want to do, after step 5, frontend use JS to check if the file is ready, using HEAD request to s3.

[ledowong]

a monkey patch in rails-api/config/initializers/active_storage_s3_monkey_patch.rb

require 'active_storage/service/s3_service'

module ActiveStorage
  class Service::S3Service < Service
    private

    def private_url(key, expires_in:, filename:, disposition:, content_type:, **client_opts)
      method = client_opts.delete(:method)
      if method == :head
        object_for(key).presigned_url method, expires_in: expires_in.to_i, **client_opts
      else
        object_for(key).presigned_url :get, expires_in: expires_in.to_i,
          response_content_disposition: content_disposition_with(type: disposition, filename: filename),
          response_content_type: content_type, **client_opts
      end
    end
  end
end