Encode Content-Disposition filenames on send_data and send_file #33829

mtsmfm · 2018-09-09T08:51:53Z

Summary

A few years ago, @stanhu tried to support non-ascii encodings for send_data and send_file.
#21461

but it's still not merged because it lacks tests and some points to be fixed.

In the PR, @jeremy told us how he encodes and actually it's the same how activestorage encodes file name now

I changed ActiveStorage::Filename::Parameters to ActionController::DataStreaming::DispositionFilenameParameters to encode in send_data and send_file and share the logic.

I tested on Chrome, IE11, Safari and Firefox

Chrome

IE11

Safari

Firefox

Script

# 1. Put this file into the root of rails repo
# 2. `bundle exec rackup -b 0.0.0.0`
# 3. `open localhost:9292`
require 'rails/all'

tmpdir = Dir.mktmpdir
Dir.chdir(tmpdir)

ENV['DATABASE_URL'] = "sqlite3://#{File.join(tmpdir, 'database.sql')}"

class TestApp < Rails::Application
  secrets.secret_key_base = 'secret_key_base'
end

Rails.application.configure do
  config.load_defaults 5.2
  config.eager_load = true
  config.active_storage.service = :local
  config.logger = ActiveSupport::Logger.new(STDOUT)
  config.active_storage.service_configurations = {
    local: {
      service: 'Disk',
      root: tmpdir
    }
  }

  config.after_initialize do |app|
    ActiveRecord::Schema.define do
      create_table :users, force: true do |t|
      end

      load File.join(Bundler.rubygems.find_name('activestorage').first.full_gem_path, 'db/migrate/20170806125915_create_active_storage_tables.rb')

      CreateActiveStorageTables.new.change
    end

    class User < ActiveRecord::Base
      has_one_attached :avatar
    end

    class UsersController < ActionController::Base
      def index
        @user = User.new

        render inline: <<~ERB
          <%= form_with model: @user do |f| %>
            <%= f.file_field :avatar %>
            <%= f.submit %>
          <% end %>

          <ul>
            <%- User.all.each do |u| %>
              <li>
                <div>id: <%= u.id %></div>
                <%= link_to 'download via send_data', user_path(u.id) %>
                <%= link_to 'download via rails_blob_path', rails_blob_path(u.avatar, disposition: 'attachment') %>
              </li>
            <% end %>
          </ul>
        ERB
      end

      def create
        User.create!(params.require(:user).permit(:avatar))

        redirect_to users_path
      end

      def show
        user = User.find(params[:id])

        send_data user.avatar.download, disposition: 'attachment', filename: user.avatar.filename.to_s
      end
    end

    app.routes.prepend do
      root to: redirect('/users')
      resources :users
    end

    user = User.new
    user.avatar.attach(io: File.open(__FILE__), filename: %(あ"い"\\う/え'お.rb))
    user.save!
  end
end
Rails.application.initialize!
run Rails.application

at_exit do
  FileUtils.rm_rf(tmpdir)
end

Other Information

~~I'm wondering if we can backport this change to older versions~~
-> I created backport gem https://github.com/mtsmfm/action_dispatch-http-content_disposition

georgeclaghorn

Can we remove ActiveStorage::Filename::Parameters?

georgeclaghorn · 2018-09-09T22:59:45Z

activestorage/app/models/active_storage/filename.rb

@@ -61,7 +61,7 @@ def sanitized
  end

  def parameters #:nodoc:
-    Parameters.new self
+    ActionController::DataStreaming::DispositionFilenameParameters.new(sanitized) # :nodoc:


Is this :nodoc: comment necessary? What is it excluding from the docs?

Sorry, it's my bad

georgeclaghorn · 2018-09-11T01:38:16Z

Thanks, @mtsmfm! A few more things:

We’re still requiring active_storage/filename/parameters here:

rails/activestorage/app/models/active_storage/filename.rb

Line 6 in 5b0b1ee

require_dependency "active_storage/filename/parameters"
There are ASt test failures like this:
```
ArgumentError: wrong number of arguments (given 1, expected 0)
```
I’m not immediately sure what’s causing them. Can you look into it?
This needs a changelog entry.
In response to the last paragraph of the PR description, I’m afraid we can’t backport this change, as it’s a new feature and not a bug fix. See the maintenance policy for more info.

mtsmfm · 2018-09-11T02:59:35Z

✅
✅Sorry, the cause is 1 🙇
✅
I thought it's kind of bug though, OK, I'll create a backport gem for older Rails.

georgeclaghorn · 2018-09-11T19:07:23Z

actionpack/CHANGELOG.md

@@ -1,3 +1,7 @@
+*   Encode Content-Disposition filenames on `send_data` and `send_file`.


This merits more detail. At minimum, we ought to name the specifications implemented (RFCs 2231 and 5987) and provide examples.

rafaelfranca · 2018-09-12T02:24:19Z

activestorage/app/models/active_storage/filename.rb

@@ -61,7 +59,7 @@ def sanitized
  end

  def parameters #:nodoc:
-    Parameters.new self
+    ActionController::DataStreaming::DispositionFilenameParameters.new(sanitized)


Don't we need to require this file one? Also action_controller because it is not in Active Storage.

I think we don't have to require because this file is under app and it's on Rails engine

Can we avoid [internally] exposing this class entirely? AFAICS, this method's result is only used in:

rails/activestorage/lib/active_storage/service.rb

Line 125 in b2eb1d1

(type.to_s.presence_in(%w( attachment inline )) || "inline") + "; #{filename.parameters}"

(And that method seems to have more in common with the surrounding lines in send_file_headers!, too.)

Can we avoid [internally] exposing this class entirely?

Hmm, how can I avoid?
AFAIK, there's no way to share this logic across gems without exposing class/module

I think Matthew’s suggesting removing the parameters method here and using the new class directly in ActiveStorage::Service#content_disposition_with.

I was thinking of something like ActionController::ContentDisposition.format(:inline, filename: "hello.jpg") # => "inline; filename=[...]"

So we still need to expose a constant (and following @rafaelfranca's suggestion to move it out of the DataStreaming namespace), but having it do the work internally, and just return a string. It just feels a bit odd to me that we're exposing the class instance, when all we want to do is call to_s on it.

Cool, I'll do that

rafaelfranca · 2018-09-12T02:27:36Z

actionpack/lib/action_controller/metal/data_streaming/disposition_filename_parameters.rb

+
+module ActionController
+  module DataStreaming
+    class DispositionFilenameParameters # :nodoc:


If we want to reuse inside Active Storage I don't think it should be inside DataStreaming. ActionController:: DispositionFilename is a better name.

okay, I'll rename

ah, you also meant DispositionFilenameParameters should be renamed to DispositionFilename, right?

Right, but maybe DispositionFilenameParameter is better.

mtsmfm · 2018-09-12T09:54:25Z

Failed components: activesupport

seems failed test isn't related to this PR

mtsmfm · 2018-09-12T10:25:05Z

actionpack/lib/action_controller/content_disposition.rb

+# frozen_string_literal: true
+
+module ActionController
+  class ContentDisposition # :nodoc:


@rafaelfranca Now I think this class should be put into the same place as ActionDispatch::Http::UploadedFile.
It seems all helper stuff is placed in ActionDispatch.
So I propose to rename this class to ActionDispatch::Http::ContentDisposition.
What do you think?

Good point. Makes sense to me.

rafaelfranca · 2018-09-12T21:26:35Z

activestorage/test/models/blob_test.rb

@@ -3,6 +3,7 @@
 require "test_helper"
 require "database/setup"
 require "active_support/testing/method_call_assertions"
+require "action_controller/content_disposition"


This require should be probably inside lib/active_storage/service.rb

rafaelfranca · 2018-09-12T21:27:51Z

actionpack/lib/action_controller/content_disposition.rb

+# frozen_string_literal: true
+
+module ActionController
+  class ContentDisposition # :nodoc:


Good point. Makes sense to me.

mtsmfm · 2018-09-13T22:43:15Z

@georgeclaghorn @rafaelfranca @matthewd I fixed all points you reviewed. Can you review again, please?

jeremy

👏

mtsmfm · 2018-09-23T14:15:56Z

@georgeclaghorn Can I ask you to review?

kaspth · 2018-09-23T17:43:14Z

Thanks!

`ActiveStorage::Filename#parameters` was removed by rails#33829.

Users downloading non-ASCII attachments would see garbled characters. When used with object storage, AWS S3 would return an InvalidArgument error: Header value cannot be represented using ISO-8859-1. Per RFC 5987 and RFC 6266, Content-Disposition should be encoded properly. This commit takes the Rails 6 implementation of ActiveSuppport::Http::ContentDisposition (rails/rails#33829) and ports it here. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/47673

Encode Content-Disposition filenames on send_data and send_file rails/rails#33829

See rails/rails#33829

* Update dependencies to support Rails 6 * Migrate /bin stubs to Rails 6 * Migrate /config files to Rails 6 * Migrate /public files to Rails 6 * Migrate /WcaOnRails files to Rails 6 * Migrate /app/views files to Rails 6 * Add Sprockets 4 manifest * Fix validator deprecation warnings * Update rspec Suite for Rails 6 compatibility * Fix update_attributes deprecation See rails/rails#31998 * Fix UTF-8 encoded Content-Disposition headers in tests See rails/rails#33829 * Hotfix for after_rollback trigger bug in Rails 6 See also rails/rails#36965 * Incorporate review comments

mtsmfm mentioned this pull request Sep 9, 2018

Support non-ASCII encodings in Content-Disposition field per RFC 5987 and RFC 6266 #21461

Closed

jeremy requested a review from georgeclaghorn September 9, 2018 22:30

georgeclaghorn reviewed Sep 9, 2018

View reviewed changes

mtsmfm force-pushed the encode-filename branch from 5e35120 to 4fb13b9 Compare September 10, 2018 15:19

mtsmfm force-pushed the encode-filename branch from 4fb13b9 to f6893aa Compare September 11, 2018 02:56

georgeclaghorn reviewed Sep 11, 2018

View reviewed changes

mtsmfm force-pushed the encode-filename branch from f6893aa to 1d3fa52 Compare September 12, 2018 02:13

rafaelfranca reviewed Sep 12, 2018

View reviewed changes

mtsmfm force-pushed the encode-filename branch 2 times, most recently from ba2d8aa to 60e29b5 Compare September 12, 2018 09:18

mtsmfm commented Sep 12, 2018

View reviewed changes

rafaelfranca reviewed Sep 12, 2018

View reviewed changes

Encode Content-Disposition filenames on send_data and send_file

890485c

mtsmfm force-pushed the encode-filename branch from 60e29b5 to 890485c Compare September 13, 2018 12:40

jeremy approved these changes Sep 18, 2018

View reviewed changes

kaspth merged commit ed56a03 into rails:master Sep 23, 2018

mtsmfm deleted the encode-filename branch September 23, 2018 21:29

y-yagi added a commit to y-yagi/rails that referenced this pull request Nov 28, 2018

Fix broken ActiveStorage::BlobTest

c2ef8bb

`ActiveStorage::Filename#parameters` was removed by rails#33829.

suketa added a commit to suketa/rails_sandbox that referenced this pull request Jun 8, 2019

Rails 6.0.0rc1 try034

d2f8340

Encode Content-Disposition filenames on send_data and send_file rails/rails#33829

kevinrobinson mentioned this pull request Nov 26, 2019

Maintenance: Upgrade to Rails 6 studentinsights/studentinsights#2679

Merged

tenpaiyomi mentioned this pull request Jul 1, 2020

ActionController::DataStreaming#send_file_headers! and ActionDispatch::Http::ContentDisposition results in incorrect file names #39772

Closed

cjcolvar mentioned this pull request Nov 18, 2020

Rails 6 support samvera/hydra-head#521

Merged

gregorbg pushed a commit to gregorbg/worldcubeassociation.org that referenced this pull request Jul 26, 2021

Fix UTF-8 encoded Content-Disposition headers in tests

eaf3783

See rails/rails#33829

gregorbg pushed a commit to gregorbg/worldcubeassociation.org that referenced this pull request Aug 3, 2021

Fix UTF-8 encoded Content-Disposition headers in tests

4026c88

See rails/rails#33829

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Encode Content-Disposition filenames on send_data and send_file #33829

Encode Content-Disposition filenames on send_data and send_file #33829

mtsmfm commented Sep 9, 2018 •

edited

Loading

georgeclaghorn left a comment

georgeclaghorn Sep 9, 2018

mtsmfm Sep 10, 2018

georgeclaghorn commented Sep 11, 2018

mtsmfm commented Sep 11, 2018

georgeclaghorn Sep 11, 2018

mtsmfm Sep 12, 2018

rafaelfranca Sep 12, 2018

mtsmfm Sep 12, 2018

matthewd Sep 12, 2018

mtsmfm Sep 12, 2018 •

edited

Loading

georgeclaghorn Sep 12, 2018

mtsmfm Sep 12, 2018

matthewd Sep 12, 2018

mtsmfm Sep 12, 2018

rafaelfranca Sep 12, 2018

mtsmfm Sep 12, 2018

mtsmfm Sep 12, 2018

rafaelfranca Sep 12, 2018

mtsmfm commented Sep 12, 2018 •

edited

Loading

mtsmfm Sep 12, 2018 •

edited

Loading

rafaelfranca Sep 12, 2018

rafaelfranca Sep 12, 2018

rafaelfranca Sep 12, 2018

mtsmfm commented Sep 13, 2018 •

edited

Loading

jeremy left a comment

mtsmfm commented Sep 23, 2018

kaspth commented Sep 23, 2018

		@@ -1,3 +1,7 @@
		* Encode Content-Disposition filenames on `send_data` and `send_file`.

Encode Content-Disposition filenames on send_data and send_file #33829

Encode Content-Disposition filenames on send_data and send_file #33829

Conversation

mtsmfm commented Sep 9, 2018 • edited Loading

Summary

Chrome

IE11

Safari

Firefox

Script

Other Information

georgeclaghorn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

georgeclaghorn commented Sep 11, 2018

mtsmfm commented Sep 11, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtsmfm Sep 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtsmfm commented Sep 12, 2018 • edited Loading

mtsmfm Sep 12, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mtsmfm commented Sep 13, 2018 • edited Loading

jeremy left a comment

Choose a reason for hiding this comment

mtsmfm commented Sep 23, 2018

kaspth commented Sep 23, 2018

mtsmfm commented Sep 9, 2018 •

edited

Loading

mtsmfm Sep 12, 2018 •

edited

Loading

mtsmfm commented Sep 12, 2018 •

edited

Loading

mtsmfm Sep 12, 2018 •

edited

Loading

mtsmfm commented Sep 13, 2018 •

edited

Loading