Encode Content-Disposition filenames on send_data and send_file by mtsmfm · Pull Request #33829 · rails/rails

mtsmfm · 2018-09-09T08:51:53Z

Summary

A few years ago, @stanhu tried to support non-ascii encodings for send_data and send_file.
#21461

but it's still not merged because it lacks tests and some points to be fixed.

In the PR, @jeremy told us how he encodes and actually it's the same how activestorage encodes file name now

I changed ActiveStorage::Filename::Parameters to ActionController::DataStreaming::DispositionFilenameParameters to encode in send_data and send_file and share the logic.

I tested on Chrome, IE11, Safari and Firefox

Chrome

IE11

Safari

Firefox

Script

# 1. Put this file into the root of rails repo
# 2. `bundle exec rackup -b 0.0.0.0`
# 3. `open localhost:9292`
require 'rails/all'

tmpdir = Dir.mktmpdir
Dir.chdir(tmpdir)

ENV['DATABASE_URL'] = "sqlite3://#{File.join(tmpdir, 'database.sql')}"

class TestApp < Rails::Application
  secrets.secret_key_base = 'secret_key_base'
end

Rails.application.configure do
  config.load_defaults 5.2
  config.eager_load = true
  config.active_storage.service = :local
  config.logger = ActiveSupport::Logger.new(STDOUT)
  config.active_storage.service_configurations = {
    local: {
      service: 'Disk',
      root: tmpdir
    }
  }

  config.after_initialize do |app|
    ActiveRecord::Schema.define do
      create_table :users, force: true do |t|
      end

      load File.join(Bundler.rubygems.find_name('activestorage').first.full_gem_path, 'db/migrate/20170806125915_create_active_storage_tables.rb')

      CreateActiveStorageTables.new.change
    end

    class User < ActiveRecord::Base
      has_one_attached :avatar
    end

    class UsersController < ActionController::Base
      def index
        @user = User.new

        render inline: <<~ERB
          <%= form_with model: @user do |f| %>
            <%= f.file_field :avatar %>
            <%= f.submit %>
          <% end %>

          <ul>
            <%- User.all.each do |u| %>
              <li>
                <div>id: <%= u.id %></div>
                <%= link_to 'download via send_data', user_path(u.id) %>
                <%= link_to 'download via rails_blob_path', rails_blob_path(u.avatar, disposition: 'attachment') %>
              </li>
            <% end %>
          </ul>
        ERB
      end

      def create
        User.create!(params.require(:user).permit(:avatar))

        redirect_to users_path
      end

      def show
        user = User.find(params[:id])

        send_data user.avatar.download, disposition: 'attachment', filename: user.avatar.filename.to_s
      end
    end

    app.routes.prepend do
      root to: redirect('/users')
      resources :users
    end

    user = User.new
    user.avatar.attach(io: File.open(__FILE__), filename: %(あ"い"\\う/え'お.rb))
    user.save!
  end
end
Rails.application.initialize!
run Rails.application

at_exit do
  FileUtils.rm_rf(tmpdir)
end

Other Information

~~I'm wondering if we can backport this change to older versions~~
-> I created backport gem https://github.com/mtsmfm/action_dispatch-http-content_disposition

georgeclaghorn

Can we remove ActiveStorage::Filename::Parameters?

georgeclaghorn · 2018-09-09T22:59:45Z

activestorage/app/models/active_storage/filename.rb

Is this :nodoc: comment necessary? What is it excluding from the docs?

Sorry, it's my bad

georgeclaghorn · 2018-09-11T01:38:16Z

Thanks, @mtsmfm! A few more things:

We’re still requiring active_storage/filename/parameters here:

rails/activestorage/app/models/active_storage/filename.rb

Line 6 in 5b0b1ee

require_dependency "active_storage/filename/parameters"
There are ASt test failures like this:
```
ArgumentError: wrong number of arguments (given 1, expected 0)
```
I’m not immediately sure what’s causing them. Can you look into it?
This needs a changelog entry.
In response to the last paragraph of the PR description, I’m afraid we can’t backport this change, as it’s a new feature and not a bug fix. See the maintenance policy for more info.

mtsmfm · 2018-09-11T02:59:35Z

✅
✅Sorry, the cause is 1 🙇
✅
I thought it's kind of bug though, OK, I'll create a backport gem for older Rails.

georgeclaghorn · 2018-09-11T19:07:23Z

actionpack/CHANGELOG.md

This merits more detail. At minimum, we ought to name the specifications implemented (RFCs 2231 and 5987) and provide examples.

rafaelfranca · 2018-09-12T02:24:19Z

activestorage/app/models/active_storage/filename.rb

Don't we need to require this file one? Also action_controller because it is not in Active Storage.

I think we don't have to require because this file is under app and it's on Rails engine

Can we avoid [internally] exposing this class entirely? AFAICS, this method's result is only used in:

rails/activestorage/lib/active_storage/service.rb

Line 125 in b2eb1d1

(type.to_s.presence_in(%w( attachment inline )) || "inline") + "; #{filename.parameters}"

(And that method seems to have more in common with the surrounding lines in send_file_headers!, too.)

Can we avoid [internally] exposing this class entirely?

Hmm, how can I avoid?
AFAIK, there's no way to share this logic across gems without exposing class/module

I think Matthew’s suggesting removing the parameters method here and using the new class directly in ActiveStorage::Service#content_disposition_with.

I was thinking of something like ActionController::ContentDisposition.format(:inline, filename: "hello.jpg") # => "inline; filename=[...]"

So we still need to expose a constant (and following @rafaelfranca's suggestion to move it out of the DataStreaming namespace), but having it do the work internally, and just return a string. It just feels a bit odd to me that we're exposing the class instance, when all we want to do is call to_s on it.

Cool, I'll do that

rafaelfranca · 2018-09-12T02:27:36Z

actionpack/lib/action_controller/metal/data_streaming/disposition_filename_parameters.rb

If we want to reuse inside Active Storage I don't think it should be inside DataStreaming. ActionController:: DispositionFilename is a better name.

okay, I'll rename

ah, you also meant DispositionFilenameParameters should be renamed to DispositionFilename, right?

Right, but maybe DispositionFilenameParameter is better.

mtsmfm · 2018-09-12T09:54:25Z

Failed components: activesupport

seems failed test isn't related to this PR

mtsmfm · 2018-09-12T10:25:05Z

actionpack/lib/action_controller/content_disposition.rb

@rafaelfranca Now I think this class should be put into the same place as ActionDispatch::Http::UploadedFile.
It seems all helper stuff is placed in ActionDispatch.
So I propose to rename this class to ActionDispatch::Http::ContentDisposition.
What do you think?

Good point. Makes sense to me.

rafaelfranca · 2018-09-12T21:26:35Z

activestorage/test/models/blob_test.rb

This require should be probably inside lib/active_storage/service.rb

rafaelfranca · 2018-09-12T21:27:51Z

actionpack/lib/action_controller/content_disposition.rb

Good point. Makes sense to me.

mtsmfm · 2018-09-13T22:43:15Z

@georgeclaghorn @rafaelfranca @matthewd I fixed all points you reviewed. Can you review again, please?

jeremy

👏

mtsmfm · 2018-09-23T14:15:56Z

@georgeclaghorn Can I ask you to review?

kaspth · 2018-09-23T17:43:14Z

Thanks!

`ActiveStorage::Filename#parameters` was removed by rails#33829.

Users downloading non-ASCII attachments would see garbled characters. When used with object storage, AWS S3 would return an InvalidArgument error: Header value cannot be represented using ISO-8859-1. Per RFC 5987 and RFC 6266, Content-Disposition should be encoded properly. This commit takes the Rails 6 implementation of ActiveSuppport::Http::ContentDisposition (rails/rails#33829) and ports it here. Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/47673

Encode Content-Disposition filenames on send_data and send_file rails/rails#33829

mtsmfm mentioned this pull request Sep 9, 2018

Support non-ASCII encodings in Content-Disposition field per RFC 5987 and RFC 6266 #21461

Closed

jeremy requested a review from georgeclaghorn September 9, 2018 22:30

georgeclaghorn reviewed Sep 9, 2018

View reviewed changes

mtsmfm force-pushed the encode-filename branch from 5e35120 to 4fb13b9 Compare September 10, 2018 15:19

mtsmfm force-pushed the encode-filename branch from 4fb13b9 to f6893aa Compare September 11, 2018 02:56

georgeclaghorn reviewed Sep 11, 2018

View reviewed changes

mtsmfm force-pushed the encode-filename branch from f6893aa to 1d3fa52 Compare September 12, 2018 02:13

rafaelfranca reviewed Sep 12, 2018

View reviewed changes

mtsmfm force-pushed the encode-filename branch 2 times, most recently from ba2d8aa to 60e29b5 Compare September 12, 2018 09:18

mtsmfm commented Sep 12, 2018

View reviewed changes

rafaelfranca reviewed Sep 12, 2018

View reviewed changes

Encode Content-Disposition filenames on send_data and send_file

890485c

mtsmfm force-pushed the encode-filename branch from 60e29b5 to 890485c Compare September 13, 2018 12:40

jeremy approved these changes Sep 18, 2018

View reviewed changes

kaspth merged commit ed56a03 into rails:master Sep 23, 2018

mtsmfm deleted the encode-filename branch September 23, 2018 21:29

y-yagi added a commit to y-yagi/rails that referenced this pull request Nov 28, 2018

Fix broken ActiveStorage::BlobTest

c2ef8bb

`ActiveStorage::Filename#parameters` was removed by rails#33829.

suketa added a commit to suketa/rails_sandbox that referenced this pull request Jun 8, 2019

Rails 6.0.0rc1 try034

d2f8340

Encode Content-Disposition filenames on send_data and send_file rails/rails#33829

kevinrobinson mentioned this pull request Nov 26, 2019

Maintenance: Upgrade to Rails 6 studentinsights/studentinsights#2679

Merged

tenpaiyomi mentioned this pull request Jul 1, 2020

ActionController::DataStreaming#send_file_headers! and ActionDispatch::Http::ContentDisposition results in incorrect file names #39772

Closed

cjcolvar mentioned this pull request Nov 18, 2020

Rails 6 support samvera/hydra-head#521

Merged

Conversation

mtsmfm commented Sep 9, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Chrome

IE11

Safari

Firefox

Script

Other Information

Uh oh!

georgeclaghorn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

georgeclaghorn commented Sep 11, 2018

Uh oh!

mtsmfm commented Sep 11, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtsmfm Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtsmfm commented Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mtsmfm Sep 12, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mtsmfm commented Sep 13, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jeremy left a comment

Choose a reason for hiding this comment

Uh oh!

mtsmfm commented Sep 23, 2018

Uh oh!

kaspth commented Sep 23, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

mtsmfm commented Sep 9, 2018 •

edited

Loading

mtsmfm Sep 12, 2018 •

edited

Loading

mtsmfm commented Sep 12, 2018 •

edited

Loading

mtsmfm Sep 12, 2018 •

edited

Loading

mtsmfm commented Sep 13, 2018 •

edited

Loading