Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extract ActiveStorage::Streaming so your own controllers can use it #41440

Merged
merged 17 commits into from Feb 19, 2021

Conversation

dhh
Copy link
Member

@dhh dhh commented Feb 14, 2021

If you want to build your own controller that streams from active storage, it's helpful to have a method that does the streaming correctly. One such already existed unextracted in ActiveStorage::BaseController.


def show
http_cache_forever public: true do
set_content_headers_from representation.image
stream representation
stream_from_storage representation.image
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this code supposed to stream the representation instead of the image? I think that is the reason why the tests are broken.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They were broken differently before. Think we need the image for the content type headers, but the representation for the streaming.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did found this weird. That you can't just stream the representation directly. Maybe @georgeclaghorn knows why that is.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All representations are supposed to produce an image, so I think this is correct?

module ActiveStorage::Streaming
DEFAULT_BLOB_STREAMING_DISPOSITION = "inline"

include ActionController::Live
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this module included in api_only mode? Asking because otherwise they won't be able to use ActiveStorage anymore

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

api mode does not support streaming? That sounds like something we should fix.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it was a legit question, I don't really know 😇 maybe everything is fine

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do verify! It's a good question.

Copy link
Contributor

@santib santib Feb 19, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After checking #32208 , #32238 and actionpack/lib/action_controller/metal/live.rb seems that the issue with ActionController::Base is only present on direct uploads because of some missing middlewares for api_only apps. So ActionController::Live won't cause any issues.

@dhh dhh merged commit ab8e3d2 into main Feb 19, 2021
@dhh dhh deleted the extract-streaming-blob-concern branch February 19, 2021 14:40
koic added a commit to koic/oracle-enhanced that referenced this pull request Feb 19, 2021
Follow rails/rails#41440.

This PR disables `Layout/FirstArgumentIndentation` cop.
koic added a commit to koic/rubocop-rails_config that referenced this pull request Feb 23, 2021
Follow rails/rails#41440.

This PR disables `Layout/FirstArgumentIndentation` cop.
@Bialogs
Copy link

Bialogs commented Apr 7, 2021

Really appreciate this change.

I am wondering if this changes behavior I noticed with the previous implementation of stream. From observation it seemed that our blob would be downloaded to the Rails server entirely, then streamed to the client. Does the ActiveStorage::BaseController stream implementation stream the blob both from S3 to Rails and from Rails to client?

@dhh
Copy link
Member Author

dhh commented Apr 8, 2021

        blob.download do |chunk|
           stream.write chunk
         end

We're streaming the download and streaming the writing. This is happening in chunks. So it doesn't download the whole thing first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants