New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Permanent URLs for Active Storage blobs #36729
Permanent URLs for Active Storage blobs #36729
Conversation
- Can you explain why we need to denote public vs private files with a column on
active_storage_blobs
? I'm don't see why this is necessary, it is contextual to the generated URL and can technically be both, right? - Why two different services? Surely it would make more sense to just work with one service.
@gmcgibbon I chose to use two different services because of two reasons:
Because I need to know where to look for the blob (in the private or public service), is why I need the |
+1 for bucket-level instead of file-level permissions. Did you consider allowing a blob to be stored in any named service? The reason I ask is that we’d like to make that possible eventually, and layering it on top of the public-private split implemented here is going to be harder down the road. The blobs table would have a |
Hey @georgeclaghorn, thanks for taking a look at this PR and giving comments! I agree that supporting multiple services to be used simultaneously is a very valuable feature and would be a better solution to this. Are you already working on this feature? If not, I can attempt to tackle it. |
I am not. All yours. |
@peterzhu2118 out of curiosity, did you end up tackling this? |
Hey @courtsimas! I had a proposal in #36835, however, it appears that #34935 is more active, so it's more likely that one will get merged. |
@peterzhu2118 are you able to rebase your work now that #34935 is merged? If we denote public v private at the service level (eg. in storage.yml
) we should be able to switch between services without too much branching logic.
daa0183
to
792ffef
Compare
b215650
to
bfd276f
Compare
Hey @gmcgibbon, I've made a rebase, and I've updated the code to follow what's been done in the merged PR. Instead of the access level of a blob being per blob level stored in the database, it is now per service level (i.e. I'm introducing a deprecation in Please let me know how it looks, and I can update documentation accordingly when the code is more finalized. |
else | ||
head :not_found | ||
end | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
First, I’d like to see this moved to a separate controller: ActiveStorage::Public::DiskController#show
.
serve_file
is general enough that it could move to Action Controller eventually, but for now we can just move it to ActiveStorage::BaseController
so ActiveStorage::DiskController
and ActiveStorage::Public::DiskController
can share it.
Second, you can also inline the key
local variable:
def show
if blob = ActiveStorage::Blob.find_by(key: params[:key])
if blob.service.public_service?
serve_file disk_service.path_for(blob.key), content_type: blob.content_type, disposition: :inline
else
head :unauthorized
end
else
head :not_found
end
end
Third, this action should honor the blob’s potentially-custom disk service by using blob.service
in place of disk_service
.
Fourth, I’d like to add ActiveStorage::Blob#public?
and use it here:
class ActiveStorage::Blob
def public?
service.public?
end
end
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've refactored by splitting the public controller portion to ActiveStorage::PublicDiskController
. I didn't see an advantage in placing this controller in another level of namespace nesting.
activestorage/lib/active_storage/service/azure_storage_service.rb
Outdated
Show resolved
Hide resolved
activestorage/app/controllers/concerns/active_storage/file_server.rb
Outdated
Show resolved
Hide resolved
e4839c4
to
7fb5870
Compare
activestorage/lib/active_storage/service/azure_storage_service.rb
Outdated
Show resolved
Hide resolved
0b866c1
to
c969d36
Compare
c969d36
to
38ceb9e
Compare
require "service/shared_service_tests" | ||
require "uri" | ||
|
||
if SERVICE_CONFIGURATIONS[:azure] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if SERVICE_CONFIGURATIONS[:azure] | |
if SERVICE_CONFIGURATIONS[:azure_public] |
require "net/http" | ||
require "database/setup" | ||
|
||
if SERVICE_CONFIGURATIONS[:s3] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if SERVICE_CONFIGURATIONS[:s3] | |
if SERVICE_CONFIGURATIONS[:s3_public] |
@@ -1,3 +1,13 @@ | |||
* Permanent URLs for blobs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* Permanent URLs for blobs. | |
* Permanent URLs for public storage blobs. |
@@ -1,3 +1,13 @@ | |||
* Permanent URLs for blobs. | |||
|
|||
Services can be configured in `configurations.yml` with a new key |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Services can be configured in `configurations.yml` with a new key | |
Services can be configured in `config/storage.yml` with a new key |
Squashed and merged with some minor edits in 94584c2 Thanks @peterzhu2118!! |
@gmcgibbon Thanks for applying the changes and merging! I just realized that I forgot to address your comments, really sorry about that! |
Summary
Addresses issue #31419. Allows access to permanent URL to the blob. This changes Active Storage to use two buckets (a public bucket and a private bucket), where the public bucket has permissions for public access and the private bucket only has private access.
Since public URLs from providers don't allow custom filenames (i.e. download as a different filename as it exists on the service), the directory structure of uploaded files differ in the public bucket and the private bucket. In the public bucket, the directory structure is
/[key]/[filename]
while the directory structure in the private bucket remains/[key]
to maintain backwards compatibility. So for example, if you upload a public filekitten.jpg
and Rails give it the key5a1d6e
, it will be uploaded to/5a1d6e/kitten.jpg
on the cloud provider and thus can be downloaded as a JPEG in the future.Note that you can also choose to just have one of public and private buckets.
Note that this requires #36715 to be merged for Azure to work.
Other Information
I haven't updated much of the documentation, let me know how the code looks, if you think this is a good idea, I'll update more of the documentation.
It also currently does not support moving a file from private to public (and vice versa), nor changing the filename of a public file. This is something that I can implement in the future (not in the scope of this PR however).