Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Active Storage to Rails #30020

Merged
merged 396 commits into from
Aug 4, 2017
Merged

Add Active Storage to Rails #30020

merged 396 commits into from
Aug 4, 2017

Conversation

rafaelfranca
Copy link
Member

@rafaelfranca rafaelfranca commented Jul 31, 2017

Need to address a few issues:

  • How should the initial migration be seeded? I'm thinking probably just generate it in db/migrations on new apps.
  • Move the direct_upload: true convenience option from the activestorage helper into actionview.
  • Address how Rails' CI can run the cloud service integration tests.

jeremy and others added 30 commits July 8, 2017 15:15
…agents don't respect Content-Disposition filename
Disk storage: ensure URLs end with the blob filename
Disk service: use binary IO throughout, not UTF-8
Pass separate primary service and list of mirrors rather than treating
the first of the services list as the primary. Nice fit for keyword args,
and something we've long wanted in the equivalent Basecamp file repository.

Upload returns the results of the underlying service uploads rather than
the io.rewind result. Rewind before uploading rather than afterward, and
demonstrate that behavior with a test.

Test that more than one mirror works.
* Move service configuration from the Engine to Service
* Delegate configuration mechanics to internal Service::Configurator
* Delegate service building to the concrete Service classes, allowing
  them to configure composed services.
* Implement for the Mirror service.
First arg is config for the service we're instantiating.

Second arg is service configurations so we can look up and configure
other services by name.
* Service.build takes the literal YAML config hash for the service and a
  reference to the Configurator that's doing the building.
* Services that compose additional services can use the Configurator to
  look them up and build them by name. See MirrorService for an example.

References #23
Clarify how a service can build other composed services
Since configuration is a nested hash we need to symbolize all keys
of the hash. Othervise fetcing will fail on start
Symbolize all keys inside configuration nested hash
# Conflicts:
#	lib/active_storage/engine.rb
#	lib/active_storage/service.rb
#	lib/active_storage/service/disk_service.rb
#	lib/active_storage/service/s3_service.rb
#	test/service/s3_service_test.rb
#	test/test_helper.rb
So tests pass when service configs aren't set up.

References #28
Since we use new aws-sdk API, I've scoped aws-sdk version
so someone doesn't accidentaly installs wrong version during the
development.
* Use simple core API for duck-type compat with other clients
* initialize: accept an existing client
* initialize: accept arbitrary client args instead of a fixed, required set
* download: use native get_object streaming, no need to implement range requests
* exists?: use head_object (which returns immediately) rather than waiting for existence
claudiob and others added 2 commits August 3, 2017 11:43
Running `rubocop activestorage` before this commit resulted in 20 offenses.
This commit only fixes:

- Trailing whitespace detected
- Space inside } missing
- Put one space between the method name and the first argument.

The other offenses are left since they are intentional according to
@georgeclaghorn (#30061 (comment))
I know those methods are unlikely to change but having one line method
is hard to read and also hard to modify.
@@ -0,0 +1,27 @@
class ActiveStorageCreateTables < ActiveRecord::Migration[5.1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think this class should be nodoc or needs to be documented?
Currently it's showing up when running rake rdoc at the very root (since it's not namespaced):

list

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Neither, since it’s a template that’s copied into apps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should probably have an extension that communicates it’s a template and excludes it from documentation, or we should otherwise exempt it if that’s at all possible.

@@ -0,0 +1,22 @@
# Take a signed permanent reference for a blob and turn it into an expiring service URL for download.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are these four controllers in app/controllers/active_storage intended to be used in a Rails app?
Are they meant as superclasses to inherit from?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are they meant as superclasses to inherit from?

No, though you could probably extend them if you so desired. Apps use them by sending requests to them, either via routes or DiskService#url{,_for_direct_upload}.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Active Storage is a Rails::Engine so the controllers (and their routes) are available in users' Rails apps 😄

David Heinemeier Hansson and others added 22 commits August 3, 2017 14:54
When Active Storage is not loaded and direct_upload is used on
file_field_tag we should not raise an exception.
Also make sure file_field doesn't mutate the original options passed in.
FormHelper includes FormTagHelper so we don't need to define two methods
It's worth considering whether we should hide these by default, but I'm kinda thinking no. It's very reasonable that someone would want to call these directly, so they should be documented.
Everything inside the app directory of a engine is autoload/eager loaded automatically so we don't need to require them.
Make Rubocop happy about ActiveStorage
We are already removing the braces around hash parameters in the last
argument in other places so we should not change the entire codebase
because of two places.
And deal with a temporary test fix until we allow you to skip active storage.
What we want to test is that two different calls to isolate_namespace
with the same module doesn't change the original railtie. We can do that
defining two different railties.

We can't call in the application because this method is not supposed to
be called in an Application class.
The test was passing before because it was not being testes correctly.

Now we create a different engine that is loaded before the already
exising and we make sure that the first call for isolate_namespace is
what takes effect.
If an AWS bucket name includes a `.` (e.g. `bucket.name`), then the canonical
URL for an object will start with "https://s3.amazonaws.com/bucket.name/"
and not with "https://bucket.name.s3.amazonaws.com/".

The URL tests have now been separated into two separate asserts, to ensure
that both the "s3.amazonaws.com" and the "bucket.name" components are included,
but not specifically in that order.
@kaspth
Copy link
Contributor

kaspth commented Aug 5, 2017

Wups! It works! ❤️

@georgeclaghorn georgeclaghorn deleted the active-storage-import branch August 14, 2017 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.