Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding validations for `size` and `content_type` #35390

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
5 participants
@abhchand
Copy link

commented Feb 25, 2019

Background

Adds ActiveModel validations for Size and Content Type to Active Storage attachments.

This feature was discussed here on the rails-core mailing list in late 2018.

The above discussion also mentions why it is difficult to validate Presence on Active Storage attachments. A separate change can/will be filed after that discussion reaches a resolution.

Scope

  • Adds Active Storage validations on Size
  • Adds Active Storage validations on Content Type
  • Adds validation helper: validates_attachment()
  • Minor fix when loading the _blob association (more information inline)
  • Tests
  • Updating Active Storage README
  • Updating Rails Edge Guides
  • Updating CHANGELOG

Validating Size

Adds validators to validate the size (in bytes) of the attached Blob object:

validates :avatar, attachment_size: { in: 0..1.megabyte }
validates :avatar, attachment_size: { minimum: 1.kilobyte }
validates :avatar, attachment_size: { maximum: 12.megabytes }

Also accepts a Range as a shortcut option for :in:

validates :avatar, attachment_size: 0..1.megabyte

Validating Content Type

Validates the content type of the attached Blob object:

validates :avatar, attachment_content_type: { in: %w[image/jpeg image/png] }
validates :avatar, attachment_content_type: { not: %w[application/pdf] }

Also accepts a Array or String as a shortcut option for :in:

validates :avatar, attachment_content_type: %w[image/jpeg image/png]
validates :avatar, attachment_content_type: "image/jpeg"

Validation Helper

Provides a more readable validation helper named validates_attachment() which provides the same functionality as validates() but does not require the attachment_ prefix on keys:

validates_attachment :avatar, size: { in: 0..1.megabyte }, content_type: "image/jpeg"

The inspiration here came from Thoughtbot's Paperclip library which implements similar functionality.

Thanks

@abhchand
Copy link
Author

left a comment

Hey @igorkasyanchuk. 👋 I opened this PR as I couldn't get in contact with the previous PR owner.

Also as I outlined there I think it needed several test scenarios included here since validation functionality should be something end users can depend on not breaking.

Also tagging @connorshea since he's been reviewing some of this functionality too. 🔎

Really appreciate everyone's feedback here, thanks!

```

See the [rails guides](https://edgeguides.rubyonrails.org/active_storage_overview.html#validations) for more information.

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Went with the trend of keeping the README. short with basic examples. It links to the rails guide as a more official source for documentation.


ActiveSupport.on_load(:i18n) do
I18n.load_path << File.expand_path("active_storage/locale/en.yml", __dir__)
end

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Loads the translation file. Similar pattern used in other places in Rails (e.g. see ActiveModel)

@@ -30,6 +30,7 @@ def upload

def save
record.public_send("#{name}_attachment=", attachment)
record.public_send("#{name}_blob=", blob)

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

I'm not sure if this is a bug or not. Realized that the #{name}_blob association is not getting properly loaded when attaching a single attachment.

I wrote out a much more detailed description of this in the commit message description itself: see 139e4b4

It's a very small change but I added as its own commit since it's not fully related to the core PR (although it was impacting my tests for it).

I'm happy to remove it if needed. Looking forward to thoughts and feedback here.

Thanks!

This comment has been minimized.

Copy link
@georgeclaghorn

georgeclaghorn Feb 25, 2019

Member

Please break this out into its own PR.

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 26, 2019

Author

@georgeclaghorn - Done! Opened #35412 and removed the commit from here.

messages:
in_between: "must be between %{minimum} and %{maximum}"
minimum: "must be greater than or equal to %{minimum}"
maximum: "must be less than or equal to %{maximum}"

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Size needed default translations for :minimum, :maximum and :in (key name :in_between).

Content Type validation leverages exiting translations in ActiveModel for :inclusion and :exclusion


def parse_shortcut_options(options)
_parse_validates_options(options)
end

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Leverage existing logic inherited from ActiveModel, just aliasing it to a name that better conveys what is being done here.

@@ -0,0 +1,91 @@
module ActiveStorage
module Validations
class AttachmentContentTypeValidator < BaseValidator

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Attachment Validators share a lot of common code, so I abstracted it to a common BaseValidator.

when marked_for_deletion? then []
else
@record.send(blob_association)
end

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Since Active Storage attachments are external to the record itself, they have safeguards that prevent them from uploading to the storage service until the record itself is succcesfully saved (i.e. attachments are saved/applied in an after_save callback)

Because of this, record model validations run before active storage blobs get saved. So validation here becomes a bit tricky: we have to validate based on what the attachment changes we expect to be applied once the record is saved.

We do this by looking at the contents of @attachment_changes.

  • If it's marked for creation then we pull the list of blobs we have already create or expect to create
  • If it's marked for deletion we wont have any blobs

assert_not_empty @user.highlights_attachments
assert_equal @user.highlights_blobs.count, 2
end

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

This and the test directly below are part of the separate commit mentioned above.

Happy to modify as needed, but I thought it would be good to have a test to ensure that the associations get loaded when creating an attachment. I suppose this would be the default expected behavior for a model field, but attachments work differently from regular model fields, so good to have confirmation here.

@@ -0,0 +1,420 @@
# frozen_string_literal: true

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Tests the following scenarios.

  1. This may seem like a lot of scenarios but even a "simple" validation covers a lot of these use cases. Plus the end users of Rails are expecting Active Storage validations to be as flexible and robust as validation functionality provided by other Rails libraries.

  2. Each test tests both one and many attachments, as well as a validation failure followed by a succesful save. Consolidated all these into one test to make runtime faster, reasonably group similar functionality, and prevent a very long list of tests.

"record has no attachment"
"new record, creating attachments"
"persisted record, creating attachments"
"persisted record, updating attachments"
"persisted record, updating some other field"
"persisted record, destroying attachments"
"destroying record with attachments"
"new record, with no attachment"
"persisted record, with no attachment"
"destroying record, with no attachment"
"specifying :in option as String"
"specifying :not option"
"specifying :not option as a String"
"specifying no options"
"specifying redundant options"
"validating with `validates()`"
"validating with `validates()`, String shortcut option"
"validating with `validates()`, Array shortcut option"
"validating with `validates()`, invalid shortcut option"
"validating with `validates_attachment()`"
"validating with `validates_attachment()`, String shortcut option"
"validating with `validates_attachment()`, Array shortcut option"
"validating with `validates_attachment()`, invalid shortcut option"
"validating with `validates_attachment_content_type()`"
"specifying a :message option"
"inheritance of default ActiveModel options"

Run it with

cd activestorage/
bundle exec ruby -Itest test/models/validations/attachment_content_type_validator_test.rb
@@ -0,0 +1,395 @@
# frozen_string_literal: true

This comment has been minimized.

Copy link
@abhchand

abhchand Feb 25, 2019

Author

Similar to above, we test the following scenarios here:

"record has no attachment"
"new record, creating attachments"
"persisted record, creating attachments"
"persisted record, updating attachments"
"persisted record, updating some other field"
"persisted record, destroying attachments"
"destroying record with attachments"
"new record, with no attachment"
"persisted record, with no attachment"
"destroying record, with no attachment"
"specifying :minimum option"
"specifying :maximum option"
"specifying both :minimum and :maximum options"
"specifying no options"
"specifying redundant options"
"validating with `validates()`"
"validating with `validates()`, Range shortcut option"
"validating with `validates()`, invalid shortcut option"
"validating with `validates_attachment()`"
"validating with `validates_attachment()`, Range shortcut option"
"validating with `validates_attachment()`, invalid shortcut option"
"validating with `validates_attachment_size()`"
"specifying a :message option"
"inheritance of default ActiveModel options"

Run it with

cd activestorage/
bundle exec ruby -Itest test/models/validations/attachment_size_validator_test.rb
[ActiveStorage] Adding validations for `size` and `content_type`
=== Size

Validates the size (in bytes) of the attached `Blob` object:

    validates :avatar, attachment_size: { in: 0..1.megabyte }
    validates :avatar, attachment_size: { minimum: 17.kilobytes }
    validates :avatar, attachment_size: { maximum: 38.megabytes }

Also accepts a `Range` as a shortcut option for `:in`:

    validates :avatar, attachment_size: 0..1.megabyte

=== Content Type

Validates the content type of the attached `Blob` object:

    validates :avatar, attachment_content_type: { in: %w[image/jpeg image/png] }
    validates :avatar, attachment_content_type: { not: %w[application/pdf] }

Also accepts a `Array` or `String` as a shortcut option for `:in`:

    validates :avatar, attachment_content_type: %w[image/jpeg image/png]
    validates :avatar, attachment_content_type: "image/jpeg"

=== Validation Helper

This commit also provides a more readable validation helper named `validates_attachment()`
which provides the same functionality as `validates()` but does not require the
`attachment_` prefix on keys:

    validates_attachment :avatar, size: { in: 0..1.megabyte }, content_type: "image/jpeg"

@abhchand abhchand force-pushed the abhchand:adding-validations branch from a1f67b1 to f393d6a Feb 26, 2019

@abhchand

This comment has been minimized.

Copy link
Author

commented Feb 26, 2019

@connorshea - Thanks so much for proofreading! Incorporated all your suggested changes 👍

@reckerswartz

This comment has been minimized.

Copy link

commented Mar 3, 2019

@abhchand
do you want help in adding ?

  • validates if file(s) attached / Validation Helper
  • validates number of uploaded files (min/max required) / Validation Helper
@connorshea

This comment has been minimized.

Copy link
Contributor

commented Mar 3, 2019

I'd say for this PR we should keep the scope limited, adding more validations in separate PRs would make it much easier to review and probably more likely to be merged.

@abhchand

This comment has been minimized.

Copy link
Author

commented Mar 4, 2019

Agreed with @connorshea on keeping this PR small, but those are great ideas for a follow up PR @reckerswartz 💯

@abhchand

This comment has been minimized.

Copy link
Author

commented Mar 8, 2019

Hey @georgeclaghorn 👋 - You had reviewed this once before, just checking back in to see if you had any further requested changes? Thanks!

@reckerswartz

This comment has been minimized.

Copy link

commented Mar 21, 2019

any update on getting this merged?

@connorshea

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2019

@georgeclaghorn apologies for another ping, could you comment on whether this could make it into Rails 6? I'm fine with waiting, I just want confirmation :)

@georgeclaghorn

This comment has been minimized.

Copy link
Member

commented Apr 8, 2019

Sorry, I’m afraid it probably isn’t going to make Rails 6. The first RC is coming any day now.

@connorshea

This comment has been minimized.

Copy link
Contributor

commented Apr 8, 2019

I'll look forward to 6.1 then :) Thanks for answering.

@krbullock

This comment has been minimized.

Copy link

commented Apr 29, 2019

@abhchand @connorshea What do you think of releasing this as a gem until it can be merged into core? The existing active_storage_validations gem has some shortcomings as you've mentioned, including (in my limited testing) a bug that allows an invalid attachment to replace an existing one. I'm willing to help with publishing and maintaining it as a gem.

@reckerswartz

This comment has been minimized.

Copy link

commented Apr 29, 2019

@krbullock if possible please open issue active_storage_validations about the bug. I am sure @igorkasyanchuk will help you get the bug fix until this merge into rails, there are a lot of users using active_storage_validations gem who are running rails 5. we are happy to provide support for them too instead of creating a new gem. 👨

@abhchand

This comment has been minimized.

Copy link
Author

commented May 16, 2019

👍 on keeping this in rails core versus a new gem. In fact the initial reasoning for opening this was for the validations to be part of rails core and community maintained.

Anyone know what the timeline for 6.1 is? Just curious, and wasn't sure where to look that up.

Thanks again to everyone's contributions here so far!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.