belongs_to should default to required: true #18233

Closed
dhh opened this Issue Dec 29, 2014 · 34 comments

Projects

None yet
@dhh
Member
dhh commented Dec 29, 2014

Almost every belongs_to declaration seems to be a required association. It's rare that you allow your foreign key to be nil. So let's have required: true be the default and required: false be the optional turn-off.

@dhh dhh added the activerecord label Dec 29, 2014
@dhh dhh added this to the 5.0.0 milestone Dec 29, 2014
@simi
Contributor
simi commented Dec 29, 2014

I can take this, but I'm afraid this can cause a lot of harm.

@dhh
Member
dhh commented Dec 29, 2014

Rocking. I think the main issue will be thinking about whether this has any backwards incompatibility issues. Like if you upgrade an app that already declares validates_presence_of or something. I don’t think it should be a problem, since the validation should just overwrite in that case, I believe. But something to be on the lookout for. 👍

On Dec 28, 2014, at 8:01 PM, Josef Šimánek notifications@github.com wrote:

I can take this.


Reply to this email directly or view it on GitHub #18233 (comment).

@simi
Contributor
simi commented Dec 29, 2014

My fears were fulfilled :(

For this naive implementation (actually it should be changed somewhere in the reflection):

diff --git a/activerecord/lib/active_record/associations/builder/belongs_to.rb b/activerecord/lib/active_record/associations/builder/belongs_to.rb
index 954ea38..e0edc28 100644
--- a/activerecord/lib/active_record/associations/builder/belongs_to.rb
+++ b/activerecord/lib/active_record/associations/builder/belongs_to.rb
@@ -23,6 +23,11 @@ module ActiveRecord::Associations::Builder
       add_counter_cache_methods mixin
     end

+    def self.define_validations(model, reflection)
+      reflection.options[:required] ||= true
+      super
+    end
+
     private

     def self.add_counter_cache_methods(mixin)

I'm getting (because unsaved objects) a lot failing tests:

4305 runs, 10485 assertions, 117 failures, 508 errors, 2 skips

Since belongs_to is heavily used in test suite:

grep -r 'belongs_to' test/ | wc -l
505

it is not good idea to add require: false everywhere or construct complex relations in setup phases in tests IMHO.

@al2o3cr
Contributor
al2o3cr commented Dec 29, 2014

@simi - the implementation given there is incorrect, since ||= will replace an explicit false with true :(

If we make this the default, we should include a warning that it alone isn't sufficient to maintain referential integrity - it suffers from the same sort of race condition as validates_uniqueness_of. Example:

Thread A                               | Thread B
p = Post.new                           |
p.blog_id = 5                          |
p.save                                 |
  BEGIN                                |
    validation verifies that blog with |
      id=5 exists                      |
                                       |
                                       | Blog.find(5).destroy
    INSERT INTO posts etc.             |
  COMMIT                               |

p.save returns true, but the record is now orphaned since the corresponding row
in the blogs table no longer exists.

I'm not personally a huge fan of options where the default is true, since it leaves "nil is different from false" rakes around for people to step on (as above :) ). Maybe required should eventually be inverted and replaced with optional? Associations that wanted to opt-out of the validation could declare optional: true.

@simi
Contributor
simi commented Dec 29, 2014

@al2o3cr sure, this is really bad implementation. But since there is no usage of required: false in whole test suite, it should not be problem for my arguments.

optional sounds good, but I'm still not sure if this is good default behavior.

@dhh
Member
dhh commented Dec 29, 2014

I like optional: true 👍

On Dec 29, 2014, at 5:32, Josef Šimánek notifications@github.com wrote:

@al2o3cr sure, this is really bad implementation. But since there is no usage of required: false in whole test suite, it should not be problem for my arguments.

optional sounds good, but I'm still not sure if this is good default behavior.


Reply to this email directly or view it on GitHub.

@dhh
Member
dhh commented Dec 29, 2014

Matt, this is to give you the best error message in the majority of the cases. You’ll still need to back it up with a unique constraint in the DB to prevent what you mention. But such an error would raise an exception, not a validation error. So the less that needs to happen, the better.

On Dec 29, 2014, at 5:23, Matt Jones notifications@github.com wrote:

@simi - the implementation given there is incorrect, since ||= will replace an explicit false with true :(

If we make this the default, we should include a warning that it alone isn't sufficient to maintain referential integrity - it suffers from the same sort of race condition as validates_uniqueness_of. Example:

Thread A | Thread B
p = Post.new |
p.blog_id = 5 |
p.save |
BEGIN |
validation verifies that blog with |
id=5 exists |
|
| Blog.find(5).destroy
INSERT INTO posts etc. |
COMMIT |

p.save returns true, but the record is now orphaned since the corresponding row
in the blogs table no longer exists.

I'm not personally a huge fan of options where the default is true, since it leaves "nil is different from false" rakes around for people to step on (as above :) ). Maybe required should eventually be inverted and replaced with optional? Associations that wanted to opt-out of the validation could declare optional: true.


Reply to this email directly or view it on GitHub.

@simi
Contributor
simi commented Dec 29, 2014

I think it will be better to introduce new relation based on belongs_to just with another defaults. I'm not sure about the name. Let me think about it for a while.

@sgrif
Member
sgrif commented Dec 29, 2014

Since we're dropping support for Ruby 1.9, why not just use keyword arguments to set the default?

@sgrif
Member
sgrif commented Dec 29, 2014

This would also constitute a major breaking change for most apps, so we should probably have a deprecation cycle.

@chancancode
Member

We would need to pair this with a corresponding change in migration generator, correct? We might want to wait till we figured out what we to do with that (hopefully soon)?

@dhh
Member
dhh commented Dec 29, 2014

This is not worth introducing another name than belongs_to for. For migration purposes, we can have a global config switch like we do in so many other cases. config.active_record.belongs_to_validates_presence = true or whatever.

On Dec 29, 2014, at 10:37, Godfrey Chan notifications@github.com wrote:

We would need to pair this with a corresponding change in migration generator, correct? We might want to wait till we figured out what we to do with that (hopefully soon)?


Reply to this email directly or view it on GitHub.

@simi
Contributor
simi commented Dec 29, 2014

Global switch sounds good. But what about AR test suite? Should that switch it be turned off for test suite or should be optional: true or required: false added everywhere? I think it is not good idea to force creating of complex db objects for AR test suite since it will slow it down a lot IMHO.

@dhh
Member
dhh commented Dec 29, 2014

We can turn that global switch off for the suite, except for where we are explicitly testing it.

On Dec 29, 2014, at 10:52, Josef Šimánek notifications@github.com wrote:

Global switch sounds good. But what about AR test suite? Should that switch it be turned off for test suite or should be optional: true or required: false added everywhere? I think it is not good idea to force creating of complex db objects for AR test suite since it will slow it down a lot IMHO.


Reply to this email directly or view it on GitHub.

@matthewd
Member

A global switch that changes the meaning of a line of code like this is a very bad idea... which is why I really don't think we do do it in so many other cases.

Global switches are exactly how we end up stranding large applications on older versions.

@dhh
Member
dhh commented Dec 30, 2014

We have global switches that set the defaults all over the place. In fact, I think this is key to allowing migrations to progress forward. It allows them to upgrade without submitting themselves to keeping up with every default change at the moment of upgrading. The alternative is just to say “this has changed”, and thus requiring to make all the changes to accommodate that change the moment you upgrade. We’ve just seen how laborious that in the Rails code base itself.

On Dec 30, 2014, at 6:27 AM, Matthew Draper notifications@github.com wrote:

A global switch that changes the meaning of a line of code like this is a very bad idea... which is why I really don't think we do do it in so many other cases.

Global switches are exactly how we end up stranding large applications on older versions.


Reply to this email directly or view it on GitHub #18233 (comment).

@dhh
Member
dhh commented Jan 8, 2015

@simi still interested in finishing this up?

@simi
Contributor
simi commented Jan 8, 2015

Sure. Is the global switch good way how to implement this? Should it be turned on for new applications?

@dhh
Member
dhh commented Jan 8, 2015

Awesome. Yes and yes.

@dhh
Member
dhh commented Feb 8, 2015

@simi Any update on this?

@simi
Contributor
simi commented Feb 14, 2015

@dhh I spent some time with this and I think @al2o3cr is right here. We should deprecate required: true and add optional: true.

@sgrif Keyword arguments make sense here also, but it is not trivial change and all association macros should be updated also.

@dhh
Member
dhh commented Feb 14, 2015

Agree, optional: true is the way to go 👍.

On Feb 13, 2015, at 16:23, Josef Šimánek notifications@github.com wrote:

@dhh I spent some time with this and I think @al2o3cr is right here. We should deprecate required: true and add optional: true.

@sgrif Keyword arguments make sense here also, but it is not trivial change and all association macros should be updated also.


Reply to this email directly or view it on GitHub.

@egilburg
Contributor

How feasible would it be do determine requirement based on db schema?

We are (or should be) encouraging good DB design for maintaining referential integrity instead of solely relying on validations which are not as consistent (e.g. model changes in future, application level bugs, batch jobs, performance-sensitive low-level updates, etc.

As such, if db table has null: false on a belongs_to field, it implies required:true on model, otherwise not. WDYT?

This would also significantly reduce migration path for users, as I can't imagine a lot of apps who prefer to not have required: true on validation when their schema requires it and would throw db error if trying to save (even with a soft save)

@dhh
Member
dhh commented Feb 14, 2015

Not keen on tying this together. Lots of schemas out there that aren’t likely to change soon, and this change still makes sense for them.

On Feb 13, 2015, at 7:40 PM, Eugene Gilburg notifications@github.com wrote:

How feasible would it be do determine requirement based on db schema?

We are (or should be) encouraging good DB design for maintaining referential integrity instead of solely relying on validations which are not as consistent (e.g. model changes in future, application level bugs, batch jobs, performance-sensitive low-level updates, etc.

As such, if db table has null: false on a belongs_to field, it implies required:true on model, otherwise not. WDYT?


Reply to this email directly or view it on GitHub #18233 (comment).

@simi
Contributor
simi commented Feb 14, 2015

@egilburg I think this can be added to gems like schema_validations.

@smolnar smolnar added a commit to otvorenesudy/otvorenesudy-api that referenced this issue Oct 5, 2015
@smolnar smolnar Revise Rails 5 belongs_to associations required by default b10617a
@rafaelfranca rafaelfranca modified the milestone: 5.0.0 [temp], 5.0.0 Dec 30, 2015
@relu relu added a commit to relu/doorkeeper that referenced this issue Feb 13, 2016
@relu relu compat with rails 5's default belongs_to presence validation
Rails 5 adds default presence validation on belongs_to associations:
rails/rails#18233

Application object need not be present for access token creation in Resource
Owner Password Credentials grant flow.

This is an alternative to #780 for backwards compatibility with rails 4
a7afbc1
@relu relu referenced this issue in doorkeeper-gem/doorkeeper Feb 13, 2016
Closed

compat with rails 5's default belongs_to presence validation #787

@tute tute added a commit to doorkeeper-gem/doorkeeper that referenced this issue Feb 26, 2016
@relu @tute relu + tute Rails 4.2 and 5 compatibility fix for belongs_to
Rails 5 adds default presence validation on belongs_to associations:
rails/rails#18233.

Application object need not be present for access token creation in
Resource Owner Password Credentials grant flow.

[fixes #780]
[fixes #787]
747a7b4
@bwheeler96

I realize I'm a little late to the party on this one, but what about the concern that now rails associations either fall into a "required" category, or a "not required" category?

It seems like this change is now adding opinions to how I should model my data, whereas previously rails just gave me the tools to create whatever unopinionated data model I wanted. Would it not make more sense to relegate this functionality to a gem or a new method, something along the lines of child_of :model.

Otherwise how are we deciding which associations are "required" and which are "not required". Has one sounds like it should be required (and belongs_to is almost always the complement of has_one). Also if possible it would be nice to see a deprecation warning included with the validation failure message

@dhh
Member
dhh commented Apr 4, 2016

Party is over months ago. Rails is opinionated software. We always had conventions about how your data should be modeled and what the defaults were. This matches more of the use, more of the time.

@dhh
Member
dhh commented Apr 4, 2016

http://rubyonrails.org/doctrine/#convention-over-configuration is a helpful guide on how these decisions are made and will continue to be made.

@sandeepravi

This seems to be causing an issue with accepts_nested_attributes_for . If I want to create the child object along with the parent, it seems to throwing an error in the child model that the parent has to exist.

This seems to be a pretty standard thing to do, right? Is there any workaround for this? Or am I missing something?

@dhh
Member
dhh commented May 24, 2016

@sandeepravi I'd suggest adding required: false to child models in that case. Not a big fan of accepts_nested_attributes_for anyway and we should figure out a long-term path away from that. But saying required: false will return things to how they were for you.

@sandeepravi

@dhh , yeah right now I've made it optional so it seems to work. But I would rather keep it required and not use accepts_nested_attributes.

Yeah, nested_attributes in multiple places can get messed up quite soon. You generally prefer to have form objects or just plain POROs?

@dhh
Member
dhh commented May 24, 2016

I like aggregation through regular Ruby objects. For example, we have a Signup model that's just a Ruby object orchestrating the build process. So I'd give that a go!

@sandeepravi

yeah, it seems the simplest option as well.

Cool, Thanks!

@ClaudioCarmeli

@dhh Is there a recommended method for cases where you still need to validate the presence of the child objects on the parent object? I am using optional: true but that makes my validation tests pass even though the parents objects can be created without any of their "required" child objects.

@willejos willejos added a commit to willejos/faasa-api that referenced this issue Dec 11, 2016
@willejos willejos remove presence validations of belongs_to assoc.
rails 5 auto-validates these
rails/rails#18233
7ab892d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment