Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

has_one relation start deleting existing record even when new record failed passing validation #48330

Closed
nov opened this issue May 29, 2023 · 23 comments · Fixed by #48406
Closed

Comments

@nov
Copy link

nov commented May 29, 2023

Steps to reproduce

class User
  has_one :totp_setting
end

class TotpSetting
  belongs_to :user
  validates :user_id, presence: true, uniqueness: true
end

# in rails 7.0.4
@user.create_totp_setting! # => first record
@user.create_totp_setting! # => uniqueness error

# in rails 7.0.5
@user.create_totp_setting! # => first record
@user.create_totp_setting! # => delete first record and succeeds

Expected behavior

@user.create_totp_setting! # => first record
@user.create_totp_setting! # => uniqueness error

Actual behavior

@user.create_totp_setting! # => first record
@user.create_totp_setting! # => delete first record and succeeds

System configuration

Rails version: 7.0.5

Ruby version: 3.1.4

@adrianna-chang-shopify
Copy link
Contributor

👋 Hey @nov ! Can you provide a full repro script? I tried this out with the following script:

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem "rails", "~> 7.0.5"
  gem "debug"
  gem "sqlite3"
end

require "active_record"
require "minitest/autorun"
require "logger"

# This connection will do for database-independent bug reports.
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Base.logger = Logger.new(STDOUT)

ActiveRecord::Schema.define do
  create_table :users, force: true do |t|
  end

  create_table :totp_settings, force: true do |t|
    t.references :user
  end
end

class User < ActiveRecord::Base
  has_one :totp_setting
end

class TotpSetting < ActiveRecord::Base
  belongs_to :user
  validates :user_id, presence: true, uniqueness: true
end

class BugTest < Minitest::Test
  def test_bug
    @user = User.create!
    @user.create_totp_setting!
    assert_raises ActiveRecord::RecordNotUnique do
      @user.create_totp_setting!
    end
  end
end

But I see the same result on Rails 7.0.4 and Rails 7.0.5:

F

Failure:
BugTest#test_bug [scripts/48330_has_one.rb:45]:
[ActiveRecord::RecordNotUnique] exception expected, not
Class: <ActiveRecord::RecordNotSaved>
Message: <"Failed to remove the existing associated totp_setting. The record failed to save after its foreign key was set to nil.">
---Backtrace---
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/has_one_association.rb:109:in `remove_target!'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/has_one_association.rb:64:in `block in replace'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/has_one_association.rb:126:in `transaction_if'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/has_one_association.rb:63:in `replace'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/has_one_association.rb:87:in `set_new_record'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/singular_association.rb:24:in `build'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/singular_association.rb:58:in `block in _create_record'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activesupport-7.0.5/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activesupport-7.0.5/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activesupport-7.0.5/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activesupport-7.0.5/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/transactions.rb:209:in `transaction'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/singular_association.rb:57:in `_create_record'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/has_one_association.rb:135:in `_create_record'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/association.rb:208:in `create!'
/Users/adriannachang/.gem/ruby/3.1.2/gems/activerecord-7.0.5/lib/active_record/associations/builder/singular_association.rb:37:in `create_totp_setting!'
scripts/48330_has_one.rb:46:in `block in test_bug'
---------------

Which seems to be reminiscent of the bug described here.

@yahonda yahonda added the more-information-needed When reporter needs to provide more information label May 30, 2023
@nov
Copy link
Author

nov commented May 31, 2023

Ah, I missed dependent: :destroy in the first code.

# frozen_string_literal: true

require "bundler/inline"

gemfile(true) do
  source "https://rubygems.org"

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem "rails", "~> 7.0.5"
  gem "debug"
  gem "sqlite3"
end

require "active_record"
require "minitest/autorun"
require "logger"

# This connection will do for database-independent bug reports.
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Base.logger = Logger.new(STDOUT)

ActiveRecord::Schema.define do
  create_table :users, force: true do |t|
  end

  create_table :totp_settings, force: true do |t|
    t.references :user
  end
end

class User < ActiveRecord::Base
  has_one :totp_setting, dependent: :destroy
end

class TotpSetting < ActiveRecord::Base
  belongs_to :user
  validates :user_id, presence: true, uniqueness: true
end

class BugTest < Minitest::Test
  def test_bug
    @user = User.create!
    @user.create_totp_setting!
    assert_raises ActiveRecord::RecordNotUnique do
      @user.create_totp_setting!
    end
  end
end

@rails-bot rails-bot bot removed the more-information-needed When reporter needs to provide more information label May 31, 2023
@adrianna-chang-shopify
Copy link
Contributor

adrianna-chang-shopify commented May 31, 2023

Okay, so this appears to be due to changes around #_create_record.
In v7.0.4, we would build (i.e initialize) the record, attempt to save it (failing with a uniqueness validation), delete the initial record, and then raise because calling #save failed:

def _create_record(attributes, raise_error = false, &block)
record = build_record(attributes, &block)
saved = record.save
set_new_record(record)
raise RecordInvalid.new(record) if !saved && raise_error
record
end

With v7.0.5, however, #_create_record replaces the initial record prior to saving it:

def build(attributes = nil, &block)
record = build_record(attributes, &block)
set_new_record(record)

which means that by the time we save it, the original record has been removed, so there is no uniqueness validation error.

The thing I find surprising is that, even with Rails 7.0.4, the initial record is deleted even though we do raise an ActiveRecord::RecordInvalid error. e.g. if I change the test to

class BugTest < Minitest::Test
  def test_bug
    @user = User.create!
    @user.create_totp_setting!
    assert_raises ActiveRecord::RecordInvalid do
      @user.create_totp_setting!
    end
    assert_equal 1, TotpSetting.count, "Expected one TotpSetting to be left"
  end
end

I get

Failure:
BugTest#test_bug [scripts/48330_has_one.rb:46]:
Expected one TotpSetting to be left.
Expected: 1
  Actual: 0

I would expect replacing the record to only take place after we've ensured the new record is valid, but since this behaviour has been constant across 7.0.4 and 7.0.5 I'd like clarification from someone on the Core team as to whether this behaviour is expected.

@yahonda
Copy link
Member

yahonda commented May 31, 2023

It looks like this behavior change has been introduced via bdbe58b

@adrianna-chang-shopify
Copy link
Contributor

@yahonda do you have thoughts on whether the v7.0.4 behaviour would be considered problematic as well? If we addressed the issue of replacing the record after validation, we'd start seeing the expected ActiveRecord::RecordInvalid error, even with the changes in bdbe58b.

To me, it seems incorrect that, despite raising the AR error, we delete the original record even in v7.0.4.

@yahonda
Copy link
Member

yahonda commented Jun 6, 2023

@adrianna-chang-shopify Thanks for the detailed investigation.

IMO, this bug report concern is about exception handling, not (only) about the number of rows. I think the Rails v7.0.4 behavior is expected at least for the 7-0-stable branch.

This issue is complicated, so I also would like to know the opinion from @byroot.

@byroot
Copy link
Member

byroot commented Jun 6, 2023

Yes, this change is undesirable.

I'll revert the PR right away, however it would be nice to add a test.

@byroot byroot reopened this Jun 7, 2023
casperisfine pushed a commit to Shopify/rails that referenced this issue Jun 7, 2023
Ref: rails#48330

When replacing the has_one target, it seems more correct to first
delete the old record, so that if the associated model has a uniqueness
validator, it won't fail.

Both the delete and the insert are in a single transaction, so if the new
record fail to be saved, the transaction will be rolled back, which seems
correct.

If `dependent: :destroy` isn't set, Active Record will try to orphan the
record, which may or may not be valid depending on the schema and validators.

Co-Authored-By: zzak <zzakscott@gmail.com>
@byroot
Copy link
Member

byroot commented Jun 8, 2023

We reviewed this is #48416, and it seems to us that the newer behavior is more correct, as it allows to replace a has_one even if you have a unicity constraint.

We added tests to make sure this behavior is preserved.

@byroot byroot closed this as completed Jun 8, 2023
@nov
Copy link
Author

nov commented Jun 8, 2023

so you mean the behaviour changes in this minor version up?

@byroot
Copy link
Member

byroot commented Jun 8, 2023

so you mean the behaviour changes in this minor version up?

I'm not sure I understand your question, but in my opinion the old behavior was faulty, and I consider this change a bug fix.

Now I understand that some code may have relied on this bug, but it's still a bug fix.

It's debatable whether this bugfix should have been backported, but now that it was, I don't think flip-floping would help.

@nov
Copy link
Author

nov commented Jun 8, 2023

I thought this is unexpected breaking change w/o any changelog mention, but if you say it's bug fix, I can fix our code base for 7.0.5

@joelmoss
Copy link

I have just come across this issue, but for a different reason when using callbacks.

This works with 7.0.4.*...

# frozen_string_literal: true

require 'bundler/inline'

gemfile(true) do
  source 'https://rubygems.org'

  git_source(:github) { |repo| "https://github.com/#{repo}.git" }

  gem 'rails', '~> 7.0.4.3'
  gem 'debug'
  gem 'sqlite3'
end

require 'active_record'
require 'active_support/testing/assertions'
require 'minitest/autorun'
require 'logger'

ActiveRecord::Base.establish_connection(adapter: 'sqlite3', database: ':memory:')
ActiveRecord::Base.logger = Logger.new(STDOUT)

ActiveRecord::Schema.define do
  create_table :appointments, force: true do |t|
    t.boolean :paid, default: false
  end

  create_table :payments, force: true do |t|
    t.references :appointment
  end
end

$before_create_call_count = 0

class Appointment < ActiveRecord::Base
  has_one :payment

  after_create do
    create_payment
  end
end

class Payment < ActiveRecord::Base
  belongs_to :appointment

  before_create do
    $before_create_call_count += 1
    appointment.update_attribute :paid, true
  end
end

class BugTest < Minitest::Test
  include ActiveSupport::Testing::Assertions

  def test_bug
    assert_difference '$before_create_call_count', 1 do
      Appointment.create!
    end
  end
end

But fails on 7.0.5

Failure:
BugTest#test_bug [hasone.rb:56]:
"$before_create_call_count" didn't change by 1.
Expected: 1
  Actual: 2

What is happening on 7.0.5, is that the Payment.before_create callback is being called twice. The first time is in the Appointment.before_create callback, and the second is when appointment.update_attribute is called in the Payment.before_create callback.

Any feedback as to whether this is a new bug or not would be greatly appreciated. thx

@sinsoku
Copy link
Contributor

sinsoku commented Jun 28, 2023

I am also having trouble with the same issue.

@byroot
Is there a way to implement that keeps the behavior similar to v7.0.4?
I'm upgrading to Rails v7.0.5.1 and need to fix the implementation so that existing records are not deleted.

I know I can maintain the behavior if all implementations don't use the has_one provided method as below:

- @user.create_totp_setting!
+ TotpSetting.create!(user: @user)

However, this is very difficult, so I'm looking for an easy migration method.

@byroot
Copy link
Member

byroot commented Jun 28, 2023

so that existing records are not deleted.

So you have an has_one relation, but you want it to map to more than one record? I don't understand the use case here.

@nov
Copy link
Author

nov commented Jun 28, 2023

No, what we want to do here is

  • prevent createing has_one relation record if it already exists
  • using model validation

@byroot
Copy link
Member

byroot commented Jun 28, 2023

Ok, so you want some kind of find_or_create_topt_setting ?

@nov
Copy link
Author

nov commented Jun 28, 2023

not find, but raise exception (or false).

@byroot
Copy link
Member

byroot commented Jun 28, 2023

To be fair, I'm not convinced create_foo on a has_one should replace an existing record at all, it should probably either noop or raise.

But using a uniqueness validator certainly wasn't a proper way to do this as it's very racy.

In the meantime @user.create_totp_setting! unless @user.topt_setting give you the old behavior.

@nov
Copy link
Author

nov commented Jun 28, 2023

yeah, I'm adding those ifs to my apps.

@sinsoku
Copy link
Contributor

sinsoku commented Jun 28, 2023

Changes that add if do not raise ActiveRecord::RecordInvalid, so we need more changes to keep the behavior.

The behavior I want to keep is below.

  • don't delete existing record if it already exists
  • Raise ActiveRecord::RecordInvalid with validation error messages

@joelmoss
Copy link

Can @byroot or anyone comment on #48330 (comment) please?

@byroot
Copy link
Member

byroot commented Jun 28, 2023

@joelmoss same as for the others. Your code should do something like create_payment unless payment.

@shrkw
Copy link

shrkw commented Jun 30, 2023

This change has huge huge huge impact. Rails team should release more carefully.

sinsoku added a commit to sinsoku/rails that referenced this issue Jul 4, 2023
Uniqueness validators no longer works on `create_association`
when using `dependent: :destroy` option in the commit below.
refs: bdbe58b, 3255411

However, we often expect validation errors as well.
refs rails#48330, rails#48632

This commit adds an option to run validations before deleting
an existing record, to reproduce the previous behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants