Non-dirty serialized columns are being saved anyway

[elado]

New Rails 3.2.9 app with one model that has a serialized column:

rails new ar_serialized_dirty_test
cd ar_serialized_dirty_test

rails g model Product name:string data:text
rake db:migrate

# product.rb

class Product < ActiveRecord::Base
  serialize :data
end

On every save, this column is being updated, regardless if it was changed or not.

> p = Product.create(name: "A")
  SQL (0.6ms)  INSERT INTO "products" ("created_at", "data", "name", "updated_at") VALUES (?, ?, ?, ?)  [["created_at", Mon, 26 Nov 2012 23:08:26 UTC +00:00], ["data", nil], ["name", "A"], ["updated_at", Mon, 26 Nov 2012 23:08:26 UTC +00:00]]
 => #<Product id: 1, name: "A", data: nil, created_at: "2012-11-26 23:08:26", updated_at: "2012-11-26 23:08:26"> 
> p.save!
   (0.3ms)  UPDATE "products" SET "updated_at" = '2012-11-26 23:08:34.660484', "data" = '---\n...\n' WHERE "products"."id" = 1
 => true 
> p.save!
   (0.3ms)  UPDATE "products" SET "updated_at" = '2012-11-26 23:08:45.777419', "data" = '---\n...\n' WHERE "products"."id" = 1
 => true 
> p.data_changed?
 => false 
> p.save!
   (0.3ms)  UPDATE "products" SET "updated_at" = '2012-11-26 23:13:20.839794', "data" = '---\n...\n' WHERE "products"."id" = 1

[felipecsl]

+1 for that

[senny]

This is the intended behavior. It was added with this commit 144e869 by @jonleighton . There is even an explantation why it behaves the way it does:

# Serialized attributes should always be written in case they've been
# changed in place.
def keys_for_partial_write
  changed | (attributes.keys & self.class.serialized_attributes.keys)
end

The problem is that serialized data-structures can be changed without using the ActiveRecord setters. Then there is no way to detect that a change actually happend. To work around this problem serialized attributes are written all the time.

[rafaelfranca]

@senny right. Closing it.

@elado thank you for reporting.

[elado]

Thanks.
Indeed, looks like _changed? method doesn't return true if I change a value within a value, p.data[:x] = 1

What about keeping the original serialized value after fetching and compare it to the current serialized value before saving?

[rafaelfranca]

@elado I think this is a unnecessary overhead, much more than writing the serialized attribute every time.

[unorthodoxgeek]

@rafaelfranca think large scale, 1M+ writes per minute to the RDB. on the ruby side, that wouldn't take too much time, but it can bring a really strong SQL machine down on its knees.

there are less expensive ways of doing this than comparing, for instance, extend the serialized object to have a changed attribute accessor and when a setter method is called on it, it will also set that value to true.

[elado]

@rafaelfranca, as @unorthodoxgeek said, it's definitely a performance issue.

Instead of an expensive DB access for no reason, Rails could either serialize the initial value after fetching and compare to it to the current serialized value for dirty check, or have #eql? method in the serializer class (the one that has #load/#dump) that'll compare things by content.

I believe usually the serialized attribute is a Hash or an Array, so #eql? shouldn't be that expensive.

Let's remember this code overhead (not performance overhead) only happens when there are serialized attributes, and current state is that a query is executed every time and this is far longer operation.

Thanks

[senny]

@elado If you can come up with a good PR I think there is nothing that will prevent it from getting merged. As the current behavior is known and expected we don't treat this as a bug though. Feel free to use the Rails Core GoogleGroup to discuss the idea and the change you want to make. Of course you can also just hack up the PR and start a discussion with it.

[rafaelfranca]

@elado @unorthodoxgeek I see. It make sense.

Mind to open a pull request?

I'm reopening this issue.

[jonleighton]

@tenderlove has talked to me about using Object#hash to determine which attributes have changed. That seems like to way to go to me. Then we don't need special handling for serialized attributes.

[al2o3cr]

The challenge here is how to "save the original" for comparison purposes - just stashing it someplace isn't sufficient. For instance, if you've got a Hash with Array values:

irb: h = { a: [1,2,3], b: [4,5,6] }
===> {:a=>[1, 2, 3], :b=>[4, 5, 6]}
irb: h2 = h.deep_dup
===> {:a=>[1, 2, 3], :b=>[4, 5, 6]}
irb: h2[:a][0] = -1
===> -1
irb: h
===> {:a=>[-1, 2, 3], :b=>[4, 5, 6]}

Neither dup nor clone will do the right thing here either - there is one reliable way to do it, but it's (somewhat) expensive and fails if an object can't be Marshalled:

irb: h = { a: [1,2,3], b: [4,5,6] }
===> {:a=>[1, 2, 3], :b=>[4, 5, 6]}
irb: h_copy = Marshal.load(Marshal.dump(h))
===> {:a=>[1, 2, 3], :b=>[4, 5, 6]}
irb: h_copy[:a][0] = -1
===> -1
irb: h
===> {:a=>[1, 2, 3], :b=>[4, 5, 6]}

There's also the side-issue that such copying would have to be done pessimistically (every time an object is loaded from the DB) or users would likely encounter bugs if they mutate the serialized attribute in-place (since calling some_serialized_attribute_will_change! isn't currently required by the "save serialized things every time" strategy). In particular, users may wind up with silent data loss...

[unorthodoxgeek]

what about marshalling it and saving an md5 of the output string? should be
a fairly good way of comparing two hard-to-compare objects...