Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perf: avoid dupes add fallback logic for coders #12188

Merged
merged 1 commit into from
Sep 11, 2013

Conversation

SamSaffron
Copy link
Contributor

This had a pretty significant perf improvement, will run numbers tomorrow and add to the ticket, it changes semantics of @column_hash , nonetheless all the tests seem to pass.

It avoids fairly significant repetitive copies of columns in a table on each instance. If this is found to be safe its a strong candidate for backport into 4.0.

@SamSaffron
Copy link
Contributor Author

NOTE: please hold off on merging this until I fully test it and run it for a bit. Comments totally welcome :)

}
}
column = @columns_hash[name] if @columns_hash
column = self.class.column_types[name] unless column
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

column ||= self.class.column_types[name]

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops :) will fix

@SamSaffron
Copy link
Contributor Author

I just run the discourse bench on this and got the following:

Before

---
home_page:
  50: 27
  75: 28
  90: 29
  99: 71
topic_page:
  50: 44
  75: 45
  90: 46
  99: 93
home_page_admin:
  50: 36
  75: 37
  90: 38
  99: 85
topic_page_admin:
  50: 60
  75: 63
  90: 65
  99: 183
timings:
  load_rails: 2479
ruby-version: 2.0.0-p247

After

---
home_page:
  50: 26
  75: 26
  90: 27
  99: 72
topic_page:
  50: 42
  75: 43
  90: 45
  99: 95
home_page_admin:
  50: 35
  75: 36
  90: 37
  99: 81
topic_page_admin:
  50: 57
  75: 60
  90: 64
  99: 177
timings:
  load_rails: 2624

Its a noticeable improvement, brings Discourse home page only a smidgen slower than it was on rails 3 but the topic page is now much faster.

For comparison, these are the numbers on Rails 3

---
home_page:
  50: 25
  75: 26
  90: 27
  99: 81
topic_page:
  50: 46
  75: 48
  90: 52
  99: 103
home_page_admin:
  50: 34
  75: 36
  90: 37
  99: 86
topic_page_admin:
  50: 61
  75: 65
  90: 69
  99: 120
timings:
  load_rails: 2463

@SamSaffron
Copy link
Contributor Author

I think this is good to merge, thoughts?

@tenderlove
Copy link
Member

@SamSaffron thanks!

tenderlove added a commit that referenced this pull request Sep 11, 2013
Perf: avoid dupes add fallback logic for coders
@tenderlove tenderlove merged commit 3d60e9d into rails:master Sep 11, 2013
tenderlove added a commit that referenced this pull request Sep 11, 2013
Perf: avoid dupes add fallback logic for coders

@attributes_cache = {}
@column_types = self.class.column_types
@column_types_override = fresh_object.instance_variable_get('@columns_types_override')

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was a typo on the instance var call here, I fixed that on master and 4-0, lets keep an eye on travis now ;)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, totally my bad

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I know why that happened ... columns_hash vs. column_types we should make that consistent at least.

column_types_hash and. column_types perhaps, maybe figure out a way of eliminating @columns_hash?

@parndt
Copy link
Contributor

parndt commented Sep 16, 2013

@tenderlove @SamSaffron I've been using 4-0-stable happily until this was merged, 918a148 is the first 'bad' commit which causes this issue:

$ bundle exec rake db:migrate db:seed
==  CreatePageImages: migrating ===============================================
-- create_table("refinery_image_pages", {:id=>false})
   -> 0.0171s
-- add_index("refinery_image_pages", :image_id)
   -> 0.0038s
-- add_index("refinery_image_pages", :page_id)
   -> 0.0020s
==  CreatePageImages: migrated (0.0309s) ======================================

rake aborted!
An error has occurred, this and all later migrations canceled:

undefined method `[]' for nil:NilClass
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods/read.rb:86:in `block in read_attribute'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods/read.rb:84:in `fetch'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods/read.rb:84:in `read_attribute'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods.rb:243:in `block in attributes'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods.rb:242:in `each'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods.rb:242:in `each_with_object'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods.rb:242:in `attributes'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods/dirty.rb:84:in `keys_for_partial_write'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods/dirty.rb:78:in `create_record'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/callbacks.rb:303:in `block in create_record'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activesupport/lib/active_support/callbacks.rb:373:in `_run__2197611318522249411__create__callbacks'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activesupport/lib/active_support/callbacks.rb:80:in `run_callbacks'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/callbacks.rb:303:in `create_record'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/timestamp.rb:57:in `create_record'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/persistence.rb:467:in `create_or_update'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/callbacks.rb:299:in `block in create_or_update'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activesupport/lib/active_support/callbacks.rb:373:in `_run__2197611318522249411__save__callbacks'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activesupport/lib/active_support/callbacks.rb:80:in `run_callbacks'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/callbacks.rb:299:in `create_or_update'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/persistence.rb:128:in `save!'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/validations.rb:57:in `save!'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/attribute_methods/dirty.rb:41:in `save!'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/transactions.rb:275:in `block in save!'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/transactions.rb:326:in `block in with_transaction_returning_status'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:200:in `transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/transactions.rb:209:in `transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/transactions.rb:323:in `with_transaction_returning_status'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/transactions.rb:275:in `save!'
/Users/parndt/.gem/ruby/2.0.0/gems/protected_attributes-1.0.3/lib/active_record/mass_assignment_security/validations.rb:17:in `create!'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:990:in `record_version_state_after_migrating'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:960:in `block in execute_migration_in_transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:1005:in `block in ddl_transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:202:in `block in transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:210:in `within_new_transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb:202:in `transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/transactions.rb:209:in `transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:1005:in `ddl_transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:958:in `execute_migration_in_transaction'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:920:in `block in migrate'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:916:in `each'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:916:in `migrate'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:764:in `up'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/migration.rb:742:in `migrate'
/Users/parndt/.gem/ruby/2.0.0/bundler/gems/rails-918a148522ed/activerecord/lib/active_record/railties/databases.rake:42:in `block (2 levels) in <top (required)>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)

There are four migrations that should have run but it gets to the end of the first one and fails to continue.

Not sure why, I'll go back to using the ref before this started failing for now.

Thanks! ❤️

@SamSaffron
Copy link
Contributor Author

@parndt is there any chance you can mail me details to a repro at sam.saffron@gmail.com

It appears that self.class.column_types is nil for some reason ... we can easily work around this with

@column_types = self.class.column_types || {} in the inits, just very curious to see what causes that.

cc @tenderlove @rafaelfranca

@parndt
Copy link
Contributor

parndt commented Sep 16, 2013

@SamSaffron here you go https://github.com/parndt/rails-pull-12188

Just clone, bundle, rake db:migrate, kaboom!

@SamSaffron
Copy link
Contributor Author

@parndt I sent through a PR to fix this (and confirmed it fixes your sample)

#12243

@parndt
Copy link
Contributor

parndt commented Sep 16, 2013

@SamSaffron wow thanks for the quick turnaround

@rafaelfranca
Copy link
Member

@parndt make sure you are using latest 4-0-stable and protected_attributes master and you issue will be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants