Skip to content
This repository has been archived by the owner. It is now read-only.

performance issues #287

Closed
doughsay opened this issue Aug 14, 2014 · 22 comments
Closed

performance issues #287

doughsay opened this issue Aug 14, 2014 · 22 comments
Milestone

Comments

@doughsay
Copy link

@doughsay doughsay commented Aug 14, 2014

First let me say that this is an amazing gem, I really enjoy working with it and it really fits my needs exactly. But I've just realized that I might not be able to use it due to performance issues, which is a real shame.

Take this rather contrived example below to see how badly it seems to perform for me:

class Foo
  attr_accessor :foo
end

class Bar
  include Mongoid::Document
  field :bar, :type => String
end

class Baz
  include Virtus.model
  attribute :baz, String
end

time_me = ->(proc) {
  start_time = Time.now.to_f
  proc.call()
  end_time = Time.now.to_f
  time_in_millis = ((end_time - start_time) * 1000).to_i
  puts time_in_millis
}

time_me.call ->{1000.times{ foo = Foo.new; foo.foo = 'foo'; }}
# => 0

time_me.call ->{1000.times{ bar = Bar.new; bar.bar = 'bar'; }}
# => 21

time_me.call ->{1000.times{ baz = Baz.new; baz.baz = 'baz'; }}
# => 592

As you can see, it appears Virtus models perform 30x slower (than mongoid documents) in simple cases like this, where mongoid is providing the same functionality, i.e. attribute definition and type coercion.

Basically in my case, what I'm seeing is Virtus taking almost 2 seconds to "hydrate" less than 10 models each with about 100 attributes. This makes it pretty much unusable... I really hope there's something that can be done about it because I'd love to be able to use Virtus.

@coladarci
Copy link

@coladarci coladarci commented Aug 14, 2014

👍 - such a shame..

@solnic solnic added this to the 2.0.0 milestone Aug 14, 2014
@solnic
Copy link
Owner

@solnic solnic commented Aug 14, 2014

This will be addressed in virtus 2.0. Poor performance is probably caused by its coercion backend which supports more types of coercions than Mongoid and relies on stuff like regexp parsing which is just very slow.

I'm marking this issue as a todo for 2.0.

@coladarci
Copy link

@coladarci coladarci commented Aug 14, 2014

@solnic, despite it probably not being a good idea, in your estimation, how complicated would it be for us to fork and remove the extra bloat if we don't need all of the extra support for the types? (in general we stay away from this method, but we do love this gem...)

@elskwid
Copy link
Collaborator

@elskwid elskwid commented Aug 14, 2014

@doughsay - I don't see the same results here. Take a look at these numbers after running this gist:

> be ruby coerce-speed.rb
Rehearsal -------------------------------------------------------
Poro                  0.020000   0.000000   0.020000 (  0.025502)
MongoidDoc            0.460000   0.000000   0.460000 (  0.464085)
VirtusModel           0.210000   0.010000   0.220000 (  0.219621)
VirtusModelNoCoerce   0.060000   0.000000   0.060000 (  0.062512)
---------------------------------------------- total: 0.760000sec

                          user     system      total        real
Poro                  0.010000   0.000000   0.010000 (  0.004206)
MongoidDoc            0.420000   0.010000   0.430000 (  0.430198)
VirtusModel           0.210000   0.000000   0.210000 (  0.242876)
VirtusModelNoCoerce   0.050000   0.000000   0.050000 (  0.056276)
@elskwid
Copy link
Collaborator

@elskwid elskwid commented Aug 14, 2014

@doughsay - that wasn't to say that what you're seeing isn't happening, just that I couldn't see it in my example. 2 seconds for 10 models sounds like a lot of time.

@coladarci - as you can see in my gist above, you can opt out of coercion with Virtus.model(coerce: false). Is that enough?

@doughsay
Copy link
Author

@doughsay doughsay commented Aug 14, 2014

@elskwid - Thanks a lot for that, this indicates that it might be something else on my end causing the slowdown. That's good news; I'll do some more investigation on my end.

My results from your gist:
ruby-1.9.3-p547 + mongoid-3.1.6 + virtus-1.0.3

Rehearsal -------------------------------------------------------
Poro                  0.010000   0.000000   0.010000 (  0.003283)
MongoidDoc            0.280000   0.000000   0.280000 (  0.283630)
VirtusModel           0.480000   0.010000   0.490000 (  0.506913)
VirtusModelNoCoerce   0.030000   0.000000   0.030000 (  0.030868)
---------------------------------------------- total: 0.810000sec

                          user     system      total        real
Poro                  0.010000   0.000000   0.010000 (  0.003304)
MongoidDoc            0.270000   0.000000   0.270000 (  0.263741)
VirtusModel           0.480000   0.010000   0.490000 (  0.495392)
VirtusModelNoCoerce   0.040000   0.000000   0.040000 (  0.041859)

ruby-2.1.2 + mongoid-4.0.0 + virtus-1.0.3

Rehearsal -------------------------------------------------------
Poro                  0.010000   0.000000   0.010000 (  0.004674)
MongoidDoc            0.330000   0.000000   0.330000 (  0.329595)
VirtusModel           0.130000   0.000000   0.130000 (  0.135011)
VirtusModelNoCoerce   0.040000   0.000000   0.040000 (  0.042119)
---------------------------------------------- total: 0.510000sec

                          user     system      total        real
Poro                  0.000000   0.000000   0.000000 (  0.004560)
MongoidDoc            0.300000   0.000000   0.300000 (  0.304607)
VirtusModel           0.130000   0.010000   0.140000 (  0.141968)
VirtusModelNoCoerce   0.040000   0.000000   0.040000 (  0.033637)

Ruby 2 is definitely faster.

@elskwid
Copy link
Collaborator

@elskwid elskwid commented Aug 14, 2014

@doughsay - Excellent! Let me know if you need a hand or a second pair of eyes on something. I find this kind of thing very interesting. Good luck!

@doughsay
Copy link
Author

@doughsay doughsay commented Aug 14, 2014

@elskwid - I've figure out that it must come from rails, or at least some other gem we're including in our project. I added your gist as a rake task in our project and got this result:

Rehearsal -------------------------------------------------------
Poro                  0.010000   0.000000   0.010000 (  0.004219)
MongoidDoc            0.270000   0.020000   0.290000 (  0.300970)
VirtusModel           4.150000   0.020000   4.170000 (  4.219307)
VirtusModelNoCoerce   0.040000   0.000000   0.040000 (  0.041164)
---------------------------------------------- total: 4.510000sec

                          user     system      total        real
Poro                  0.000000   0.000000   0.000000 (  0.004330)
MongoidDoc            0.220000   0.000000   0.220000 (  0.227933)
VirtusModel           3.880000   0.010000   3.890000 (  3.919318)
VirtusModelNoCoerce   0.040000   0.000000   0.040000 (  0.037428)

Any idea what could be causing this? Also, I don't really know how to continue my investigation from this point...

This is rails 3.2.13, and those results were using ruby-2.1.2, but we usually run this project with 1.9.3.

@doughsay
Copy link
Author

@doughsay doughsay commented Aug 14, 2014

Found it! gem 'better_errors', '1.1.0' causes performance to go to 💩

@elskwid
Copy link
Collaborator

@elskwid elskwid commented Aug 14, 2014

Oh snap! Great detective work. I wonder if it's actually binding_of_caller since that gets in the callstack.

@doughsay
Copy link
Author

@doughsay doughsay commented Aug 15, 2014

Yep, that's it. You can still include better_errors, just not binding_of_caller. Good call.

@elskwid
Copy link
Collaborator

@elskwid elskwid commented Aug 15, 2014

@doughsay - I'm going to close this. Let us know if you run into any more trouble.

@elskwid elskwid closed this Aug 15, 2014
@skade
Copy link

@skade skade commented Aug 15, 2014

To add an anecdote (I have to dig for numbers), we found that the coercer plays really well with the JRuby JIT.

@jgdreyes
Copy link

@jgdreyes jgdreyes commented Nov 26, 2014

@doughsay - I was seeing performance issues with Virtus as well. Though I love the coercion, type specifier and default values for my form objects, I don't always need these features. I ran a benchmark as well (outside of rails) using value objects that are mutable: https://github.com/codeshoppe/simple_attrs_benchmark. I found that Virtus still performs slower than a few other gems.

I still use Virtus for form_objects and harness all of its powerful features, but I have switched to simple_attrs for a simple attribute DSL.

(Note that the performance issues I see worsen as I add more attributes.)

@solnic
Copy link
Owner

@solnic solnic commented Nov 26, 2014

@jgdreyes performance will be addressed in virtus 2.0 rewrite, there are lots of features that I think should be removed, some features that should be optional and probably a couple of low-hanging fruits which could speed things up. Stay tuned :)

@jgdreyes
Copy link

@jgdreyes jgdreyes commented Nov 26, 2014

@solnic I did read that and I'm super excited. I really do love Virtus and still use it in production. It fits my needs perfectly for most scenarios.

Specifically I'd like to see performance improvements as the number of attributes grow within a single virtus model. If you look at the benchmark code I added in my previous comment, I instantiate 100,000 objects with 25 attributes. The use case for this is to consume an external API that returns back an array of hashes. I'd like to work with plain old ruby objects instead of hashes so that I can layer on additional functionality to each record. However, with my little spike it was clear that the performance wasn't acceptable. Temporarily I've switched to using a simple DSL, but would love to continue using Virtus. :) Keep up the great work and thanks for the reply!

@mbj
Copy link
Collaborator

@mbj mbj commented Nov 27, 2014

@solnic I'd be interested in a small pairing where we figure out if morpher can help you to reduce the code and actually speed stuff up for 2.0. Morpher is pretty fast in non evaluation history mode and allows the VM to inline most of the method calls (Jruby / rbx), even on MRI I was not able to measure a difference from hand written code for complex nested transforms. Are you available for such a pairing this / next week?

@solnic
Copy link
Owner

@solnic solnic commented Nov 27, 2014

@mbj thanks Markus, before I dive into any coding I need to define what Virtus 2.0 is supposed to be :) let me first come up with a list of things I want it to support and then we'll be able to figure out how morpher fits here. Does that sound reasonable?

@mbj
Copy link
Collaborator

@mbj mbj commented Nov 27, 2014

@solnic Yeah it does. My point was to give you some examples on how to map virtus 1.0 to morpher + anima. To give you the ideas you need for 2.0 ;)

@solnic
Copy link
Owner

@solnic solnic commented Nov 27, 2014

@mbj virtus 1.0 can't be used with anima because it'd break its public interface, that's why I plan to use morpher + maybe anima in 2.0. Main problem is that anima requires all keys to be present in the attribute hash that's passed to the constructor and probably a couple more differences

@mbj
Copy link
Collaborator

@mbj mbj commented Nov 27, 2014

@solnic I was not saying you should use virtus 1.0 backed by anima + morpher, I was saying we should talk on how it could be used to see a path to 2.0 as an explorational thought experiment.

@mbj
Copy link
Collaborator

@mbj mbj commented Nov 27, 2014

@solnic BTW computing absent keys is very easy and a total morpher transformer in front of the anima instantiation can easily fill the gabs with nil valued keys.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants
You can’t perform that action at this time.