Benchmarking MRuby vs Ruby

[proyb6]

I’m interested to see a blog on benchmarking between Ruby and MRuby?

[christopheraue]

Not a general benchmark, but I measured the performance of one of my gems which runs in both environments: http://www.rubydoc.info/github/christopheraue/m-ruby-concurrently/file/guides/Performance.md

The gem is written entirely in ruby, no C. Hence it can probably serve as a guideline for general ruby scripts. Long story short: CRuby is roughly twice as fast as mruby. This is mainly because method calls are twice as fast in CRuby. Some aspects are faster in mruby than CRuby though. If I remember correctly, simple stuff like while loops and array operations is faster.

I sort of agree. But it should be a fair comparison e. g. it would be unfair to compare mruby without gems, to MRI with 100 gems, and so on. (This should also be mentioned in the benchmark by the way, so that people can reproduce the results on their machines.)

Perhaps this could be done systematically in the future too. Matz used the 3x3 (or x3) phrase for MRI improvements compared to, I think, ruby 2.0 as the reference. With mjit currently being worked on, perhaps after some time we could get fair comparisons, some kind of benchmarks, to compare mruby and MRI for comparable datasets (both optimized to the target work at hand).

Long story short: CRuby is roughly twice as fast as mruby.

Your comparison is interesting, but to me personally I am missing lots of things that would be important to me.

For example:

• a bunch of .rb files that do some tasks, and to run it under both MRI and mruby, and compare the results.
• a comparison of the main ruby classes, such as Hash, Array and String. In particular Strings.

In bioinformatics, with lots of genomic datasets (strings representing one of the four nucleotides), speed is important, so that is something I am interested in too - loading large datasets, how fast it would be. I assume that one may have to go into C/C++ anyway, or perhaps crystal, but I am interested here in the comparison between MRI and mruby. So I agree with proyb6 - I think it would be interesting if we could do extensive benchmarking + comparison.

Does not necessarily have to be now, but perhaps in the past mjit day of MRI, to see how the two ruby implementations compare to one another. :)

[christopheraue]

Yes, I am interested in a more systematic and detailed comparison myself.

[proyb6]

In addition, I'm interested to see how it fare on VPS cloud with 1-core and multiple cores CPU, this is useful to establish the speed and stability compare to other languages like Go, Swift, Node (V8).

Being new to MRuby from PHP, Swift and Javascript world. I haven't tried MRuby task if that is a built-in feature and the stability it can process in the cloud?

[cookrn]

If it's interesting to folks, I'm very open to contributions of more benchmarks to https://github.com/cookrn/mruby-bench-tool

Cruby 2.6.0-preview2 and Mruby 1.4.1 seem fairly comparable performance-wise.

I have a presentation I can upload there with some visuals as well if desired.