New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Big performance overhead for Opal #520

Closed
davispuh opened this Issue Mar 23, 2014 · 18 comments

Comments

Projects
None yet
7 participants
@davispuh
Contributor

davispuh commented Mar 23, 2014

Basically JS generated by Opal is more than 100x slower (in my specific test) than just writing pure JS, so it seems all performance important code (heavy loops etc) must be written in JS. I also found that Ruby itself is quite slow when comparing to JS :(

Let's say I want to find next Prime number after a specified number, in my tests I used 1294268491 as starting point and so next prime is 1294268779

Results

Firefox 31

Implementation runs per sec error runs sampled
asm-js 1 204 ±0.43% 62
js 1 171 ±0.75% 60
opal 4.58 ±4.19% 15

Chrome 33

Implementation runs per sec error runs sampled
asm-js 1 215 ±0.32% 53
js 1 186 ±0.73% 53
opal 2.76 ±21.96% 12

Internet Explorer 11

Implementation runs per sec error runs sampled
asm-js 128 ±0.52% 56
js 153 ±0.93% 53
opal 8.86 ±2.54% 27

Ruby 2.0 (MRI)

runs per sec runs sampled
16.5 20

Here's Ruby code

module PrimeFinder
  # Check if given number is a Prime
  # @param [Fixnum] n number to be checked
  # @return [Fixnum]
  def self.isPrime?(n)
    to = Math.sqrt(n).to_i
    2.upto(to) do |i|
      return 0 if n % i == 0
    end
    1
  end

  # Find first prime after n
  # @param [Fixnum] n number after which start to look
  # @return [Fixnum]
  def self.findPrime(n)
    loop do
      n = n + 1
      return n if isPrime?(n) > 0
    end
    0
  end

end

And my implementation in asm.js (to test non-asm, just comment out "use asm")

function PrimeFinder(stdlib) {
  "use asm";

  var sqrt = stdlib.Math.sqrt;
  var floor = stdlib.Math.floor;

  function isPrime(n) {
    n = n | 0;
    var i = 2;
    var to = 0;
    to = ~~(+floor(+sqrt(+(n | 0))));
    do {
      if ((((n | 0) % (i | 0)) | 0) == 0) return 0;
      i = (i + 1) | 0;
    } while ((i | 0) <= (to | 0));
    return 1;
  }

  function findPrime(n) {
    n = n | 0;
    while (1) {
     n = (n + 1) | 0;
     if (isPrime(n | 0) | 0) return n | 0;
    }
    return 0;
  }
  return { isPrime: isPrime,  findPrime: findPrime};
}

To use PrimeFinder(window).findPrime(1294268491);

If that Ruby code could compile to this JS (doesn't even need to be asm.js) that would be really awesome as then this Ruby JS would be way faster than Ruby MRI 😄

So anyway because Ruby language have great meta-programming capabilities and very dynamic nature I guess it's not really possible to directly translate it and that's why I propose special directive for compiling limited subset of Ruby to fast JS (or preferably asm.js). Idea is that if Ruby module have # "use asm" directive and if that module uses only stdlib functions which are compatible with asm.js (such as Math) then compile it to this effective asm.js and then it would be possible to have really high-performance things written in nice Ruby and executing way faster in JS than with MRI.

Also it looks like in this specific test asm.js doesn't really give much improvement in speed thus just pure JS would be fine too, but I think in some cases it might give better improvement and that's why should better go for asm.js.

@vendethiel

This comment has been minimized.

Show comment
Hide comment
@vendethiel

vendethiel Mar 23, 2014

Contributor

The asm version is probably slower without the "use asm" due to the | 0 etc. I know v8 optimizes by pattern, but ironmonkey optimizes mostly by "use asm".

Contributor

vendethiel commented Mar 23, 2014

The asm version is probably slower without the "use asm" due to the | 0 etc. I know v8 optimizes by pattern, but ironmonkey optimizes mostly by "use asm".

@elia

This comment has been minimized.

Show comment
Hide comment
@elia

elia Mar 23, 2014

Member

@davispuh interesting! do you have the full perf script somewhere? (e.g. on jsperf)

Member

elia commented Mar 23, 2014

@davispuh interesting! do you have the full perf script somewhere? (e.g. on jsperf)

@vendethiel

This comment has been minimized.

Show comment
Hide comment
@vendethiel

vendethiel Mar 23, 2014

Contributor

Also, use asm is probably too narrow to be really useful.

Contributor

vendethiel commented Mar 23, 2014

Also, use asm is probably too narrow to be really useful.

@adambeynon

This comment has been minimized.

Show comment
Hide comment
@adambeynon

adambeynon Mar 23, 2014

Contributor

This test is hammering opal at the thing opal is bad at: math operators. We compile math ops into real method calls, which are great magnitudes slower than native js math ops. This is definitely a use case that you don't want to use opal for (unless you use xstrings). Still, interesting to see the speed comparisons.

Contributor

adambeynon commented Mar 23, 2014

This test is hammering opal at the thing opal is bad at: math operators. We compile math ops into real method calls, which are great magnitudes slower than native js math ops. This is definitely a use case that you don't want to use opal for (unless you use xstrings). Still, interesting to see the speed comparisons.

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh Mar 23, 2014

Member

Yes, this is known, if you want to do math either use xstrings or don't use Opal.

Same issue with Ruby really, every operator call is done at runtime, there's no way to even optimize 2 + 2 in Ruby, and that should say a lot.

There were discussions about a pragma to disable operators as method calls or some kind of DSL, but it's really not worthwhile when you can just write pure JavaScript to optimize the hot path.

Member

meh commented Mar 23, 2014

Yes, this is known, if you want to do math either use xstrings or don't use Opal.

Same issue with Ruby really, every operator call is done at runtime, there's no way to even optimize 2 + 2 in Ruby, and that should say a lot.

There were discussions about a pragma to disable operators as method calls or some kind of DSL, but it's really not worthwhile when you can just write pure JavaScript to optimize the hot path.

@davispuh

This comment has been minimized.

Show comment
Hide comment
@davispuh

davispuh Mar 23, 2014

Contributor

No, there's really no much difference between pure JS and asm.js, FF tells if it compiles to asm.js and commenting it out can see it doesn't and even if you remove all |0 and just make it like normal JS it's still almost same performance.

I just created http://jsperf.com/opal you can check it out

Contributor

davispuh commented Mar 23, 2014

No, there's really no much difference between pure JS and asm.js, FF tells if it compiles to asm.js and commenting it out can see it doesn't and even if you remove all |0 and just make it like normal JS it's still almost same performance.

I just created http://jsperf.com/opal you can check it out

@vendethiel

This comment has been minimized.

Show comment
Hide comment
@vendethiel

vendethiel Mar 23, 2014

Contributor

I think YARV generates a different opcode for operators to get a speed boost though, doesn't it ?

Contributor

vendethiel commented Mar 23, 2014

I think YARV generates a different opcode for operators to get a speed boost though, doesn't it ?

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh Mar 23, 2014

Member

@Nami-Doc with some tracing it can infer the operator hasn't been redefined, otherwise it can't do much.

If you redefine Fixnum#+ you can't fold 2 + 2 without giving a wrong result.

Member

meh commented Mar 23, 2014

@Nami-Doc with some tracing it can infer the operator hasn't been redefined, otherwise it can't do much.

If you redefine Fixnum#+ you can't fold 2 + 2 without giving a wrong result.

@vendethiel

This comment has been minimized.

Show comment
Hide comment
@vendethiel

vendethiel Mar 23, 2014

Contributor

Sure, but a JIT should be able to optimize Fixnum#+ and deoptimize it if it's redefine

Contributor

vendethiel commented Mar 23, 2014

Sure, but a JIT should be able to optimize Fixnum#+ and deoptimize it if it's redefine

@davispuh

This comment has been minimized.

Show comment
Hide comment
@davispuh

davispuh Mar 23, 2014

Contributor

Well, it would be really nice to write in Ruby even if with just small subset as Ruby's syntax IMO is better/prettier than JS's besides asm.js compiling could be also very handy for some high-performance parts as writing that manually is big pain. Of course this all probably doesn't matter to 99% of apps, but I just thought to check how big difference there are and I really didn't expected that it would be so slow. I think this might be worth implementing in future as opal is already compiling to JS so it could compile to asm.js too in some cases. I wouldn't say asm.js is limited, it's important for exactly these kinds of tasks which are CPU intensive, ie. math. Useful for hashing/cryptography and such.

Contributor

davispuh commented Mar 23, 2014

Well, it would be really nice to write in Ruby even if with just small subset as Ruby's syntax IMO is better/prettier than JS's besides asm.js compiling could be also very handy for some high-performance parts as writing that manually is big pain. Of course this all probably doesn't matter to 99% of apps, but I just thought to check how big difference there are and I really didn't expected that it would be so slow. I think this might be worth implementing in future as opal is already compiling to JS so it could compile to asm.js too in some cases. I wouldn't say asm.js is limited, it's important for exactly these kinds of tasks which are CPU intensive, ie. math. Useful for hashing/cryptography and such.

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh Mar 24, 2014

Member

@Nami-Doc from what I recall YARV wasn't that advanced, when you also take into consideration they went the opposite direction of Rubinius and went with everything in C, I'm pretty sure Rubinius does that kind of stuff tho.

@davispuh the problem is pragmas complicate the compiler by a long shot, and something like an Opal.math helper would be kind of ugly on the compiler side.

Your best bet for now is to just write the hot paths in inlined JS, and call the methods from the Opal code.

Member

meh commented Mar 24, 2014

@Nami-Doc from what I recall YARV wasn't that advanced, when you also take into consideration they went the opposite direction of Rubinius and went with everything in C, I'm pretty sure Rubinius does that kind of stuff tho.

@davispuh the problem is pragmas complicate the compiler by a long shot, and something like an Opal.math helper would be kind of ugly on the compiler side.

Your best bet for now is to just write the hot paths in inlined JS, and call the methods from the Opal code.

@davispuh

This comment has been minimized.

Show comment
Hide comment
@davispuh

davispuh Mar 24, 2014

Contributor

I forgot to mention, because Ruby is dynamic language and you can't tell what data types for parameters method uses until run-time, I had idea that it could use documentation

  # @param [Fixnum] n number to be checked
  # @return [Fixnum]
  def self.isPrime?(n)

And so Fixnum would be n = n | 0;

Anyway this is just an idea, maybe not suitable for opal and there might be place for other Ruby to asm.js compiler for this. Although Ruby itself might not be best language for compiling to asm.js because of dynamic types as in asm.js you need to know them beforehand.

It's not really big deal as always can write in JS things that are slow in Opal but my main idea is about asm.js compiling. It seems currently you can only compile C/C++ to asm.js

Contributor

davispuh commented Mar 24, 2014

I forgot to mention, because Ruby is dynamic language and you can't tell what data types for parameters method uses until run-time, I had idea that it could use documentation

  # @param [Fixnum] n number to be checked
  # @return [Fixnum]
  def self.isPrime?(n)

And so Fixnum would be n = n | 0;

Anyway this is just an idea, maybe not suitable for opal and there might be place for other Ruby to asm.js compiler for this. Although Ruby itself might not be best language for compiling to asm.js because of dynamic types as in asm.js you need to know them beforehand.

It's not really big deal as always can write in JS things that are slow in Opal but my main idea is about asm.js compiling. It seems currently you can only compile C/C++ to asm.js

@meh

This comment has been minimized.

Show comment
Hide comment
@meh

meh Mar 24, 2014

Member

@davispuh compiling to asm.js is kind of pointless for Opal, since we'd have to write our own garbage collector etc.

Member

meh commented Mar 24, 2014

@davispuh compiling to asm.js is kind of pointless for Opal, since we'd have to write our own garbage collector etc.

@adambeynon

This comment has been minimized.

Show comment
Hide comment
@adambeynon

adambeynon May 7, 2014

Contributor

This big overhead just comes from the math ops, as they are method calls rather than "low level" javascript math ops. We could (and used to) otptimize this by compiling an inline if-else check to try and optimize these inline. This reduced math op overhead to being 2x as slow, rather than 100x as slow. However, the ugly code it generates is not worth it. Ruby is also very slow at prime numbers (using a simple mri benchmark). If you are doing anything math based, then it is best to use x-strings.

Contributor

adambeynon commented May 7, 2014

This big overhead just comes from the math ops, as they are method calls rather than "low level" javascript math ops. We could (and used to) otptimize this by compiling an inline if-else check to try and optimize these inline. This reduced math op overhead to being 2x as slow, rather than 100x as slow. However, the ugly code it generates is not worth it. Ruby is also very slow at prime numbers (using a simple mri benchmark). If you are doing anything math based, then it is best to use x-strings.

@adambeynon adambeynon closed this May 7, 2014

@giuan

This comment has been minimized.

Show comment
Hide comment
@giuan

giuan Nov 10, 2014

I see that the issue is close, but i don't understand why is not possible to optimize aritmetic.
I think that nobody redefine math operator for number.

Anyway opal is good!

fib is 181 x slower
fibjs manually optimized is 26 x slower

def fib(n)
  return 1 if n < 2
  fib(n-1) + fib(n-2)
end

def fibjs(n)
  return 1 if `n < 2`
  `this.$fibjs(n-1) + this.$fibjs(n-2)`
end

require 'benchmark'

n = 29
fib(n); fibjs(n);
puts Benchmark.measure {puts "fib opal #{fib(n)}" }
puts Benchmark.measure {puts "fib js #{fibjs(n)}" }

%x{
    function fib_pure(n) {
      if (n < 2) return 1
      return fib_pure(n-1) + fib_pure(n-2)
    }
}
puts "Pure javascript"
puts Benchmark.measure {
    `fib_pure(n)`
}

/* Generated by Opal 0.7.0.beta2 */
fib opal 832040
1.632
fib js 832040
0.234
Pure javascript
0.009

giuan commented Nov 10, 2014

I see that the issue is close, but i don't understand why is not possible to optimize aritmetic.
I think that nobody redefine math operator for number.

Anyway opal is good!

fib is 181 x slower
fibjs manually optimized is 26 x slower

def fib(n)
  return 1 if n < 2
  fib(n-1) + fib(n-2)
end

def fibjs(n)
  return 1 if `n < 2`
  `this.$fibjs(n-1) + this.$fibjs(n-2)`
end

require 'benchmark'

n = 29
fib(n); fibjs(n);
puts Benchmark.measure {puts "fib opal #{fib(n)}" }
puts Benchmark.measure {puts "fib js #{fibjs(n)}" }

%x{
    function fib_pure(n) {
      if (n < 2) return 1
      return fib_pure(n-1) + fib_pure(n-2)
    }
}
puts "Pure javascript"
puts Benchmark.measure {
    `fib_pure(n)`
}

/* Generated by Opal 0.7.0.beta2 */
fib opal 832040
1.632
fib js 832040
0.234
Pure javascript
0.009
@elia

This comment has been minimized.

Show comment
Hide comment
@elia

elia Nov 10, 2014

Member

There is a new compiler/build flag you can use to enable math ops optimizations which is called inline_operators (-V from the command line).

Member

elia commented Nov 10, 2014

There is a new compiler/build flag you can use to enable math ops optimizations which is called inline_operators (-V from the command line).

@eregon

This comment has been minimized.

Show comment
Hide comment
@eregon

eregon Oct 19, 2015

Contributor

@meh, @vendethiel For the record, YARV treats some math operators like + specially by just checking a flag if the Fixnum#+ method was not redefined and then performs the computation in place, avoiding call overhead. A JIT compiler can assume Fixnum#+ is not redefined in compiled code and deopt if it is, leading to close to 0 overhead.

Contributor

eregon commented Oct 19, 2015

@meh, @vendethiel For the record, YARV treats some math operators like + specially by just checking a flag if the Fixnum#+ method was not redefined and then performs the computation in place, avoiding call overhead. A JIT compiler can assume Fixnum#+ is not redefined in compiled code and deopt if it is, leading to close to 0 overhead.

@vendethiel

This comment has been minimized.

Show comment
Hide comment
@vendethiel

vendethiel Oct 19, 2015

Contributor

Thanks @eregon. I think there was a bit about this in "ruby under a microscope" as well – interesting stuff :).

Contributor

vendethiel commented Oct 19, 2015

Thanks @eregon. I think there was a bit about this in "ruby under a microscope" as well – interesting stuff :).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment