Huge Discrepancy Between Native Go and Compiled JS Performance #192

Closed
albrow opened this Issue Mar 6, 2015 · 9 comments

Projects

None yet

3 participants

@albrow
Contributor
albrow commented Mar 6, 2015

This is possibly related to #142, but I'm not sure because I don't know the underlying cause. I'm also dealing with a much lighter workload.

I built vdom, a virtual DOM and diff algorithm written in go that is intended to be compiled with gopherjs and run in the browser. We're going to use it to improve rendering performance for humble.

Unfortunately, the compiled code is not performing as well as I had hoped. The go version performs fine, but the compiled javscript takes 70-75x longer on my machine. It actually turns out slower than setting innerHTML for larger templates, which defeats the entire purpose of the library :(

BenchmarkParse BenchmarkXMLDecode BenchmarkDiff
Native Go 23932 ns/op 9193 ns/op 3307 ns/op
Gopherjs 1814000 ns/op 677500 ns/op 253400 ns/op

The benchmarks are not doing anything very intense. It's working with a ul with three items. The benchmark code is here.

Is this kind of discrepancy to be expected? Is there anything I can do to improve the performance of the gopherjs generated code? Or if there's nothing I can do, are there any optimizations that gopherjs can do to narrow the gap? I would really love to get this library working fast enough so we can continue down the path of using gopherjs for client-side apps!

Let me know if you need any additional information.

@neelance
Member
neelance commented Mar 7, 2015

I just tried your benchmarks with the old version 30fcbd8 and it was much faster. This is the last version before getting rid of the //gopherjs:blocking requirement. I have drafted a proposal in #193 which should enable you to make your code faster with the new versions. If it is okay for you I would like to close this issue until I've implemented #193, then we can check again. :-)

@albrow
Contributor
albrow commented Mar 8, 2015

Thank you for taking a look at this! I'll comment on #193 directly but I am probably in favor of it.

For reference, I compiled with version 30fcbd8 and compared the results:

BenchmarkParse BenchmarkXMLDecode BenchmarkDiff
Native Go 23932 ns/op 9193 ns/op 3307 ns/op
Assume Blocking 1814000 ns/op 677500 ns/op 253400 ns/op
Assume Non-Blocking 1212000 ns/op 411333 ns/op 222400 ns/op
Native/Non-Blocking 50.6 44.7 67.2

While it's certainly better, the compiled js version is still much slower than the native go version. I'm currently writing some in-browser benchmarks to compare Parse+Diff+Patch to just setting innter html. I will report back soon. Can we keep the issue open until I can report those numbers?

@shurcooL
Member
shurcooL commented Mar 8, 2015

While it's certainly better, the compiled js version is still much slower than the native go version.

I expect that's normal to some degree. JavaScript, even the best hand-written and tuned for performance, will not be faster than native compiled Go code. There are some unavoidable overheads due to type conversions and if you want to interact with the JavaScript world. How much slower it is is something that can be improved (and has been improved compared to earlier versions of GopherJS).

@albrow
Contributor
albrow commented Mar 8, 2015

Okay. As @neelance guessed, my version of node was outdated, which affects the results to a surprising degree. Here's with the latest version of node (0.12.0):

BenchmarkParse BenchmarkXMLDecode BenchmarkDiff
Native Go 23914 ns/op 9056 ns/op 3257 ns/op
Assume Blocking 1836000 ns/op 1011500 ns/op 142700 ns/op
Assume Non-Blocking 458000 ns/op 170500 ns/op 50133 ns/op
Native/Non-Blocking 19.16 18.83 15.39

Instead of 75x slower its closer to 15-20x slower which is much more reasonable.

@shurcooL, I understand some performance difference is expected. I just wasn't expecting the discrepancy to be as huge as the post-30fcbd8 benchmarks. These benchmarks show javascript V8 as anywhere from 9x slower to 14x faster, but usually 2-3x slower. I also understand there is a decent amount of unavoidable overhead in the gopherjs-generated code. It's also awesome that gopherjs performance has been continuously improving!

@neelance thanks again for taking a look at this! I'm consistently impressed with your response to issues. If it's okay with you, I'd still like to hold off on closing this until I can report the in-browser benchmarks compared to SetInnerHTML. That represents a hard performance constraint for me as opposed to what is admittedly a sort of arbitrary comparison to native go performance. Working on this now.

@neelance
Member
neelance commented Mar 9, 2015

Please try again now. :-)

@albrow
Contributor
albrow commented Mar 9, 2015

@neelance I'm getting a runtime error with the latest version. It doesn't occur in version 9cd1448, which is just prior to the latest version.

Here's the output of gopherjs test github.com/albrow/vdom --run=none --bench=.

/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:1400
      throw err;
            ^
TypeError: undefined is not a function
    at $packages.syscall.$pkg.Syscall6 (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:5555:8)
    at $packages.syscall.sysctl (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:5935:12)
    at $packages.syscall.nametomib (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:5775:9)
    at Object.$packages.syscall.$pkg.Sysctl (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:5735:12)
    at $packages.os.init$3 (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:10141:20)
    at Object.$packages.os.$pkg.$init (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:10311:3)
    at Object.$packages.fmt.$pkg.$init (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:19359:11)
    at Object.$packages.encoding/xml.$pkg.$init (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:23556:12)
    at Object.$packages.github.com/albrow/vdom.$pkg.$init (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:33137:12)
    at $packages.main.$pkg.$init (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:38651:13)
    at goroutine (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:1390:19)
    at $schedule (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:1430:5)
    at $go (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:1419:3)
    at Object.<anonymous> (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:38666:1)
    at Object.<anonymous> (/private/var/folders/jb/68dw2ksn5y1_61tslh_66mcc0000gn/T/test.293305114:38669:4)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Function.Module.runMain (module.js:501:10)
    at startup (node.js:129:16)
    at node.js:814:3

@neelance
Member
neelance commented Mar 9, 2015

Fixed, please give it another shot.

@albrow
Contributor
albrow commented Mar 9, 2015

Yep it works now. Here's the results from the latest version:

BenchmarkParse BenchmarkXMLDecode BenchmarkDiff
Native Go 23914 ns/op 9056 ns/op 3257 ns/op
Assume Blocking 1836000 ns/op 1011500 ns/op 142700 ns/op
Assume Non-Blocking 458000 ns/op 170500 ns/op 50133 ns/op
Optimized Blocking 562500 ns/op 274000 ns/op 70350 ns/op

This is a big improvement! As you expected, it's almost as fast as the assume non-blocking version.

Forget what I said about benchmarking against SetInnerHTML and using that as a performance target. I've learned there are too many confounding variables (e.g. the amount of css, the other elements on the page) for a synthetic benchmark to be representative. Instead I plan to see how it performs in real apps.

I'm going to continue to optimize vdom, and my team and I will be developing some real apps with humble over the next few months. Can I report back if we hit any performance roadblocks? I think the current version will be fast enough for most use cases, but I can't say for sure without building some more things out. For now I think this issue can be closed.

@neelance
Member
neelance commented Mar 9, 2015

I'm glad that you're happy. :-) Of course you can always create a new issue if you see cases where performance improvements might be possible.

@neelance neelance closed this Mar 9, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment