Compile out all calls to process.env.NODE_ENV #5

Closed
wants to merge 2 commits into
from

Projects

None yet

4 participants

@aickin
aickin commented Jan 14, 2017 edited

While PR #3 was a good step towards following production best practices, it only went part way. Implementing prod mode in React rendering on the server has two parts:

  1. Run node with NODE_ENV=production (which was accomplished in #3).
  2. Compile out all calls to process.env.NODE_ENV altogether.

This is because process.env is not a simple JavaScript object, and it is surprisingly expensive to use it. Even when running in production mode, just the calls to check the procces.env.NODE_ENV can easily take 30% of rendering time in React.

In the current master code, this benchmark doesn't compile react-dom/server at all, so its perf is sub-optimal.

In this PR, I've implemented this change for both the renderToString benchmarks and the server benchmarks. To keep things apples to apples, I tried to compile out process.env.NODE_ENV for Rax and Vue, too. However, in Vue's case, I couldn't do it because Vue's renderer is not compatible with webpack. So the Vue code I left basically the same as it was, while I compile Rax and React to remove process.env.NODE_ENV. If anyone has expertise in Vue and can advise about perf optimization for it, I'd be grateful.

The qps benchmark seems to bounce around a lot from run to run (which I will file as a separate issue), but here's the data from a relatively representative run on my machine after these changes:

renderToString qps
Rax 41.2ms 1822 / sec
React 44.8ms 2005 / sec
Vue 45.7ms 1703 / sec

Environment: Retina MBP mid-2014, 2.8GHz i7, 16GB RAM, node 6.9.1

Thanks for all your work, and I'm excited to play around with Rax!

aickin added some commits Jan 14, 2017
@aickin aickin Follow best practices for prod deployment of React, compiling out all…
… calls to process.env.NODE_ENV
4f33893
@aickin aickin Oops. Missed renaming a require, which made the vue server benchmark …
…fail. Didn't see it because I hadn't cleared out my build folder. Sorry.
ec66722
@aickin
aickin commented Jan 14, 2017 edited

Oops, I had a silly bug with the Vue server benchmark that I fixed in ec66722. The bug caused an error in the vue renderer, which made vue run very fast, and therefore biased the test towards Vue.

The change means that unlike my initial belief, vue is the slowest in the qps test, not the fastest (although, as I noted above, I have NO IDEA how to optimize Vue, so take that with a grain of salt). I've updated the numerical results above with a recent run of the benchmark after applying ec66722. Sorry for the mixup.

Also, in case you care about IP issues, I declare that I am the sole author of the code changes in this PR, and I offer them irrevocably to the public domain. Use as you wish.

@STRML
STRML commented Jan 14, 2017 edited

You can also fix this process.env performance issue by doing the following on boot:

process.env = Object.assign({}, process.env);

You will lose the ability to listen to env changes, but I doubt many people are using that.

@aickin
aickin commented Jan 15, 2017

@STRML Yeah, that will work for most cases, but as you note, it could break some code. I prefer to compile the references out because it's more provably correct, but in practice it's probably the same 99.9% of the time.

@yyx990803
Contributor
yyx990803 commented Jan 15, 2017 edited

@aickin I tested applying the same compilation to the Vue case but didn't see significant difference.

On the other hand, I found out that increasing the number of items displayed in mocks seems to skew the results quite a bit.

For example changing the list length to 30 instead of 5:

renderToSring QPS [#/sec]
Rax 101.65ms 589.25
React 117.40ms 944.19
Vue 93.00ms 1039.67

And increasing the list count to 100:

renderToSring QPS [#/sec]
Rax 324.44ms 233.13
React 264.26ms 293.40
Vue 265.57ms 466.28

What I can tell from these numbers:

  • Rax seems to demonstrate a bigger slowdown for larger dataset / markup size

  • Vue demonstrates better QPS due to its rendering being async which avoids the server being blocked on large renders (as seen in the 100 count case). This is also not yet taking streaming mode into account - in my own benchmarks streaming mode provides roughly another ~20% improvement for large renders. I'd be interested to see a comparison between react-dom-stream and Vue's stream mode, but that might be off topic :)

Ultimate I think the numbers will depend heavily on the actual content being rendered and the specific optimization strategies used (e.g. with full streaming mode + component caching in Vue).

@aickin
aickin commented Jan 15, 2017 edited

@yyx990803 Thanks so much for taking time to play with this!

That all mostly makes sense to me, although I'm a little confused about streaming mode being faster than non-streaming. IME streaming tends to add a slight overhead, but I know absolutely nothing about Vue and its SSR, so I'm probably just uninformed.

FWIW, I don't have a high level of trust in the current results of this test because of the reasons I outlined in #6, but it looks like those are getting fixed in #7. (I mean, I still suspect Vue will come out on top, especially for larger workloads, but I'm reserving judgment until I see the numbers πŸ˜„). I'm excited to see the results once those issues have been addressed!

(Side note: Vue has server side component caching??? That's SO cool. Component caching was a feature I prototyped in react-dom-stream on a lark before my talk on React SSR perf last year; was that possibly an inspiration for it in Vue?)

(ETA: To be clear, I'm not trying to take credit for Vue's component caching at all; you implemented it and made it work and should be proud of it! I'm merely interested in how ideas travel and if it was a case of interchange of ideas or "great minds think alike".)

@yyx990803 yyx990803 referenced this pull request Jan 15, 2017
Merged

Use benchmark.js #7

@yyx990803
Contributor
yyx990803 commented Jan 15, 2017 edited

@aickin re streaming: that's what I thought as well, but when I run Vue's own ssr benchmark streaming mode always finishes faster (note this is intentionally benching a big workload). Streaming should also result in better end-user perceived performance because the browser will be able to start rendering sooner in a streaming fashion.

The streaming implementation is indeed heavily inspired by react-dom-stream, so thank you so much for making that work first! Component caching was actually born out of a request by a consulting client - but the cache key interface is also inspired by react-dom-stream (thanks again!)

@imsobear
Collaborator

Closed temporarily. ref to #10. Thank you ~

@imsobear imsobear closed this Jan 17, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment