Skip to content
This repository has been archived by the owner on Jun 2, 2021. It is now read-only.

improve: use lower case header #4

Merged
merged 1 commit into from
Aug 31, 2016
Merged

improve: use lower case header #4

merged 1 commit into from
Aug 31, 2016

Conversation

fengmk2
Copy link
Contributor

@fengmk2 fengmk2 commented Aug 31, 2016

No description provided.

@fengmk2
Copy link
Contributor Author

fengmk2 commented Aug 31, 2016

@zensh header 越多效果越明显 eggjs/egg-security#3

benchmark https://gist.github.com/fengmk2/1634043ec1d40e09b53197b817411131

@coveralls
Copy link

coveralls commented Aug 31, 2016 •
edited
Loading

Coverage Status

Coverage remained the same at 99.545% when pulling 08f3798 on fengmk2:use-lower-case-header into 4d08f1d on toajs:master.

@zensh zensh merged commit 2cf39b3 into toajs:master Aug 31, 2016
@zensh
Copy link
Member

zensh commented Aug 31, 2016

@fengmk2 之前看 HTTP Header Fields 相关文档，以为大小写形式是标准，就全部改成大小写形式了。还是要多看规范~

@fengmk2 fengmk2 deleted the use-lower-case-header branch August 31, 2016 06:31
@fengmk2
Copy link
Contributor Author

fengmk2 commented Aug 31, 2016

规范是不区分大小的。

@fengmk2
Copy link
Contributor Author

fengmk2 commented Aug 31, 2016

@zensh 又可以跟 koa 性能拉开距离了。

@zensh
Copy link
Member

zensh commented Aug 31, 2016

感觉的确涨了：

  koa, 1 middleware:  5453.33
  koa, 5 middleware:  5407.36
  koa, 10 middleware:  5251.76
  koa, 15 middleware:  5132.42
  koa, 20 middleware:  5269.80
  koa, 30 middleware:  4800.27
  koa, 50 middleware:  4403.85
  koa, 100 middleware:  3759.32

  toa, 1 middleware:  7893.65
  toa, 5 middleware:  7660.13
  toa, 10 middleware:  7304.92
  toa, 15 middleware:  6951.81
  toa, 20 middleware:  6656.89
  toa, 30 middleware:  6290.28
  toa, 50 middleware:  5587.16
  toa, 100 middleware:  4556.21

@fengmk2
Copy link
Contributor Author

fengmk2 commented Aug 31, 2016

要对比之前的版本才能看到效果

@fundon
Copy link

fundon commented Sep 1, 2016

@zensh 在本地跑了几回 benchmarks，koa 和 toa 的数据很接近，没跑出优势出来（姿势不对吗？）

机器：

MacBook Pro (Retina, 13-inch, Late 2012)
Processor 2.5 GHz Intel Core i5
Memory 8 GB 1600 MHz DDR3
Graphics Intel HD Graphics 4000 1536 MB

  koa, 1 middleware:  5625.95

  koa, 5 middleware:  5432.86

  koa, 10 middleware:  5418.29

  koa, 15 middleware:  5049.00

  koa, 20 middleware:  5059.07

  koa, 30 middleware:  4950.70

  koa, 50 middleware:  4529.12

  koa, 100 middleware:  3896.83

  toa, 1 middleware:  5418.43

  toa, 5 middleware:  5348.51

  toa, 10 middleware:  5152.05

  toa, 15 middleware:  4885.31

  toa, 20 middleware:  4820.79

  toa, 30 middleware:  4545.88

  toa, 50 middleware:  4083.60

  toa, 100 middleware:  3097.28

@zensh
Copy link
Member

zensh commented Sep 1, 2016

上面贴的是 v6.4 下的结果，v4.5 下的结果是：（MacBook Pro 2.4 GHz Intel Core i5）

koa, 1 middleware: 5004.12

koa, 5 middleware: 4934.42

koa, 10 middleware: 5010.89

koa, 15 middleware: 4852.47

koa, 20 middleware: 4776.92

koa, 30 middleware: 4660.21

koa, 50 middleware: 4466.62

koa, 100 middleware: 3998.61

toa, 1 middleware: 6168.07

toa, 5 middleware: 6047.31

toa, 10 middleware: 6077.15

toa, 15 middleware: 5837.98

toa, 20 middleware: 5710.34

toa, 30 middleware: 5386.09

toa, 50 middleware: 5163.73

toa, 100 middleware: 4351.49

你用的 node 什么版本？

@fundon
Copy link

fundon commented Sep 1, 2016

Node v6.5.0

@zensh
Copy link
Member

zensh commented Sep 1, 2016

我在 v6.5下跑出来是这个样子：

➜  toa git:(master) npm run bench

> toa@1.8.10 bench /Users/zensh/git/js/toa
> cd bench && make


  koa, 1 middleware:  5725.18

  koa, 5 middleware:  5418.50

  koa, 10 middleware:  5290.23

  koa, 15 middleware:  5278.08

  koa, 20 middleware:  4965.67

  koa, 30 middleware:  4832.33

  koa, 50 middleware:  4447.08

  koa, 100 middleware:  3750.47

  toa, 1 middleware:  7836.58

  toa, 5 middleware:  7682.31

  toa, 10 middleware:  7198.45

  toa, 15 middleware:  7021.99

  toa, 20 middleware:  6738.97

  toa, 30 middleware:  6281.69

  toa, 50 middleware:  5451.08

  toa, 100 middleware:  4228.90

单独在 bench 目录下 make toa 基本上都是 7700+。应该并没有什么特别的啊

@zensh
Copy link
Member

zensh commented Sep 1, 2016

@fundon 刚蔡伟这边跑了下与我的结果也差不多。我的 osx 版本是 10.11.6，wrk v4.0.1

@fundon
Copy link

fundon commented Sep 1, 2016 •
edited
Loading

刚又单独跑了下，结果如下：

环境：

macOS 10.12 Beta 8
wrk v4.0.2

 λ ~/D/o/t/bench git:(master) > make toa

  toa, 1 middleware:  5613.04

  toa, 5 middleware:  5591.04

  toa, 10 middleware:  5316.98

  toa, 15 middleware:  5248.12

  toa, 20 middleware:  4985.92

  toa, 30 middleware:  4718.63

  toa, 50 middleware:  4168.84

  toa, 100 middleware:  3230.30

 λ ~/D/o/t/bench git:(master) > make koa

  koa, 1 middleware:  5959.68

  koa, 5 middleware:  5874.09

  koa, 10 middleware:  5660.43

  koa, 15 middleware:  5606.25

  koa, 20 middleware:  5412.67

  koa, 30 middleware:  5221.31

  koa, 50 middleware:  4844.04

  koa, 100 middleware:  4106.99

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016

我加了一下对比，koa 和 toa 基本无差异。

Benchmark

Results

  • node: 4.5.0
  • alinode: 1.6.0
  • koa: 1.2.2
  • toa: 1.8.1
  • egg: 0.2.0
Scene QPS Avg RT (ms) Stdev RT Max RT
egg Hello World 8485 5.73 2.77 63.12
koa Hello World 11852 4.23 2.99 84.56
toa Hello World 12169 4.22 2.59 39.82
egg nunjucks 6534 7.49 4.15 96.07
koa nunjucks 9702 5.34 5.52 145.75
toa nunjucks 9420 5.32 2.84 52.41

Scenes

  • Hello World: $ EGG_SERVER_ENV=prod node benchmarks/simple/dispatch.js
  • nunjucks: $ EGG_SERVER_ENV=prod node benchmarks/simple_view/dispatch.js

Scripts

  • koa: wrk http://remote-ip:7002/ -d 10 -c 50 -t 8
  • toa: wrk http://remote-ip:7003/ -d 10 -c 50 -t 8
  • egg: wrk http://remote-ip:7001/ -d 10 -c 50 -t 8

Server

  • CPU x4: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
  • Mem: 10G

Details

Hello World

  • koa
wrk http://10.209.84.139:7002/ -d 10 -c 50 -t 8
Running 10s test @ http://10.209.84.139:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.23ms    2.99ms  84.56ms   94.80%
    Req/Sec     1.50k   278.02     2.65k    77.36%
  119711 requests in 10.10s, 17.92MB read
Requests/sec:  11852.36
Transfer/sec:      1.77MB
  • toa
wrk http://10.209.84.139:7003/ -d 10 -c 50 -t 8
Running 10s test @ http://10.209.84.139:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.22ms    2.59ms  39.82ms   92.84%
    Req/Sec     1.54k   181.11     2.83k    77.09%
  122898 requests in 10.10s, 20.63MB read
Requests/sec:  12169.05
  • egg
wrk http://10.209.84.139:7001/ -d 10 -c 50 -t 8
Running 10s test @ http://10.209.84.139:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.73ms    2.77ms  63.12ms   84.02%
    Req/Sec     1.07k   205.02     2.04k    69.40%
  85700 requests in 10.10s, 24.11MB read
Requests/sec:   8485.16
Transfer/sec:      2.39MB

nunjucks

  • koa
wrk http://10.209.84.139:7002/ -d 10 -c 50 -t 8
Running 10s test @ http://10.209.84.139:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.34ms    5.52ms 145.75ms   98.23%
    Req/Sec     1.23k   239.23     2.48k    82.69%
  97999 requests in 10.10s, 244.96MB read
Requests/sec:   9702.78
Transfer/sec:     24.25MB
  • toa
wrk http://10.209.84.139:7003/ -d 10 -c 50 -t 8
Running 10s test @ http://10.209.84.139:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.32ms    2.84ms  52.41ms   92.35%
    Req/Sec     1.19k   134.30     2.73k    82.92%
  95139 requests in 10.10s, 239.53MB read
Requests/sec:   9420.63
Transfer/sec:     23.72MB
  • egg
wrk http://10.209.84.139:7001/ -d 10 -c 50 -t 8
Running 10s test @ http://10.209.84.139:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.49ms    4.15ms  96.07ms   89.68%
    Req/Sec   824.74    175.90     1.75k    67.79%
  65988 requests in 10.10s, 173.63MB read
Requests/sec:   6534.15
Transfer/sec:     17.19MB

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016 •
edited
Loading

@zensh 我再将3个框架按真实的来测，将 router 功能加上，给 koa 和 toa 的中间件数目加到跟 egg 自动的 security 数目一样的时候，基本就没什么差别了。 https://github.com/eggjs/egg/blob/2c82a46058d169ed384c4607eaaa9cc9bf3301ab/benchmarks/README.md

image

@zensh
Copy link
Member

zensh commented Sep 1, 2016

@fengmk2 你的结果应该是具有代表性的。

只是为啥我这边的电脑跑出来 toa 总比 koa 高一截呢~

@zensh
Copy link
Member

zensh commented Sep 1, 2016

@fengmk2 toa 都 1.8.11 了，10 个版本的差距期间好多 improve 的 😊

@fundon
Copy link

fundon commented Sep 1, 2016

@fengmk2 太赞了，这样比对很有说服力。

@zensh 看来你的电脑带有主角光环（玩笑）。

我本地跑基本都是持平。

还有可以考虑去掉 delegates 模块，性能有提升。

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016

@zensh 我写错了，版本是 1.8.10

image

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016

@zensh 你将 node_modules 删除了重新安装，再跑跑看？我们这边也跑了几台电脑，发现 koa 和 toa 的性能是持平的。

@zensh
Copy link
Member

zensh commented Sep 1, 2016

6269.85 vs 7909.23，还是有近 20% 多的差距。 @fengmk2 @fundon JSConf 有机会的话就面基一下，真不信邪

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016

@zensh 跑一下这个呢？ https://github.com/eggjs/egg/blob/master/benchmarks/simple/run.sh

@zensh
Copy link
Member

zensh commented Sep 1, 2016 •
edited
Loading

@fengmk2

------- egg hello -------
Hello World, egg
Running 10s test @ http://127.0.0.1:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.04ms   11.98ms 264.45ms   97.84%
    Req/Sec     0.88k   147.83     1.18k    87.55%
  69538 requests in 10.02s, 19.56MB read
Requests/sec:   6938.87
Transfer/sec:      1.95MB
run.sh: line 16: sleep3: command not found
------- koa hello -------
Hello World, koa
Running 10s test @ http://127.0.0.1:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.46ms    3.53ms 114.04ms   95.89%
    Req/Sec     1.14k   139.49     1.62k    79.25%
  90835 requests in 10.01s, 13.60MB read
Requests/sec:   9070.61
Transfer/sec:      1.36MB
run.sh: line 25: sleep3: command not found
------- toa hello -------
Hello World, toa
Running 10s test @ http://127.0.0.1:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.67ms    5.45ms 107.20ms   91.74%
    Req/Sec     1.23k   273.52     1.93k    68.25%
  98138 requests in 10.03s, 16.47MB read
Requests/sec:   9782.77
Transfer/sec:      1.64MB

------- egg view -------
    <title>egg view example</title>

Running 10s test @ http://127.0.0.1:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.51ms   16.93ms 326.19ms   97.51%
    Req/Sec   628.81    116.21     0.88k    86.63%
  49748 requests in 10.02s, 130.90MB read
Requests/sec:   4962.60
Transfer/sec:     13.06MB
------- koa view -------
    <title>koa view example</title>

Running 10s test @ http://127.0.0.1:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.39ms    4.81ms 154.82ms   97.63%
    Req/Sec   845.66     88.20     1.02k    86.73%
  67324 requests in 10.01s, 168.28MB read
Requests/sec:   6722.36
Transfer/sec:     16.80MB
------- toa view -------
    <title>toa view example</title>

Running 10s test @ http://127.0.0.1:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.42ms    5.99ms 138.43ms   91.89%
    Req/Sec     0.89k   138.14     1.18k    75.22%
  71110 requests in 10.03s, 179.03MB read
Requests/sec:   7091.85
Transfer/sec:     17.86MB

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016

果然是很神奇，只能 cc @dead-horse 的8核 mbp 了。

@fundon
Copy link

fundon commented Sep 1, 2016 •
edited
Loading

神奇，why?

最新测试结果：

------- egg hello -------
Hello World, egg
Running 10s test @ http://127.0.0.1:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    11.85ms   12.13ms 261.17ms   96.57%
    Req/Sec   565.18    116.74     0.86k    76.26%
  44863 requests in 10.03s, 12.62MB read
Requests/sec:   4474.63
Transfer/sec:      1.26MB
------- koa hello -------
Hello World, koa
Running 10s test @ http://127.0.0.1:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.72ms    7.11ms 145.60ms   93.93%
    Req/Sec   670.50    105.30     0.93k    81.35%
  53425 requests in 10.02s, 8.00MB read
Requests/sec:   5331.48
Transfer/sec:    817.42KB
------- toa hello -------
Hello World, toa
Running 10s test @ http://127.0.0.1:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.00ms    6.74ms 120.33ms   91.54%
    Req/Sec     0.85k   131.62     1.16k    73.09%
  67715 requests in 10.02s, 11.37MB read
Requests/sec:   6757.36
Transfer/sec:      1.13MB

@fengmk2
Copy link
Contributor Author

fengmk2 commented Sep 1, 2016

我的 mbp 显示是 toa 快的

------- egg hello -------
Hello World, egg
Running 10s test @ http://127.0.0.1:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.58ms    4.96ms 111.66ms   94.09%
    Req/Sec   824.26    134.42     1.21k    75.90%
  63356 requests in 10.00s, 17.82MB read
Requests/sec:   6334.79
Transfer/sec:      1.78MB
------- koa hello -------
Hello World, koa
Running 10s test @ http://127.0.0.1:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.06ms   11.57ms 133.25ms   95.94%
    Req/Sec     0.97k   298.72     1.50k    73.94%
  74996 requests in 10.00s, 11.23MB read
Requests/sec:   7498.98
Transfer/sec:      1.12MB
------- toa hello -------
Hello World, toa
Running 10s test @ http://127.0.0.1:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.65ms   10.69ms 152.54ms   94.78%
    Req/Sec     1.15k   342.72     1.95k    72.18%
  87788 requests in 10.00s, 14.73MB read
Requests/sec:   8778.48
Transfer/sec:      1.47MB

@dead-horse
Copy link 


  koa, 1 middleware:  9820.92

  koa, 5 middleware:  10186.17

  koa, 10 middleware:  9364.13

  koa, 15 middleware:  8585.03

  toa, 1 middleware:  8829.55

  toa, 5 middleware:  8036.16

  toa, 10 middleware:  7799.06

  toa, 15 middleware:  7683.82

node 6.5.0， toa master 分支的代码

@dead-horse
Copy link

egg 的 benchmark 测试 koa 和 toa 差距不大，基本持平

@dead-horse
Copy link 

------- egg hello -------
Hello World, egg
Running 10s test @ http://127.0.0.1:7001/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.52ms    4.12ms  81.54ms   95.33%
    Req/Sec     1.91k   472.16     3.40k    69.79%
  143466 requests in 10.00s, 40.36MB read
Requests/sec:  14347.16
Transfer/sec:      4.04MB
simple/run.sh: line 16: sleep3: command not found
------- koa hello -------
Hello World, koa
Running 10s test @ http://127.0.0.1:7002/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.84ms    2.69ms  59.21ms   93.28%
    Req/Sec     2.44k   762.83     5.00k    69.00%
  181035 requests in 10.00s, 27.11MB read
Requests/sec:  18103.98
Transfer/sec:      2.71MB
simple/run.sh: line 25: sleep3: command not found
------- toa hello -------
Hello World, toa
Running 10s test @ http://127.0.0.1:7003/
  8 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.85ms    4.21ms  67.42ms   94.12%
    Req/Sec     2.44k   836.16     5.43k    69.36%
  180013 requests in 10.00s, 30.21MB read
Requests/sec:  18004.01
Transfer/sec:      3.02MB
2016-09-02 00:27:21,238 INFO 52680 [master] exit with code: 0

三次一次 koa 高，一次 toa 高，一次持平

@dead-horse
Copy link

不纠结这种东西，不带业务逻辑的 benchmark 测来玩，没意思

@fundon
Copy link

fundon commented Sep 1, 2016

@dead-horse 赞同，还是紧随业务才更有意义。

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@fengmk2 @coveralls @zensh @fundon @dead-horse