Add prism (Crystal) #184

waghanza · 2018-04-11T07:11:35Z

vladfaust · 2018-04-11T08:32:14Z

Oh, I personally find it useless. Each Crystal framework is ultra-fast and the micro-differences only depend on amount of features a framework has. Prism uses the same Radix tree implementation as others to resolve paths, so I don't think it makes any sense.

waghanza · 2018-04-11T12:52:37Z

I understand you point of view, but I disagree.

Each Crystal framework is ultra-fast and the micro-differences only depend on amount of features a framework has.

If you see the results (of some other benchmarks), we notice it a difference between frameworks, and it could be interesting to display those informations 😛

OvermindDL1 · 2018-04-11T14:38:46Z

Crystal itself is single-core fast, but as a server it is dreadfully slow because of the way these keep getting implemented. As the Crystal devs themselves stated, the servers need to be run one per core with REUSE socket enabled. Running them as they are in this benchmark is entirely wrong, so before more are added, the existing ones need to be fixed first.

andymans · 2018-05-20T19:07:51Z

Crystal looks like a very interesting language. However, attracting people to it on the basis of these benchmarks is counter-productive. As soon a people try doing anything beyond a trivial 'hello world' these benchmarks collapse. As an example, JSON processing in crystal is only fractionally faster than writing the data down by hand, scanning it, and sending it to the user via email :-)

waghanza · 2018-05-20T19:34:23Z

The purpose of this benchmarking project is not to attract people on crystal.

The main idea is to compare tools (frameworks)

vladfaust · 2018-05-20T22:11:19Z

@andymans, see https://carc.in/#/r/43pg

  to json 107.79k (  9.28µs) (± 1.23%) fastest
from json 119.36k (  8.38µs) (± 2.09%) fastest

100k+ on my 0.9Ghz machine. Multiplied by 4 cores ($2.99 on Scaleway) results in 400k+ per second. It should be more than enough for the most of applications 🤔

Attracting people to it on the basis of these benchmarks is actually a good thing, because if another language framework sucks even with hello world, it's a reason to think of migrating to an initially faster solution 👍

BTW, I'm finishing Crystal version of http://realworld.io and it literally kills other languages 🔫

andymans · 2018-05-20T22:38:23Z

Excellent - very much look forward to seeing the result! The realworld is a useful app specifically because it tests > 1 facet - e.g. the router, json, plus non-infrastructure items such as jwt etc. To my mind, a good score on realworld is a far more realistic yardstick of what the language can deliver.

OvermindDL1 · 2018-05-21T16:03:43Z

and it literally kills other languages gun

How so in what way? I'd love to see a blog on how! :-)

vladfaust · 2018-05-21T16:24:29Z

@OvermindDL1 I'll definitely post a Medium article on this!

OvermindDL1 · 2018-05-21T16:37:01Z

@vladfaust Awesome, ping me with the URL when it's up? :-)

waghanza · 2018-06-09T11:37:57Z

@vladfaust You seems to be right. On the last test https://github.com/waghanza/which_is_the_fastest/blob/wrk/README.md#ranking-framework all crystal frameworks seems to be very close (what is now true for other languages)

vladfaust · 2018-08-28T13:42:56Z

So, I'm benchmarking it on my Xiaomi Mi Air 12':

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                4
Thread(s) per core:    2
Core(s) per socket:    2
Model name:            Intel(R) Core(TM) m3-6Y30 CPU @ 0.90GHz
CPU MHz:               500.010
CPU max MHz:           2200,0000
CPU min MHz:           400,0000

Wrk script with threads = cpu_count + 1:

printf "> Running GET /\n"
wrk -t5 -d15s -c1000 http://127.0.0.1:3000
printf "\n> Running POST /\n"
wrk -s bench/wrk/post.lua -t5 -d15s -c1000 http://127.0.0.1:3000
printf "\n> Running GET /user/42\n"
wrk -t5 -d15s -c1000 http://127.0.0.1:3000/user/42

Prism

@ v0.4.0-beta.3:

require "prism"

struct UserAction
  include Prism::Action
  include Prism::Action::Params

  params do
    type id : Int32
  end

  def call
    text(params[:id])
  end
end

router = Prism::Router.new do
  on "/", methods: %w(get post)
  get "/user/:id", UserAction
end

server = Prism::Server.new([router])
server.bind_tcp(3000, reuse_port: true)
server.listen

Results for a single server instance:

> Running GET /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    22.80ms   13.37ms 451.56ms   99.31%
    Req/Sec     8.82k     1.13k   16.35k    89.73%
  658183 requests in 15.08s, 38.92MB read
Requests/sec:  43635.38
Transfer/sec:      2.58MB

> Running POST /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    26.74ms    8.84ms 237.52ms   91.17%
    Req/Sec     7.24k   814.26    10.44k    81.20%
  540225 requests in 15.07s, 31.94MB read
Requests/sec:  35852.77
Transfer/sec:      2.12MB

> Running GET /user/42
Running 15s test @ http://127.0.0.1:3000/user/42
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    27.32ms   20.43ms 467.08ms   95.25%
    Req/Sec     7.26k     1.27k   16.68k    90.93%
  542030 requests in 15.04s, 53.76MB read
Requests/sec:  36039.95
Transfer/sec:      3.57MB

Results for 4 server instances:

> Running GET /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.64ms    8.52ms  86.31ms   73.66%
    Req/Sec    18.88k     3.17k   37.65k    73.50%
  1403442 requests in 15.07s, 82.98MB read
Requests/sec:  93142.05
Transfer/sec:      5.51MB

> Running POST /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    13.14ms   11.58ms 160.23ms   83.03%
    Req/Sec    15.81k     3.13k   26.50k    67.30%
  1179816 requests in 15.10s, 69.76MB read
Requests/sec:  78155.70
Transfer/sec:      4.62MB

> Running GET /user/42
Running 15s test @ http://127.0.0.1:3000/user/42
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    36.74ms   80.77ms   1.06s    92.19%
    Req/Sec    12.67k     3.12k   21.46k    74.59%
  939867 requests in 15.09s, 93.22MB read
Requests/sec:  62296.41
Transfer/sec:      6.18MB

Kemal

@ v0.24.0

require "kemal"

Kemal.config do |cfg|
  cfg.serve_static = false
  cfg.logging = false
end

get "/" do |env|
  nil
end

get "/user/:id" do |env|
  env.params.url["id"]
end

post "/user" do |env|
  nil
end

Kemal.config.env = "production"
Kemal.run do |cfg|
  cfg.server.not_nil!.bind_tcp 3000, reuse_port: true
end

Results for a single server instance:

> Running GET /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    27.34ms   16.93ms 467.08ms   96.37%
    Req/Sec     7.21k     1.22k   12.47k    90.80%
  538466 requests in 15.08s, 55.46MB read
Requests/sec:  35708.89
Transfer/sec:      3.68MB

> Running POST /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    80.74ms   39.34ms 935.51ms   97.81%
    Req/Sec     2.42k   432.41     3.42k    82.53%
  180781 requests in 15.05s, 4.33GB read
Requests/sec:  12014.53
Transfer/sec:    294.53MB

> Running GET /user/42
Running 15s test @ http://127.0.0.1:3000/user/42
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    35.61ms   48.60ms 971.49ms   98.75%
    Req/Sec     6.11k     1.52k   16.65k    91.67%
  445184 requests in 15.05s, 46.70MB read
Requests/sec:  29573.52
Transfer/sec:      3.10MB

Results for 4 server instances:

> Running GET /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    23.08ms   57.65ms 920.91ms   94.80%
    Req/Sec    15.86k     3.27k   31.29k    74.01%
  1179546 requests in 15.09s, 121.49MB read
Requests/sec:  78157.36
Transfer/sec:      8.05MB

> Running POST /
Running 15s test @ http://127.0.0.1:3000
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    58.09ms   29.44ms 788.61ms   94.35%
    Req/Sec     3.47k   700.61     5.02k    63.34%
  257917 requests in 15.08s, 6.17GB read
Requests/sec:  17108.43
Transfer/sec:    419.36MB

> Running GET /user/42
Running 15s test @ http://127.0.0.1:3000/user/42
  5 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    30.82ms   70.33ms 770.44ms   94.50%
    Req/Sec    12.00k     3.25k   28.77k    73.01%
  890524 requests in 15.09s, 93.42MB read
Requests/sec:  59026.08
Transfer/sec:      6.19MB

TL;DR: Requests/sec

i stands for instance

	1i GET /	4i GET /	1i POST /	4i POST /	1i GET /user/42	4i GET/user/42
Prism	43.6k	93.1k	35.8k	78.1k	36.0k	62.2k
Kemal	35.7k	78.1k	12.0k	17.1k	29.5k	59.0k
Raze	46.2k	91.9k	21.5k	30.3k	38.1k	62.3k

@sdogruyol I took Kemal code from this repo, it may be outdated, can it be improved for more performance?

Edit: I took Kemal for comparison because it's the one I've had experience with, and it is "sinatra-like", just like Prism. I haven't benched other frameworks.

Edit: Added Raze.

waghanza · 2018-08-28T14:03:04Z

@vladfaust Thanks for update 😛
So, you say that the assertion you made

Each Crystal framework is ultra-fast and the micro-differences

is not true

btw, the results here are no ready for any conclusion ;-) the code is quite update, so thanks for yours ❤️

aichholzer · 2018-08-29T01:40:55Z

@andymans -This is probably the best and most to-the-point comment I have ever read on a code thread. I keep giggling every time I read it...
😆

waghanza · 2018-09-04T11:04:18Z

@aichholzer the comment above is indeed very interesting, but the assumption that this project is trying to attract people in crystal community is wrong ;-)

vladfaust · 2018-09-04T11:20:10Z

@waghanza we should find a way to test multiple instances of Crystal servers.

waghanza · 2018-09-04T11:22:08Z

@vladfaust what do you mean by multiple instances ?

stakach · 2018-09-04T11:29:30Z

@vladfaust I did: #333 (comment)

waghanza · 2018-09-04T11:31:13Z

ah ok, using all available cores on each machines ... has to be implemented on each frameworks

if you have time to PR, I'll be glad to learn ❤️ or I'll implement later 😛

waghanza added the language:crystal label Aug 25, 2018

vladfaust mentioned this issue Aug 28, 2018

Benchmarks onyxframework/http#36

Closed

waghanza mentioned this issue Sep 4, 2018

Introduce prism #340

Merged

waghanza closed this as completed Sep 4, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add prism (Crystal) #184

Add prism (Crystal) #184

waghanza commented Apr 11, 2018

vladfaust commented Apr 11, 2018

waghanza commented Apr 11, 2018 •

edited

OvermindDL1 commented Apr 11, 2018

andymans commented May 20, 2018

waghanza commented May 20, 2018

vladfaust commented May 20, 2018 •

edited

andymans commented May 20, 2018

OvermindDL1 commented May 21, 2018

vladfaust commented May 21, 2018

OvermindDL1 commented May 21, 2018

waghanza commented Jun 9, 2018

vladfaust commented Aug 28, 2018 •

edited

waghanza commented Aug 28, 2018 •

edited

aichholzer commented Aug 29, 2018

waghanza commented Sep 4, 2018

vladfaust commented Sep 4, 2018

waghanza commented Sep 4, 2018

stakach commented Sep 4, 2018

waghanza commented Sep 4, 2018

Add prism (Crystal) #184

Add prism (Crystal) #184

Comments

waghanza commented Apr 11, 2018

vladfaust commented Apr 11, 2018

waghanza commented Apr 11, 2018 • edited

OvermindDL1 commented Apr 11, 2018

andymans commented May 20, 2018

waghanza commented May 20, 2018

vladfaust commented May 20, 2018 • edited

andymans commented May 20, 2018

OvermindDL1 commented May 21, 2018

vladfaust commented May 21, 2018

OvermindDL1 commented May 21, 2018

waghanza commented Jun 9, 2018

vladfaust commented Aug 28, 2018 • edited

Prism

Kemal

TL;DR: Requests/sec

waghanza commented Aug 28, 2018 • edited

aichholzer commented Aug 29, 2018

waghanza commented Sep 4, 2018

vladfaust commented Sep 4, 2018

waghanza commented Sep 4, 2018

stakach commented Sep 4, 2018

waghanza commented Sep 4, 2018

waghanza commented Apr 11, 2018 •

edited

vladfaust commented May 20, 2018 •

edited

vladfaust commented Aug 28, 2018 •

edited

waghanza commented Aug 28, 2018 •

edited