Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make this fst-er #2

Merged
merged 5 commits into from Oct 20, 2018
Merged

Make this fst-er #2

merged 5 commits into from Oct 20, 2018

Conversation

killercup
Copy link
Owner

cf. #1

It's noticeably slower, but this is the generating step. It should need
to be *super* fast.
@killercup
Copy link
Owner Author

killercup commented Oct 20, 2018

Shitty benchmark to see if this crate is even remotely useful or if file systems caches are actually enough.

tl;dr

Seems to work well for this specific case.

Server Req/Sec
Node 10 6.3k
PHP 7.1 0.8k
sfz (Rust) 10.6k
miniserve (Rust) 26.5k
caddy 0.11 (Go) 28.9k
static-filez 77.3k

Details

I benchmarked

  • serving files (actually: a file) generated by cargo doc on this branch
  • by running wrk -t2 -c50 -d10s http://127.0.0.1:8080/arrayvec/index.html
    • (this is wrk 4.1.0 using kqueue)
  • on my late 2016 15" MacBook Pro (using an encrypted APFS disk)
  • against the following servers

Node

Using https://www.npmjs.com/package/static-server, a quick and easy server using node's http lib and async magic

$ node -v
v10.12.0
$  npx static-server -p 8080 target/doc/ > /dev/null
Running 10s test @ http://127.0.0.1:8080/arrayvec/index.html
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.88ms    2.40ms  29.18ms   88.50%
    Req/Sec     3.21k   493.25     3.91k    83.00%
  63856 requests in 10.01s, 503.51MB read
Requests/sec:   6380.90
Transfer/sec:     50.31MB

PHP

A quick thing I use for dev purposes sometimes

$ php -v
PHP 7.1.16 (cli) (built: Mar 31 2018 02:59:59) ( NTS )
...
$ php -S 0.0.0.0:8080 -t target/doc/ 2> /dev/null
Running 10s test @ http://127.0.0.1:8080/arrayvec/index.html
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    56.61ms   21.28ms 145.58ms   68.63%
    Req/Sec   400.85    147.41   730.00     65.66%
  7976 requests in 10.07s, 62.28MB read
Requests/sec:    792.14
Transfer/sec:      6.19MB

sfz (Rust)

10s of searching turned up https://crates.io/crates/sfz which is based on hyper and tokio, just like this crate. Installed it via homebrew.

$ sfz -V
sfz 0.0.3
$ sfz -p 8080 target/doc/
Running 10s test @ http://127.0.0.1:8080/arrayvec/index.html
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.74ms    1.37ms  42.19ms   94.39%
    Req/Sec     5.35k   460.87     5.85k    82.00%
  106476 requests in 10.00s, 840.37MB read
Requests/sec:  10646.19
Transfer/sec:     84.03MB

miniserve (Rust)

sfz seemed slow, so I looked further and found https://crates.io/crates/miniserve, based on actix-web. I downloaded the osx binary from https://github.com/svenstaro/miniserve/releases/tag/v0.2.0.

Sadly, I was not able to get the arrayvec index page, I only got the file listing. So, I cloned the repo, ran cargo update (hoping this was fixed upstream in a dependency), followed by cargo install --path ., and letting it compile for a looong while. That didn't help at all, so removed the .show_files_listing() line and compiled again (for only a minute this time). After getting a blank page, I noticed it was Firefox caching this somehow. And there we go.

$ miniserve -h 
miniserve 0.2.0
...
$ miniserve -p 8080 target/doc/
Running 10s test @ http://127.0.0.1:8080/arrayvec/index.html
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.88ms    1.12ms  39.25ms   86.34%
    Req/Sec    13.41k     2.10k   29.36k    77.11%
  268128 requests in 10.10s, 2.07GB read
Requests/sec:  26546.40
Transfer/sec:    209.90MB

Caddy (Go)

"Big league" server, written in Go, has lots of features, well-tuned, but is still super easy to install via homebrew.

$ caddy -version
Caddy 0.11.0 (unofficial)
$ caddy -port 8080 -root target/doc/

Note: I didn't notice at first, but /arrayvec/index.html just returned a 301 (quite quickly I might add), so I tested this using wrk -t2 -c50 -d10s http://127.0.0.1:8080/arrayvec/ (just removed the index.html).

Running 10s test @ http://127.0.0.1:8080/arrayvec/
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.63ms    1.25ms  19.74ms   77.25%
    Req/Sec    14.54k     2.44k   26.31k    80.00%
  289939 requests in 10.02s, 2.23GB read
Requests/sec:  28928.43
Transfer/sec:    227.52MB

static-filez

$ rustc --version
rustc 1.31.0-nightly (4699283c5 2018-10-13)
$ cargo run --release -- -vvvv build target/doc/ docs.archive
$ cargo run --release -- serve --port 8080 docs.archive
Running 10s test @ http://127.0.0.1:8080/arrayvec/index.html
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   631.42us  106.26us   2.16ms   85.90%
    Req/Sec    38.89k     1.36k   40.53k    82.50%
  773773 requests in 10.00s, 1.89GB read
Requests/sec:  77371.25
Transfer/sec:    193.10MB

@killercup killercup merged commit 93335f4 into master Oct 20, 2018
@killercup killercup deleted the make-this-fst-er branch October 20, 2018 19:05
@killercup
Copy link
Owner Author

Quick addendum:

This is the speed of hyper's hello world (literally copied from https://hyper.rs/):

Running 10s test @ http://127.0.0.1:3000
  2 threads and 50 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   620.01us   64.11us   2.12ms   88.87%
    Req/Sec    39.76k     1.59k   41.58k    84.16%
  799108 requests in 10.10s, 67.83MB read
Requests/sec:  79112.94
Transfer/sec:      6.71MB

Seems like we're not adding much on top of that! (So it'd actually be futile to optimize this anymore.)

@killercup killercup mentioned this pull request Oct 22, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant