Add benchmark for net/http

[bartlomieju]

Benchmark for HTTP server from https://github.com/denoland/net

Results from my machine:

$ ./tools/http_benchmark.py target/debug/deno ./target/debug/hyper_hello
http_benchmark testing DENO.
Listening on 127.0.0.1:4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.28ms  288.76us  10.82ms   93.95%
    Req/Sec     3.93k   390.48     4.37k    76.00%
  78223 requests in 10.00s, 3.80MB read
Requests/sec:   7822.43
Transfer/sec:    389.59KB

http_benchmark testing DENO using net/http.
Compiling /Users/biwanczuk/dev/deno/tests/xhttp_bench.ts
Downloading https://deno.land/x/net/http_status
Downloading https://deno.land/x/net/util
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.20ms  446.88us  17.78ms   94.81%
    Req/Sec     2.29k   160.77     2.39k    92.57%
  45952 requests in 10.10s, 2.19MB read
Requests/sec:   4549.43
Transfer/sec:    222.14KB

http_benchmark testing NODE.
port 4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   353.27us  281.79us  14.09ms   97.62%
    Req/Sec    14.55k     1.78k   17.08k    77.72%
  292399 requests in 10.10s, 31.23MB read
Requests/sec:  28949.17
Transfer/sec:      3.09MB

http_benchmark testing node_tcp.js
port 4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   156.49us   74.79us   3.37ms   85.96%
    Req/Sec    31.94k     3.12k   37.15k    76.24%
  642134 requests in 10.10s, 31.23MB read
Requests/sec:  63581.47
Transfer/sec:      3.09MB

http_benchmark testing RUST hyper.
Listening on http://127.0.0.1:4544
third_party/wrk/mac/wrk -d 10s http://127.0.0.1:4544/
Running 10s test @ http://127.0.0.1:4544/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   347.32us  302.80us   9.27ms   92.58%
    Req/Sec    15.22k   709.28    16.42k    86.14%
  305713 requests in 10.10s, 25.66MB read
Requests/sec:  30269.37
Transfer/sec:      2.54MB

[ry]

I updated the branch to use submodules. But I'm getting an unexpected error now

~/src/deno> ./tools/http_benchmark.py target/release/deno
http_benchmark testing DENO using net/http.
Compiling /Users/rld/src/deno/js/deps/https/deno.land/x/net/http_bench.ts
Downloading https://deno.land/x/net/http.ts.ts
Downloading https://deno.land/x/net/http.ts.js
NotFound: Cannot resolve module "./http.ts" from "/Users/rld/src/deno/js/deps/https/deno.land/x/net/http_bench.ts"
    at DenoError (deno/js/errors.ts:19:5)
    at maybeError (deno/js/errors.ts:38:12)
    at maybeThrowError (deno/js/errors.ts:26:15)
    at sendSync (deno/js/dispatch.ts:67:5)
    at codeFetch (deno/js/os.ts:41:19)
    at _resolveModule (deno/js/compiler.ts:235:38)
    at moduleNames.map.name (deno/js/compiler.ts:514:31)

[kitsonk]

Seems like for some reason it starts thing a relative module is a remote module.

Just posting this as I am digging into it, with the debug flag:

http_benchmark testing DENO using net/http.
DEBUG RS - mkdir -p /Users/kkelly/github/deno/js/gen
DEBUG RS - set dir perm to 493
DEBUG RS - mkdir -p /Users/kkelly/github/deno/js/deps
DEBUG RS - set dir perm to 493
DEBUG RS - mkdir -p /Users/kkelly/github/deno/js/deps/http
DEBUG RS - set dir perm to 493
DEBUG RS - mkdir -p /Users/kkelly/github/deno/js/deps/https
DEBUG RS - set dir perm to 493
DEBUG RS - root /Users/kkelly/github/deno/js
DEBUG RS - gen /Users/kkelly/github/deno/js/gen
DEBUG RS - deps /Users/kkelly/github/deno/js/deps
DEBUG RS - deps_http /Users/kkelly/github/deno/js/deps/http
DEBUG RS - deps_https /Users/kkelly/github/deno/js/deps/https
DEBUG RS - msg_from_js Start sync true
DEBUG JS - cwd /Users/kkelly/github/deno
DEBUG JS - args [ "js/deps/https/deno.land/x/net/http_bench.ts", "127.0.0.1:4544" ]
DEBUG JS - runner.run js/deps/https/deno.land/x/net/http_bench.ts /Users/kkelly/github/deno/
DEBUG JS - compiler.resolveModule { moduleSpecifier: "js/deps/https/deno.land/x/net/http_bench.ts", containingFile: "/Users/kkelly/github/deno/" }
DEBUG JS - compiler.resolveFileName { moduleSpecifier: "js/deps/https/deno.land/x/net/http_bench.ts", containingFile: "/Users/kkelly/github/deno/" }
DEBUG JS - os.ts codeFetch js/deps/https/deno.land/x/net/http_bench.ts /Users/kkelly/github/deno/
DEBUG RS - code_fetch. module_specifier js/deps/https/deno.land/x/net/http_bench.ts containing_file /Users/kkelly/github/deno/
DEBUG RS - resolve_module module_specifier js/deps/https/deno.land/x/net/http_bench.ts containing_file /Users/kkelly/github/deno/
DEBUG RS - module_name: /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts, filename: /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG RS - load_cache code: /Users/kkelly/github/deno/js/gen/f3edd7b970114a3de67ef3d42f7e6c05109380e5.js map: /Users/kkelly/github/deno/js/gen/f3edd7b970114a3de67ef3d42f7e6c05109380e5.js.map
DEBUG RS - msg_from_js CodeFetch sync true
DEBUG JS - resolveModule sourceCode length: 341
DEBUG JS - resolveModule has outputCode: false
DEBUG JS - resolveModule has source map: false
DEBUG JS - resolveModule has media type: TypeScript
DEBUG JS - compiler.setFileName { moduleSpecifier: "js/deps/https/deno.land/x/net/http_bench.ts", containingFile: "/Users/kkelly/github/deno/" }
DEBUG JS - runner._gatherDependencies /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG JS - runner._makeDefine /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
Compiling /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG JS - getCurrentDirectory()
DEBUG JS - getScriptSnapshot() /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG JS - getScriptVersion() /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG JS - getScriptKind() /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG JS - getCompilationSettings()
DEBUG JS - getDefaultLibFileName()
DEBUG JS - compiler.resolveModule { moduleSpecifier: "lib.deno_runtime.d.ts", containingFile: "$asset$" }
DEBUG JS - compiler.resolveFileName { moduleSpecifier: "lib.deno_runtime.d.ts", containingFile: "$asset$" }
DEBUG JS - resolveModule sourceCode length: 59543
DEBUG JS - resolveModule has outputCode: true
DEBUG JS - resolveModule has source map: true
DEBUG JS - resolveModule has media type: TypeScript
DEBUG JS - compiler.setFileName { moduleSpecifier: "lib.deno_runtime.d.ts", containingFile: "$asset$" }
DEBUG JS - resolveModuleNames() { moduleNames: [ "deno", "./http.ts" ], containingFile: "/Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts" }
DEBUG JS - compiler.resolveModule { moduleSpecifier: "lib.deno_runtime.d.ts", containingFile: "$asset$" }
DEBUG JS - compiler.resolveFileName { moduleSpecifier: "lib.deno_runtime.d.ts", containingFile: "$asset$" }
DEBUG JS - compiler.resolveModule { moduleSpecifier: "./http.ts", containingFile: "/Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts" }
DEBUG JS - compiler.resolveFileName { moduleSpecifier: "./http.ts", containingFile: "/Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts" }
DEBUG JS - os.ts codeFetch ./http.ts /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG RS - code_fetch. module_specifier ./http.ts containing_file /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts
DEBUG RS - resolve_module module_specifier ./http.ts containing_file https://deno.land/x/net/http_bench.ts
DEBUG RS - module_name: https://deno.land/x/net/http.ts, filename: /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http.ts
DEBUG RS - Trying https://deno.land/x/net/http.ts.ts...
Downloading https://deno.land/x/net/http.ts.ts
DEBUG RS - resolving host="deno.land"
DEBUG RS - connecting to 13.249.87.93:443
DEBUG RS - flushed 51 bytes
DEBUG RS - read 399 bytes
DEBUG RS - parsed 9 headers
DEBUG RS - incoming body is empty
DEBUG RS - Redirecting to https://raw.githubusercontent.com/denoland/deno_net/master/http.ts.ts...
DEBUG RS - resolving host="raw.githubusercontent.com"
DEBUG RS - pooling idle connection for "https://deno.land"
DEBUG RS - connecting to 151.101.0.133:443
DEBUG RS - flushed 86 bytes
DEBUG RS - read 725 bytes
DEBUG RS - parsed 20 headers
DEBUG RS - incoming body is content-length (15 bytes)
DEBUG RS - incoming body completed
DEBUG RS - Trying https://deno.land/x/net/http.ts.js...
Downloading https://deno.land/x/net/http.ts.js
DEBUG RS - resolving host="deno.land"
DEBUG RS - dropping I/O source: 0
DEBUG RS - connecting to 13.249.87.87:443
DEBUG RS - dropping I/O source: 0
DEBUG RS - flushed 51 bytes
DEBUG RS - read 399 bytes
DEBUG RS - parsed 9 headers
DEBUG RS - incoming body is empty
DEBUG RS - Redirecting to https://raw.githubusercontent.com/denoland/deno_net/master/http.ts.js...
DEBUG RS - resolving host="raw.githubusercontent.com"
DEBUG RS - connecting to 151.101.64.133:443
DEBUG RS - pooling idle connection for "https://deno.land"
DEBUG RS - flushed 86 bytes
DEBUG RS - read 723 bytes
DEBUG RS - parsed 20 headers
DEBUG RS - incoming body is content-length (15 bytes)
DEBUG RS - incoming body completed
DEBUG RS - msg_from_js CodeFetch sync true
DEBUG RS - dropping I/O source: 1
DEBUG RS - op err Cannot resolve module "./http.ts" from "/Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts"
DEBUG RS - dropping I/O source: 0
NotFound: Cannot resolve module "./http.ts" from "/Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts"
    at maybeError (deno/js/errors.ts:38:12)
    at maybeThrowError (deno/js/errors.ts:26:15)
    at sendSync (deno/js/dispatch.ts:67:5)
    at Object.codeFetch (deno/js/os.ts:41:19)
    at Compiler._resolveModule (deno/js/compiler.ts:239:38)
    at moduleNames.map.name (deno/js/compiler.ts:548:31)
    at Array.map (<anonymous>)
    at Compiler.resolveModuleNames (deno/js/compiler.ts:540:24)
    at Object.compilerHost.resolveModuleNames (<anonymous>)
    at resolveModuleNamesWorker (<anonymous>)

[kitsonk]

It is going down this path:

deno/src/deno_dir.rs

Lines 262 to 263 in 568ac0c

	if filename_path.starts_with(&self.deps) {
	let (rest, prefix) = if filename_path.starts_with(&self.deps_https) {

I added a debug! in there and get out:

DEBUG RS - filename: /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts filename_path: /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http_bench.ts deps: /Users/kkelly/github/deno/js/deps

So because it believes it is the DENO_DIR and the URL matches the pattern of where https:// resources are cached, and therefore it changes the containing module to a URL and then resolves the subsequent module to that, which doesn't exist.

[kevinkassimo]

@kitsonk Yikes, I found the real reason. Instead, it is caused by these lines:

deno/src/deno_dir.rs

Lines 152 to 154 in 568ac0c

	let source = fs::read_to_string(&p)?;
	let content_type = fs::read_to_string(&mt)?;
	(source, map_content_type(&p, Some(&content_type)))

and with this:

deno/src/deno_dir.rs

Lines 137 to 139 in 568ac0c

	let mut media_type_filename = filename.to_string();
	media_type_filename.push_str(".mime");
	let mt = Path::new(&media_type_filename);

We don't have .mime files here, so it crashes

[kitsonk]

@kevinkassimo while that certainly might also be an issue, the issue of the containing file changing is being mis-understood in src_file_to_url. I am not seeing any panics in the at the moment and the -D is pretty clear.

[kevinkassimo]

@kitsonk I have tried fixed these lines and seems benchmark is working now.
I don't think the lines you listed are the direct cause. Notice that from your log everything runs fine until these:

// everything about http_bench.ts is okay
DEBUG RS - module_name: https://deno.land/x/net/http.ts, filename: /Users/kkelly/github/deno/js/deps/https/deno.land/x/net/http.ts <-- This looks fine
DEBUG RS - Trying https://deno.land/x/net/http.ts.ts... <-- Ah-oh, the default no-extension attempt has failed (somewhere)

I added a bunch of println!s and also printed out the actual error message, and it is exactly dying at this line:

deno/src/deno_dir.rs

Line 153 in 568ac0c

let content_type = fs::read_to_string(&mt)?;

(You can try add logging immediately before and after this line. There is no panic because the extension guessing mechanism consumes the error...)

src_file_to_url is working fine in this case, as we are actually trying to create the fake impression that deps is the $DENO_DIR/deps folder so that we can use HTTP imports just fine in files

I'm making a PR for this

[kitsonk]

@kevinkassimo yeah, I see what you are saying, but it shouldn't even be looking on the network for the file... I can see how the lack of .mime in the hard cache also adds to the problem, but it seems that the current logic is that any modules that locally cached, when they load additional modules, the local cache for that module isn't ever evaluated, it will always go refetch it.

[kevinkassimo]

@kitsonk I might be understanding it wrong, but I still believe it won't look on the network if a locally cached file is present. resolve_module might seem to call src_file_to_url first to convert to a url, but then we have

deno/src/deno_dir.rs

Lines 322 to 342 in 568ac0c

	match j.scheme() {
	"file" => {
	let mut p = deno_fs::normalize_path(j.to_file_path().unwrap().as_ref());
	module_name = p.clone();
	filename = p;
	}
	"https" => {
	module_name = j.to_string();
	filename = deno_fs::normalize_path(
	get_cache_filename(self.deps_https.as_path(), &j).as_ref(),
	)
	}
	"http" => {
	module_name = j.to_string();
	filename = deno_fs::normalize_path(
	get_cache_filename(self.deps_http.as_path(), &j).as_ref(),
	)
	}
	// TODO(kevinkassimo): change this to support other protocols than http
	_ => unimplemented!(),
	}

And thus converting back to a local cached path again... With the file exist at the path no remote fetch would happen...

The thing that actually makes me worried instead is that we are not currently checking if files with added suffixes exist in the cache. However, that is another issue.

[kitsonk]

@kevinkassimo yeah, checking in .mimes alongside of the other modules fixes the problem. I will raise a PR to that repo.

[ry]

Rebased on top of Kevin's #1291. It's working locally for me now.

[ry]

LGTM - thanks @bartlomieju @kitsonk @kevinkassimo !