problems pipelining requests with async routes

[mattaylor]

I can get screaming performance by pipelining simple requests through httpbeast however when my route has to await before it can send then any subsequent requests in the pipeline are blocked.

Is this because async requests are somhow routed through asynchttpserver rather than httpbeast and therefore do not support pipelining?

I have am usng a server based on httpbeast with my own router and a a fork of asyncpg which address some compatibility issues with the nim devel branch.

I have benchmarked my TechEmpower implementation (Whipower) against the FastHttp Go Implementation (FastPower) and so far, with non pipelined requests my server is just a little slower on json and text requests, almost 2x as fast on the database requests but about 5x slower on the fortune html template. (not sure how best to optimize this yet).

In Pipelined mode whip+beast is up to 25% for basic 'json' and 'plaintext' requests but all the other the db related tests are dead in the water.

All my tests so far are without threads as when I try enabling threads the performance always seems slower, and there are many more issues with async routes.

Threads aside, it would be great if you could provide some pointers to better support request pipelining in these cases. Any other tips on how to optimise this service would be also very usefull as it would be awesome to see Nim in the top rankings of all the techempower tests.

These are the latest results I have..

Non Pipelined Results:

Benchmarking fastpower

Running 10s test @ http://localhost:8080/json
Requests/sec:  75883.00

Running 10s test @ http://localhost:8080/plaintext
Requests/sec:  87931.02

Running 10s test @ http://localhost:8080/fortune
Requests/sec:  31852.30

Running 10s test @ http://localhost:8080/queries?queries=10
Requests/sec:   5357.72

Running 10s test @ http://localhost:8080/update?queries=10
Requests/sec:   2274.38

------------------------------------------

Benchmarking whipower

Running 10s test @ http://localhost:8080/json
Requests/sec:  70098.06

Running 10s test @ http://localhost:8080/plaintext
Requests/sec:  70161.92

Running 10s test @ http://localhost:8080/fortune
Requests/sec:  12953.60

Running 10s test @ http://localhost:8080/queries?queries=10
Requests/sec:  11350.36

Running 10s test @ http://localhost:8080/update?queries=10
Requests/sec:    5283.15

------------------------------------------

Pipelined Results:

Benchmarking fastpower

Running 10s test @ http://localhost:8080/json
Requests/sec: 190507.34

Running 10s test @ http://localhost:8080/plaintext
Requests/sec: 199306.43

Running 10s test @ http://localhost:8080/fortune
Requests/sec:  38954.41

Running 10s test @ http://localhost:8080/queries?queries=10
Requests/sec:   5670.90

Running 10s test @ http://localhost:8080/update?queries=10
Requests/sec:   2395.86

------------------------------------------

Benchmarking whipower

Running 10s test @ http://localhost:8080/json
Requests/sec: 254655.27

Running 10s test @ http://localhost:8080/plaintext
Requests/sec: 251232.13

Running 10s test @ http://localhost:8080/fortune
Requests/sec:    196.87

Running 10s test @ http://localhost:8080/queries?queries=10
Requests/sec:    104.55

Running 10s test @ http://localhost:8080/update?queries=10
Requests/sec:     33.87

------------------------------------------

Links:

• Whip Framework source: https://github.com/mattaylor/whip/blob/master/whip.nim
• Whip TechEmpower Server: https://github.com/mattaylor/whip/blob/master/tests/whipower.nim
• benchmarking script: https://github.com/mattaylor/whip/blob/master/empower.sh

[dom96]

Indeed, that's a massive slowdown. It looks like a bug to me, can you check whether wrk reports any errors? It usually shows a count of requests which failed.

It's likely that the pipelining implementation is buggy since I threw it in there rather quickly and just made sure it works for the simple benchmarks. Can you create some code that reproduces this without your framework? That would help me take a look (I'll understand if that's not possible, I'll still do my best to have a look but it may take me significantly longer to do so).

Is this because async requests are somhow routed through asynchttpserver rather than httpbeast and therefore do not support pipelining?

Nope. In fact, AFAIK, asynchttpserver doesn't support pipelining at all.

I have am usng a server based on httpbeast with my own router and a a fork of asyncpg which address some compatibility issues with the nim devel branch.

It could be some strange interaction between asyncpg and httpbeast, hard to say. If you could reproduce it with a simple example that would be awesome.