haywire will now keep allocating memory as required, as it's not guar…

[nmdguerreiro]

…anteed that we'll get one and only one fully formed request on each combination of alloc/read calls.

Previously, we were assuming that the alloc callback was only called once per request and that the read callback was called when all the data was available.
Unfortunately, that's not the case, and the read callback may be called when only a fraction of the incoming data is available, especially under high concurrency.

Buffers are now managed in http_request_buffers.*. We use a mark/sweep technique to get rid of requests previously handled by a connection whenever a new buffer chunk is fully processed. This allows us to keep a relatively small buffer. Also, it's now possible to register interest in a given memory region by placing a pin on it and retrieving its location once the request has been fully read in.

We will now also deal with bad requests that were causing crashes before, requests that are too long to be processed and that could cause us to run out of memory. Unknown errors are also dealt with now, by returning an appropriate error code.

Removed redundant header imports.

Makes #93 redundant.

Benchmarks

My tests were run on two m4.xlarge machines on AWS with 4 cores each, with the server being run with: ./build/hello_world --threads 4. The benchmark was run using wrk: ./wrk -c 300 -t 300 -d 1m -s pipelined_get.lua --latency http://172.31.5.65:8000 -- 64.

Results

master

Running 1m test @ http://172.31.5.65:8000
  300 threads and 300 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   103.53ms  173.03ms   1.99s    91.22%
    Req/Sec     2.45k     1.56k   49.58k    68.31%
  Latency Distribution
     50%   23.92ms
     75%  146.86ms
     90%  254.43ms
     99%  834.11ms
  34420176 requests in 1.00m, 5.64GB read
  Socket errors: connect 0, read 0, write 0, timeout 28
Requests/sec: 572704.23
Transfer/sec:     96.13MB

this change

Running 1m test @ http://172.31.5.65:8000
  300 threads and 300 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    87.12ms  145.41ms   1.80s    91.38%
    Req/Sec     2.69k     1.64k   44.80k    54.73%
  Latency Distribution
     50%   18.67ms
     75%  128.81ms
     90%  217.65ms
     99%  677.48ms
  39001682 requests in 1.00m, 5.59GB read
  Socket errors: connect 0, read 0, write 0, timeout 8
Requests/sec: 648948.92
Transfer/sec:     95.31MB

cc'ing @jpz @botdes @violetta-baeva

[kellabyte]

I've benchmarked this on my high performance 10GbE environment.

Master

Running 10s test @ http://server:8000
  40 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    13.28ms   52.23ms   1.69s    94.56%
    Req/Sec   127.07k    41.48k  290.75k    70.97%
  Latency Distribution
     50%    6.32ms
     75%   18.23ms
     90%   37.69ms
     99%    0.00us
  50762199 requests in 10.10s, 8.32GB read
  Socket errors: connect 0, read 177, write 0, timeout 0
  Non-2xx or 3xx responses: 1487
Requests/sec: 5026888.03
Transfer/sec:    843.71MB

This PR

Running 10s test @ http://server:8000
  40 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    12.06ms   45.48ms   1.70s    94.71%
    Req/Sec   135.44k    39.22k  306.90k    72.12%
  Latency Distribution
     50%    6.17ms
     75%   16.82ms
     90%   37.30ms
     99%    0.00us
  54164776 requests in 10.10s, 7.77GB read
  Socket errors: connect 0, read 70, write 0, timeout 0
Requests/sec: 5364399.47
Transfer/sec:    787.85MB

[kellabyte]

Is there a reason why we don't want debug information? I don't really know a lot about these flags. This could be a great idea, please educate me :)

[nmdguerreiro]

Here's the difference when it comes to compilation flags when using Release and `RelWithDebInfo``:

//Flags used by the compiler during release builds.
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG

//Flags used by the compiler during release builds with debug info.
CMAKE_C_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG

We're already setting -O3, so the only other difference is the usage of -g, which enables debug symbols to be embedded in the executable. It won't change the code paths, but it will make the executable a bit larger and may or may not cause additional page faults. I guess that's not a big deal given that Haywire is relatively small though.

I'm happy to revert the change if you prefer to have debug symbols by default.

[kellabyte]

Yeah lets keep release with debug info. I talked to a bunch of people to just gauge what is common and it seems the sentiment is that if there is a crash having a stack trace is valuable.

[nmdguerreiro]

Will get this one reverted.

[kellabyte]

I learned from @hyc that debug info is in its own segment of the file and never gets paged in during a regular run.

[kellabyte]

I've reviewed this enough to be comfortable and this is a really important PR so 👍 on merging this after making the discussed changes to void* arguments and some of the comments.

When you're done, feel free to merge! I'm excited about this change :)

When you have the diagrams complete let me know. I'll find a good spot for them in the repo :)

[nmdguerreiro]

Merging then. I've added documentation on how buffers are handled in https://github.com/haywire/haywire/blob/1ff747bbe00a69d7c206764299824c5e51d0b52f/docs/buffers.md.