Add support for Expect: 100-continue

[sanikolaev]

Manticore currently doesn't support HTTP header Expect: 100-continue which some clients (e.g. curl) send and then wait for some time before sending POST data.

The specification of this functionality on HTTP level can be found here:

• https://datatracker.ietf.org/doc/html/rfc2616#section-10.1.1
• https://datatracker.ietf.org/doc/html/rfc2616#section-8.2.3

The task is to add support for Expect: 100-continue the way that Manticore always confirms immediately the client can continue.

[githubmanticore]

➤ Sergey Nikolaev commented:

Just an example of the performance difference this issue can bring:

w/o `Expect: ```

snikolaev@dev:~$ php load_json.php 100 16 100000 
preparing... 
100%       querying... 
finished inserting 
Total time: 63.29199886322 
1580 docs per sec 

with Expect: :

    curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/x-ndjson', 'Expect: ']); 

snikolaev@dev:~$ php load_json.php 100 16 100000 
preparing... 
found in cache 
querying... 
finished inserting 
Total time: 1.2009081840515 
83270 docs per sec 

[sanikolaev]

Done in ffc5d0e

Previously:

snikolaev@dev:~$ time curl -v -sX POST http://localhost:9318/bulk -H "Content-Type: application/x-ndjson" --data '{"insert": {"index": "user", "doc":  {"name":"Prof. Matt Heaney IV","email":"ibergnaum@yahoo.com","description":"Tempora ullam eaque consequatur. Vero aut minima ut et ut omnis officiis vel. Molestiae quis voluptatum sint numquam.","age":15,"active":1}}}
{"insert": {"index": "user", "doc":  {"name":"Prof. Boyd McKenzie","email":"carlotta11@hotmail.com","description":"Blanditiis maiores odio corporis eaque illum. Aut et rerum iste. Neque et ullam quisquam officia dignissimos quo cumque.","age":84,"active":1}}}
{"insert": {"index": "user", "doc":  {"name":"Mr. Johann Smith","email":"stiedemann.tristin@ziemann.com","description":"Temporibus amet magnam consequatur omnis consequatur illo fugit. Debitis natus doloremque est tempore deserunt vero. Harum eos corrupti nemo ut.","age":89,"active":1}}}
{"insert": {"index": "user", "doc":  {"name":"Hector Pouros","email":"hickle.mafalda@hotmail.com","description":" as voluptatem inventore sit. Aliquam fugit perferendis est id aut odio et sapiente.","age":64,"active":1}}}'
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9318 (#0)
> POST /bulk HTTP/1.1
> Host: localhost:9318
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/x-ndjson
> Content-Length: 1025
> Expect: 100-continue
>
* Done waiting for 100-continue
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Server: 4.2.0 15e927b@211223 release (columnar 1.11.4 327b3d4@211223)
< Content-Type: application/json; charset=UTF-8
< Content-Length: 434
<
* Connection #0 to host localhost left intact
{"items":[{"insert":{"_index":"user","_id":2811798918248005633,"created":true,"result":"created","status":201}},{"insert":{"_index":"user","_id":2811798918248005634,"created":true,"result":"created","status":201}},{"insert":{"_index":"user","_id":2811798918248005635,"created":true,"result":"created","status":201}},{"insert":{"_index":"user","_id":2811798918248005636,"created":true,"result":"created","status":201}}],"errors":false}
real	0m1.022s
user	0m0.001s
sys	0m0.010s

Now:

snikolaev@dev:~$ time curl -v -sX POST http://localhost:9318/bulk -H "Content-Type: application/x-ndjson" --data '{"insert": {"index": "user", "doc":  {"name":"Prof. Matt Heaney IV","email":"ibergnaum@yahoo.com","description":"Tempora ullam eaque consequatur. Vero aut minima ut et ut omnis officiis vel. Molestiae quis voluptatum sint numquam.","age":15,"active":1}}}
{"insert": {"index": "user", "doc":  {"name":"Prof. Boyd McKenzie","email":"carlotta11@hotmail.com","description":"Blanditiis maiores odio corporis eaque illum. Aut et rerum iste. Neque et ullam quisquam officia dignissimos quo cumque.","age":84,"active":1}}}
{"insert": {"index": "user", "doc":  {"name":"Mr. Johann Smith","email":"stiedemann.tristin@ziemann.com","description":"Temporibus amet magnam consequatur omnis consequatur illo fugit. Debitis natus doloremque est tempore deserunt vero. Harum eos corrupti nemo ut.","age":89,"active":1}}}
{"insert": {"index": "user", "doc":  {"name":"Hector Pouros","email":"hickle.mafalda@hotmail.com","description":" as voluptatem inventore sit. Aliquam fugit perferendis est id aut odio et sapiente.","age":64,"active":1}}}'
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 9318 (#0)
> POST /bulk HTTP/1.1
> Host: localhost:9318
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: application/x-ndjson
> Content-Length: 1025
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
< Server: 4.2.1 63e5749@220405 dev
< Content-Type: application/json; charset=UTF-8
< Content-Length: 0
* We are completely uploaded and fine
< HTTP/1.1 200 OK
< Server: 4.2.1 63e5749@220405 dev
< Content-Type: application/json; charset=UTF-8
< Content-Length: 147
<
* Connection #0 to host localhost left intact
{"items":[{"bulk":{"_index":"user","_id":2811798919590182916,"created":4,"deleted":0,"updated":0,"result":"created","status":201}}],"errors":false}
real	0m0.015s
user	0m0.005s
sys	0m0.004s