Migrate to async futures with reqwest

[messense]

No description provided.

[fcsonline]

Wow! Awesome job! 👏

Let me test it with some common use cases to give you come feedback.

[fcsonline]

In the examples/server you can start a small nodejs server to test some of the example we have. I have played with the throughput.yml one, tweaking the number of threads and iterations to (10 and 1000 respectively) and I have seen those numbers.

Master

Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   3ms
Fetch users               Average time per request  3ms
Fetch users               Sample standard deviation 1ms

Concurrency Level         10
Time taken for tests      3.0 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       3321.76 [#/sec]
Median time per request   3ms
Average time per request  3ms
Sample standard deviation 1ms

Async:

Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   17ms
Fetch users               Average time per request  19ms
Fetch users               Sample standard deviation 6ms

Concurrency Level         10
Time taken for tests      21.3 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       469.37 [#/sec]
Median time per request   17ms
Average time per request  19ms
Sample standard deviation 6ms

Do you know the reason why the number of requests per second is much lower?

[messense]

Do you know the reason why the number of requests per second is much lower?

Yes, it is expected to be slower in its current form, it still runs requests sequentially in each thread which isn't ideal. This patch is the minimal changes to make drill work with async-await, refactoring needed to actually utilize the async runtime.

[messense]

@fcsonline Can you try again with latest commit? On my local machine (MacBook Pro (15-inch, 2018) 2.2 GHz 6-Core Intel Core i7 16 GB 2400 MHz DDR4) the result is:

drill sync

Threads 10
Iterations 1000
Rampup 0
Base URL http://localhost:9000


Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   23ms
Fetch users               Average time per request  32ms
Fetch users               Sample standard deviation 104ms

Concurrency Level         10
Time taken for tests      32.0 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       312.92 [#/sec]
Median time per request   23ms
Average time per request  32ms
Sample standard deviation 104ms

drill async

Threads 10
Iterations 1000
Rampup 0
Base URL http://localhost:9000


Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   17ms
Fetch users               Average time per request  21ms
Fetch users               Sample standard deviation 16ms

Concurrency Level         10
Time taken for tests      21.6 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       463.52 [#/sec]
Median time per request   17ms
Average time per request  21ms
Sample standard deviation 16ms

[messense]

I've tested on a Linux server, async version is still slower.

Sync

Threads 10
Iterations 1000
Rampup 0
Base URL http://localhost:9000


Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   2ms
Fetch users               Average time per request  2ms
Fetch users               Sample standard deviation 1ms

Concurrency Level         10
Time taken for tests      2.2 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       4543.37 [#/sec]
Median time per request   2ms
Average time per request  2ms
Sample standard deviation 1ms

Async

Threads 10
Iterations 1000
Rampup 0
Base URL http://localhost:9000


Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   2ms
Fetch users               Average time per request  2ms
Fetch users               Sample standard deviation 0ms

Concurrency Level         10
Time taken for tests      9.8 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       1018.73 [#/sec]
Median time per request   2ms
Average time per request  2ms
Sample standard deviation 0ms

[messense]

By keep reqwest::Client around instead of recreating it on every iteration the async version's performance is now on par with sync

Sync:

Threads 10
Iterations 1000
Rampup 0
Base URL http://localhost:9000


Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   2ms
Fetch users               Average time per request  2ms
Fetch users               Sample standard deviation 1ms

Concurrency Level         10
Time taken for tests      2.2 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       4538.68 [#/sec]
Median time per request   2ms
Average time per request  2ms
Sample standard deviation 1ms

Async

Threads 10
Iterations 1000
Rampup 0
Base URL http://localhost:9000


Fetch users               Total requests            10000
Fetch users               Successful requests       10000
Fetch users               Failed requests           0
Fetch users               Median time per request   2ms
Fetch users               Average time per request  2ms
Fetch users               Sample standard deviation 1ms

Concurrency Level         10
Time taken for tests      2.0 seconds
Total requests            10000
Successful requests       10000
Failed requests           0
Requests per second       5046.84 [#/sec]
Median time per request   2ms
Average time per request  2ms
Sample standard deviation 1ms

[messense]

This needs a little refactor to pass no_check_certificate config to reqwest::Client in Request::new

[fcsonline]

Don't worry about it for now. After merging this async branch I want to start dealing with the keep-alive feature. It is a requirement for a long time that will increase the throughput drastically.

The idea is to have a pool of clients, (domain -> Client), to be used here. The domain will be determined from the interpolated_url.

[fcsonline]

I created this branch forked form this one to play a bit. e6b51ff

Keep Alive is awesome! 💪

[fcsonline]

Unlocking async will be an awesome milestone for drill. I have been testing and I'm really happy with the current performance. Right after this, I can open the keep-alive pull request(e6b51ff) that will improve even more throughput. Also, it enables two more branches I'm working on.

What do you think about merging this?

[messense]

@fcsonline Let's merge it.