Skip to content
GitHub no longer supports this web browser. Learn more about the browsers we support.
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade to `tokio v0.2` and `std::future::Future` #1142

Open
LucioFranco opened this issue Nov 4, 2019 · 2 comments
Open

Upgrade to `tokio v0.2` and `std::future::Future` #1142

LucioFranco opened this issue Nov 4, 2019 · 2 comments

Comments

@LucioFranco
Copy link
Member

@LucioFranco LucioFranco commented Nov 4, 2019

Background

Current runtime and futures version

Originally, when we first started working on vector only futures 0.1 and tokio 0.1 was released. There was no language level support for futures nor async/await. In that time since we first started working on vector, we have seen the addition of std::future::Future, std::task::{Context, Poll}, and the async_await feature (which lands on stable Nov 7th).

Async/await itself is a huge shift in how we write futures and in my experience has reduced the amount of code by up to 40-60%. The biggest benefit is that it allows us to write more "rusty" code because we can now borrow accross yield points which was not possible before without a lot of work.

Tokio 0.2

As some of you may have seen tokio has recently implemented a new scheduler along with its upgrade from futures 0.1 to std::future::Future. This new scheduler has a couple of improvements that impact us directly. Notably less aggressive work-stealing and better optimizations around sendings on channels. You can read more in this blog post.

The biggest issue with moving to tokio 0.2 will be the fact that we have a lot of code that uses tokio-reactor 0.1. All tokio 0.2 runtimes do not provide access to this reactor and thus our code will not work if we just swap out runtimes. To fix this issue the tokio team has written a tokio-compat runtime which uses the tokio 0.2 executor/scheduler, tokio-net (which is the new reactor for 0.2), tokio-timer 0.2 and a background tokio-reactor 0.1/tokio-timer 0.1. This allows us to execute futures that are written in both futures 0.1 and std::future::Future as well as futures that are from tokio 0.1 and tokio 0.2.

Performance

During my benchmarking this weekend I was able to see up to 6x performance increases and on average at least a 100% increase in performance in just upgrading the executor to tokio 0.2.

Benchmarks comparing tokio v0.1.22 and tokio-compat rev f64aee/tokio rev e699d4653

Note: tests with a change +/-5 were ignored.

group                                                     tokio01                                 tokio02
-----                                                     -------                                 -------
batch/no compression 10mb with 2mb batches                1.11      8.3±0.24ms  1154.7 MB/sec     1.00      7.4±0.14ms  1284.5 MB/sec
elasticsearch_indexes/dynamic                             1.12    606.0±0.99ns        ? B/sec     1.00    542.1±3.66ns        ? B/sec
elasticsearch_indexes/static                              1.23    110.2±0.29ns        ? B/sec     1.00     89.8±0.22ns        ? B/sec
http/http_gzip                                            2.01    231.9±6.54ms    41.1 MB/sec     1.00    115.6±3.89ms    82.5 MB/sec
http/http_no_compression                                  2.00   227.6±10.62ms    41.9 MB/sec     1.00    114.0±3.08ms    83.7 MB/sec
interconnected                                            1.12    274.5±6.67ms    69.5 MB/sec     1.00    244.9±3.68ms    77.9 MB/sec
partitioned_batch/no compression 10mb with 2mb batches    1.08     12.5±0.33ms   760.4 MB/sec     1.00     11.6±0.33ms   819.5 MB/sec
pipe                                                      3.19    361.0±1.70ms    26.4 MB/sec     1.00    113.3±2.55ms    84.2 MB/sec
pipe_with_huge_lines                                      1.16   1028.3±4.74ms   185.5 MB/sec     1.00   883.8±14.71ms   215.8 MB/sec
pipe_with_many_writers                                    6.09   717.5±18.69ms    13.3 MB/sec     1.00    117.9±1.14ms    80.9 MB/sec
pipe_with_tiny_lines                                      3.43    305.7±6.16ms   319.5 KB/sec     1.00     89.1±0.65ms  1096.0 KB/sec
transforms                                                3.09    556.9±3.29ms    18.8 MB/sec     1.00    180.0±2.99ms    58.3 MB/sec
unflatten array                                           1.06  1636.0±136.63ns        ? B/sec    1.00   1540.8±9.39ns        ? B/sec

A c5.4xlarge was used. Code can be found here https://github.com/timberio/vector/tree/lucio/compat-bench

Some of these performance numbers are extremely good and slightly confirm some of the theories we've had about certain performance issues that we've seen with vector.

Proposal

I suggest that we start moving to the tokio-compat runtime which will allow us to support tokio 0.1 and tokio 0.2 items. This will also enable us to spawn std::future::Future and future 0.1 futures at the same time. This should then allow us to incrementally move to the new futures version and async/await.

The actual code change can be done in about 10 lines of code since we already refactored the runtimes out in #1098. That said there are a few blockers:

  • The file sink will need a rewrite since it uses tokio 0.1 blocking annotations which are not currently supported in tokio-compat. This is probably the biggest change that we will need but the sink is very new and I don't think its in a very mature state like some of our others. I have a couple ideas how to refactor this sink easily but it should not be much work since the sink is very simple.
  • We have two places where tokio_threadpool::blocking is used which will not work in tokio-compat. We can just upgrade these to the new tokio::executor::thread_pool::blocking fn that was added recently.
  • The last blocking issue is refactoring our Topologly::stop to use our own shutdown mechanism. tokio 0.2 will not provide the same shutdown api that tokio 0.1 did. Our current shutdown code and Topology::stop implementation has already show some issues and this would be good to combine with #1091.

Beyond these few blockers, I was able to run almost all our tests and most of our benchmarks this weekend with no changes done to them.

Before we move to this new runtime I would also like to ensure that we run it under a week of validation using @lukesteensen's reliability work and run it against our test harness that @binarylogic wrote.

You can look here to see how easy it was to move to the new runtime d56a945

This content would also be a great technical blog post talking about how the new executor improved performance and other novel rust futures-based things.

cc @a-rodin @Jeffail @lukesteensen @binarylogic

@LucioFranco

This comment has been minimized.

Copy link
Member Author

@LucioFranco LucioFranco commented Nov 4, 2019

cc @hawkw I know you've been working on some of these blocking issues but there might be more info here that you find interesting and could be helpful.

@lukesteensen

This comment has been minimized.

Copy link
Member

@lukesteensen lukesteensen commented Nov 7, 2019

Sounds great! I think the compat runtime is definitely the way to go. If we can slip it in with minimal changes, we can start to run some tests against that diff alone before tackling any conversions.

You can look here to see how easy it was to move to the new runtime master...lucio/compat-benchdiff-8a2d636766ae1fa1c9124070ce519445R3-R71

It looks like this link is broken.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.