Skip to content
This repository has been archived by the owner on Oct 30, 2019. It is now read-only.

Testing of asynchronous code #38

Closed
ashfordneil opened this issue Aug 2, 2018 · 4 comments
Closed

Testing of asynchronous code #38

ashfordneil opened this issue Aug 2, 2018 · 4 comments
Labels
WG async/await Issues relevant to the async subgroup

Comments

@ashfordneil
Copy link

Right now, rust's standard way of writing tests and the standard way of executing futures (tokio) are at odds with eachother. The test framework expects a panic on test failure - tokio catches panics and treats them as normal errors. The test framework tries to run multiple tests in parallel where it can - tokio immediately spawns one thread per CPU core when you run it. This makes testing asynchronous code pretty hard, and that's something we should fix.

I think to fix this, we need a simpler futures executor that doesn't include quite so many batteries. Instead, a run function that just runs like a regular function - without trying to do anything with threads or panic catching. Ideally, in the name of keeping test and prod environments similar, I would suggest we at least consider making tokio that simpler executor*. Also possible would be taking something like toykio further, or potentially even going the full way to supporting #[test] async fn ... in future.

*and separating all of the threading magic done by tokio out into a wrapper around a simpler core executor. This would be a big move, and I may be missing a whole bunch of reasons why its a bad idea, but on the bright side the extra flexibility could help people who

  • want really high performance, and need custom control over how things work across threads in their program
  • want a really light footprint for their code (nobody likes the app that spawns 8 threads just to transfer a file to another computer)
  • don't expect high enough load for their program that the effort to make their code threadsafe is justified for the performance gains it would bring
@carllerche
Copy link

The Tokio current_thread::Runtime spawns no threads.

@Nemo157
Copy link
Member

Nemo157 commented Aug 2, 2018

futures itself already provides a light weight executor that can be used for tests, you wouldn’t be able to use tokio types with it as they additionally require a reactor for wake notifications, but you can use in-memory pseudo-IO types with it. They’re a bit limited at the moment for test purposes as they always return Ready, I would expect at some point there will be a library for test purposes that provide in-memory types that act more like real IO types by actually returning NotReady occasionally.

The test framework expects a panic on test failure - tokio catches panics and treats them as normal errors.

This is not an issue anymore, as long as the top-level executor call returns these panics-as-errors to you, your test function can now return the error to signal a failure.

@Thomasdezeeuw
Copy link

Not related to Tokio, but creating a Future's Context is a bit hard in the context of testing. For both the LocalWaker and Executor you need something of a runtime. Maybe a future-test crate would be useful, it would provide a Context that easy is to use in (unit) tests. For example calls to wake would just add a notification of some kind to a collection. The Executor would do something along the same lines.

@Nemo157
Copy link
Member

Nemo157 commented Jun 14, 2019

In case there are subscribers to this that haven't seen it, quite a while back a futures-test-preview crate was added that contains common utilities for testing futures, e.g. constructing different kinds of Context or wrapping synchronous/in-memory async IO implementations in ones that attempt to emulate being async, see the blog post and the documentation.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
WG async/await Issues relevant to the async subgroup
Projects
None yet
Development

No branches or pull requests

5 participants