meta: Roadmap to 1.0

[carllerche]

We will be aiming to release Tokio 1.0 by the end of 2020. This issue is a high-level proposal of how to get there.

Impacted crates

The 1.0 release applies to the tokio and tokio-macros crates. The remaining crates, tokio-util and tokio-test will remain at 0.x versions.

Changes

At a high level, Tokio 1.0 will be fairly close to Tokio 0.2. Major differences are:

• Updated I/O traits (io: propose new AsyncRead / AsyncWrite traits #2716, Possibly refactor AsyncBufRead #2428).
• Remove mio from the public API (io: remove mio from the public API. #2728).
• Update to mio 0.7 (tokio: update to mio 0.7 #1190).
• Update to bytes 1.0 (io: depend on bytes 1.0 #3058).
• Move remaining async fns to an intrusive waker style (io: rework I/O driver to use an intrusive linked list for wakers #2779, fs: switch to an intrusive waker style #2927, timer: sleep future should become !Unpin #3028).
• Runtime and #[tokio::main] refinements (rt: Runtime refinements #2720).
• Tweak (or temporarily remove) DelayQueue (issue TBD).
• Misc API tweaks (meta: breaking API changes #2928).

Open questions

There are still some questions that need to be finalized.

• Coordinating with the addition of Stream in std (stream: coordinating Tokio 1.0 with the availability of Stream in std #2870).
• Consider making JoinHandle cancel on drop (Consider making JoinHandle cancel on drop #1830).

Pause on adding new APIs

Focus should be put on work that impacts existing APIs, either by altering them or removing them. Once Tokio 1.0 is released, there will be limited flexibility for altering imperfect APIs. However, it is always possible to add new APIs. There may be cases in which a new API is high value and very straight forward or a reason that adding the API after 1.0 is difficult. In this case, we can explore adding the API before 1.0.

In other words, now is the time to voice complaints about any existing API.

Target release date

Previously, Q3 2020 was mentioned as the target release date. However, productivity this year has been lower than expected due to unforeseen global events. We are now going to target the end of 2020 as a release date.

Intermediate 0.x releases

There will be a Tokio 0.3 release. This release will contain at the very least:

• New I/O traits.
• Runtime tweaks.

There also will be an "0.x" release made as a 1.0 candidate. So, there will be a minimum of two more 0.x releases before 1.0.

1.0 LTS guarantees

Tokio 1.0 will come with LTS guarantees. The proposal is:

• A minimum of 5 years of maintenance.
• A minimum of 3 years before a hypothetical 2.0 release.

The goal of these guarantees is to provide stability to the ecosystem.

MSRV

Tokio 1.0 will need a well-defined "minimum supported Rust version" policy. My proposal is:

• All 1.x Tokio releases will support at least a 6-month old Rust release.
• The MSRV will only be increased on 1.x releases.

Versioning policy

Because Tokio is pre-1.0, features have been released in 0.2.x releases. This will no longer be the case after 1.0.

• 1._.x releases should only contain bug fixes. Besides this, these releases should not substantially change runtime behavior.
• 1.x releases may contain new functionality and larger internal implementation changes.

[seanmonstar]

I assume at 1.0, we shouldn't publicly expose any non-1.0 dependencies. Or, more generally, any exposed dependencies cannot be upgraded to breaking-changes versions without that upgrade being a breaking change in Tokio.

So, since bytes is publicly exposed (the Buf and BufMut) trait, we'd either need to:

• Promote bytes to 1.0.
• Extract Buf and BufMut to some other crate that can be released at 1.0.

[hawkw]

• Extract Buf and BufMut to some other crate that can be released at 1.0.

This seems like potentially the better option, since it allows us to continue versioning bytes more aggressively? It might also encourage more experimentation with other implementations of Buf/BufMut, which seems beneficial to the ecosystem.

[carllerche]

@seanmonstar @hawkw a third option is to "hide" Buf and BufMut from the Tokio public API for the initial 1.0 release. I added that to the original issue and will write up my thoughts on it.

[seanmonstar]

It's probably also worth being sure that tokio-compat makes it real easy to try out each of these releases. The amount of refactoring needed from now on should be very low, but if it's possible to make it a cinch to try out 0.3 and 0.4, we can get more testing before solidifying our unseen mistakes forever.

[LucioFranco]

Remove mio from the public API (issue TBD).

Do we have any idea of how far mio is from 1.0?

Another question, what is our general game plan for io_uring? Do we want to go the route of a separate crate so that we can 1.0 and deal with it later? Or will it be some internal refactor of the driver?

Runtime tweaks.

What does this entail?

Otherwise, this is +1, lets for sure break this up and spread the load.

[carllerche]

@LucioFranco Removing Mio from the public API isn't really based on Mio not being 1.0 but more that it exposes too much implentation detail IMO.

"Runtime tweaks" has a linked issue w/ details.

[hawkw]

re:

• Tweak (or temporarily remove) DelayQueue (issue TBD).

Should we just put DelayQueue in util? IIRC, it doesn't depend on anything private that it needs to be in tokio for (though I'd have to double check).

[carllerche]

DelayQueue is definitely not a top priority, so if we need to cut something, we definitely can move it to tokio-util initially.

[KamilaBorowska]

I believe that #2777 should be handled in some way before release of tokio 1.0, even if that way is to remove tokio::time::{pause, resume, advance} entirely (it could be restored later). In the current state I consider this API harmful as it makes a programmer think it works when it doesn't really.

Consider the following example:

use tokio::task::JoinError;
use tokio::time::{self, Duration, Instant};

#[tokio::main]
async fn main() -> Result<(), JoinError> {
    time::pause();
    println!("{:?}", Instant::now());
    tokio::spawn(time::sleep(Duration::from_secs(10)));
    tokio::spawn(async { }).await?;
    println!("{:?}", Instant::now());
    Ok(())
}

This will advance the time by 10 seconds because of automatic time advancement, which I believe makes tokio::time::pause a broken API that cannot be used correctly with tasks. This actually did work correctly before merging in #2059, which is a breaking change that went unnoticed due to nobody using tokio::time::pause.

I think that when a time is paused sleeping should deadlock until the time is manually advanced.

[Darksonn]

@xfix Thank you for the feedback. Can you open a new issue? I will mark it with the 1.0 milestone so it doesn't get lost.

[KamilaBorowska]

@xfix Thank you for the feedback. Can you open a new issue? I will mark it with the 1.0 milestone so it doesn't get lost.

#3108

[KyleRicardo]

Can 1.0 be released this month? I'm looking forward to it!

[carllerche]

Tokio has been released: https://tokio.rs/blog/2020-12-tokio-1-0 🎉

Well done everyone ❤️

[Kixunil]

Any chance of reconsidering MSRV to be "whatever version is available in packages of popular distros"? I believe Tokio is about as important crate as std and as a Debian stable user I'm not very happy about MSRV being 1.45 - Debian has rustc 1.41.1. Not sure about other distros.

Out of curiosity I tried to compile tokio with full feature and all failures seem to be cosmetic (matches macro and u64::MAX). Doesn't seem to be too difficult to change. I would be happy to make a PR if accepted.

[dekellum]

I accept your point @Kixunil, as: why set MSRV higher for only cosmetic reasons?

However, I suspect we also have set the MSRV for the 1.0.0-1.0.1 releases at 1.45.2 because it can only be changed in a minor release, and it problably won't be frequently changed, at that. In other words, by setting it higher then absolutely necessary now, Tokio gets some room to expand into using those MSRV compiler features.

Said another way, when Tokio does increase the MSRV, in a minor release, it will most likely always be to the newest compiler that is at least six (6) months old at that time. Am I making any sense? Tokio maintainers please correct me if I have it wrong.

[KamilaBorowska]

"whatever version is available in packages of popular distros" is pretty much "the oldest Rust version that can be used to build oldest supported Firefox ESR release". In theory, this isn't a bad idea, Firefox ESR releases every year ensuring updates being rather common. Firefox uses newest Rust 2 weeks after it is released, and there are 2 months from nightly soft freeze and stable release date. Additionally there are at least three months where Firefox ESR support for different ESR versions overlaps. This sums up to 5 and a half months, which is close enough to 6 months guaranteed by tokio's rolling MSRV policy.

Of course, it's worth noting that while Firefox's Rust version policy indicates that a given release of Firefox can use features from a newest stable Rust release 2 weeks after it is released, it's not necessarily the case that it actually will - in fact, Firefox 78 could have used features from Rust 1.43, but didn't, restricting itself to Rust 1.41 features.

That said, following Rust releases required for oldest supported Firefox ESR has a notable advantage of pretty much everybody having access to those versions - most Linux distributions include Firefox or Firefox ESR, and need a modern enough Rust in their repositories to build Firefox.

[benesch]

Out of curiosity I tried to compile tokio with full feature and all failures seem to be cosmetic (matches macro and u64::MAX). Doesn't seem to be too difficult to change. I would be happy to make a PR if accepted.

As I understand it, it’s not so much about using the minor cosmetic features available in Rust v1.45, but about the lag between when a new feature is added to Rust and when Tokio can make use of it.

Concretely, std::stream::Stream will be added to Rust soon, and will unlock some nice stream-based APIs in Tokio that had to be removed for 1.0. With the current policy, Tokio will be able to release stream support 6mo after it stabilizes in Rust. If that length of time were increased to, say, a year, that means we’ll have to wait twice as long for stream support in Tokio.

6mo seems like a good balance, personally, but I can certainly appreciate why folks would want to move the needle in one direction or the other. Just wanted to flag that the general policy question is unfortunately not as simple as just removing some uses of matches!.

[Darksonn]

Afaik, mio v0.8 will have a MSRV of 1.46, so we would also have to increase the MSRV to upgrade mio, and that happens already in February.

[Nemo157]

Concretely, std::stream::Stream will be added to Rust soon, and will unlock some nice stream-based APIs in Tokio that had to be removed for 1.0. With the current policy, Tokio will be able to release stream support 6mo after it stabilizes in Rust. If that length of time were increased to, say, a year, that means we’ll have to wait twice as long for stream support in Tokio.

It doesn't seem to be mentioned anywhere, but I would hope that Tokio allows for non-default features to have distinct MSRV policies in order to have optional std::stream::Stream support when it is released instead of 6 months later. Might be something to bring up on #3386 and clarify in the documentation it adds.

[carllerche]

@Nemo157 we won’t have different MSRV per feature flag. I do think we can use a conditional compilation strategy. When std::Stream is available, use that, otherwise use a sealed identical Stream trait defined by Tokio.

[carllerche]

@Kixunil if you want to propose lowering the MSRV, open an issue with the pros / cons. I am not super familiar with the distro policies for packaging rustc

[Nemo157]

Ah yeah, an autocfg approach would work too.

[Kixunil]

@carllerche thanks for reply, I opened #3412