RFC: Evolving Rust through Epochs

[aturon]

Rust's ecosystem, tooling, documentation, and compiler are constantly improving. To make it easier to follow development, and to provide a clear, coherent "rallying point" for this work, this RFC proposes that we declare a epoch every two or three years. Epochs are designated by the year in which they occur, and represent a release in which several elements come together:

• A significant, coherent set of new features and APIs have been stabilized since the previous epoch.
• Error messages and other important aspects of the user experience around these features are fully polished.
• Tooling (IDEs, rustfmt, Clippy, etc) has been updated to work properly with these new features.
• There is a guide to the new features, explaining why they're important and how they should influence the way you write Rust code.
• The book has been updated to cover the new features.
  • Note that this is already required prior to stabilization, but in general these additions are put in an appendix; updating the book itself requires significant work, because new features can change the book in deep and cross-cutting ways. We don't block stabilization on that.
• The standard library and other core ecosystem crates have been updated to use the new features as appropriate.
• A new edition of the Rust Cookbook has been prepared, providing an updated set of guidance for which crates to use for various tasks.

Sometimes a feature we want to make available in a new epoch would require backwards-incompatible changes, like introducing a new keyword. In that case, the feature is only available by explicitly opting in to the new epoch. Existing code continues to compile, and crates can freely mix dependencies using different epochs.

Rendered

Update: there's a Request for Explanation podcast episode about this RFC, which is a good way to quickly get up to speed!

Edit: fixed rendered link

[aturon]

People may be interested to review a recent draft from the C++ world, C++ Stability, Velocity, and Deployment Plans, which tries to clarify plans around C++'s evolution and has many similarities with this RFC.

[steveklabnik]

I have long been advocating for a "never Rust 2.0", and I see this RFC as reaffirming our commitment to stability.

I can't tell the future, of course, but for me, this RFC is the nail in the ⚰️ . No Rust 2.0! ❣️

[est31]

I came to Rust because it was stable, and value stability greatly. If this RFC means that I don't have to suffer from the new module system but can continue to use the current, magnificent one, I'm a great supporter!

[steveklabnik]

I don't have to suffer from the new module system but can continue to use the current, magnificent one,

To be clear, there isn't a module system proposal yet 😉

[cmr]

(amusingly, I was just complaining about the lack of this exact feature and philosophy in the OPLSS chat)

[SimonSapin]

ATCs, impl Trait, and specialization all becoming available […] 2018

Be careful what things you put in the same sentence, it could sound like a promise ;)

[cuviper]

Will all newly-stabilized features only go into the current epoch? There could be some new things that would be perfectly fine on the older epochs too. My gut says to only push forward, but I could imagine some cases might be harder to try and isolate their implementations.

Is it a semver-breaking change for a crate to start requiring a new epoch? Bumping rustc requirements at all was controversial in #1619.

Will cargo new always default to the newest epoch right away?

[Ixrec]

Rust compilers are expected to support multiple epochs, and a crate dependency graph may involve several different epochs simultaneously. Thus, epochs do not split the ecosystem nor do they break existing code.

existing deprecations may turn into hard errors ... This is the only kind of change a new epoch can make.

the RFC proposes to limit epochal changes to be "superficial", i.e. occurring purely in the early front-end stages of the compiler

That answers every single question and concern I had about epochs in exactly the way I hoped they'd be answered, and even the parts of this RFC that I was not expecting (e.g. epoch previews) make perfect sense. +1000. Let's do this.

Though it might be worth explicitly stating that the pretty graph with green and orange bars is not committing us to actually creating those specific epochs at those specific future dates.

[aatxe]

Evidence of @cmr lamenting this missing feature.

[Ixrec]

@cuviper The cargo question is answered explicitly in the RFC:

cargo new will produce a Cargo.toml with the latest epoch value, including -preview epochs when applicable.

The semver one less explicitly, but I believe it's very strongly implied by this quote that there is no such thing as "requiring" an epoch and merely using a new epoch in your crate won't break any client code:

To be crystal clear: Rust compilers are expected to support multiple epochs, and a crate dependency graph may involve several different epochs simultaneously. Thus, epochs do not split the ecosystem nor do they break existing code.

But it wouldn't hurt to be even more crystal clear and say "there is no such thing as 'requiring' a minimum epoch of client code, only requiring minimum rustc/cargo versions" assuming I am interpreting this right.

[aturon]

@cuviper

Will all newly-stabilized features only go into the current epoch? There could be some new things that would be perfectly fine on the older epochs too. My gut says to only push forward, but I could imagine some cases might be harder to try and isolate their implementations.

Ooh, that is a very interesting question that the RFC doesn't address at all. I pretty much agree with both of your sentiments here, and hence suspect we won't have a hard rule, but rather a default tendency. In particular, the benefits of collecting bundles of features together under an umbrella name are decreased somewhat if some of them become available in older epochs.

Is it a semver-breaking change for a crate to start requiring a new epoch? Bumping rustc requirements at all was controversial in #1619.

Another good question. I personally would like to see this issue addressed through LTS releases, which could act as a point of coordination for users wanting to be conservative with their upgrades. I don't think epochs introduce anything particularly new here.

In particular, "requiring" a new epoch means nothing more than requiring a new compiler version, so it reduces to the same question we have today (which we don't have a good answer for yet.)

Will cargo new always default to the newest epoch right away?

Yes. Part of the benefit of keeping the rapid release model at the center here is that we should be able to avoid the usual pitfalls of a "X.0" release.

[cuviper]

Right, a new epoch requires a new compiler, so yes it's basically the same as bumping rustc generally.

One more - If a 2018+ crate wants to use a 2015 crate with catch in its API, are they just out of luck?

[est31]

To be clear, there isn't a module system proposal yet

@steveklabnik yes, that's right. The proposals I saw didn't change much in the past few weeks so I'd be suprised if it turned out much different from what was proposed initially on irlo months ago. I don't want to go into the details, as continuing this discussion seems not appropriate for this thread. I only wanted to give motivation for why I like this RFC.

The general point I wanted to make is that I clearly don't want to adapt my code to entirely new concepts/systems every 2 years. Knowing that the proponents of such changes will get them through one way or another, I really do prefer if they were done cleanly (this also helps the feature itself!) and done in newer "epochs" of Rust.

The biggest risk of going with this RFC is I think that it may be misunderstood by people that Rust's stability promise is broken, and making people distrust the promises made by the language, turning them away. To emphasize that epochs are not about breakage but about allowing future evolution, I would really like if it isn't just possible but also accepted and not a sign of bad style to stay with your evolving codebase on an older epoch.

[Ixrec]

In particular, the benefits of collecting bundles of features together under an umbrella name are decreased somewhat if some of them become available in older epochs.

I suspect the most useful "bundles of features" are ones where some of the features don't come into their own until the rest of the bundle is also stabilized. Like how impl Trait is useful today, but practically mandatory when futures and async/await are involved. Or how CTFE and const fn aren't that useful without each other. So in that sense I don't think stabilizing some "non-breaking" features will weaken a good "bundle".

[est31]

One more - If a 2018+ crate wants to use a 2015 crate with catch in its API, are they just out of luck?

I guess there is a limit to the compatibility this RFC can provide?

[SimonSapin]

Code that compiles without warnings on the previous epoch (under the latest compiler release) will compile without warnings or errors on the next epoch

As written, I think this excludes ever adding new warnings. You may want to weaken this to "without errors on the next epoch" (but not necessarily without warnings), or "without warnings or errors in the first compiler version that supports the next epoch".

Even then, deprecated stuff cannot be removed or repurposed until two epochs later. This is nice for stability, but maybe a bit heavy for language evolution flexibility. If it is intended, the RFC should mention this.

[Manishearth]

This is interesting.

I prima facie agree with this choice. It makes sense for the CLI tool to default to the latest epoch for ease of use.

But this does bring into question the stability of said CLI tool. We have thus far treated rustc as a tool with stable flags and behavior; with the stable release not having any unstable flags (etc).

However, this does mean that rustc foo.rs may stop compiling if you upgrade your compiler (for reasons other than soundness bugfixes yada yada), which is not "stable" as I understand the current definition of stability for the rustc CLI tool.

We can, of course, choose to change this definition (or I may simply be wrong as to how we evaluate stability for rustc) but I do think that this is a subtlety that should be highlighted and perhaps discussed more. So far we largely assume that most consumers of rustc are using it through cargo, but we are likely to have more and more direct consumers as it gets integrated with alternate build systems (like Bazel). We can make it the responsibility of these build systems to explicitly set the epoch, but again, this is technically a breaking change.

[aturon]

Yeah, I figured that people with stability concerns here would just always pin an epoch in their arguments. However, I think this is a small detail that could easily go either way.

[cuviper]

FWIW, this can also happen with GCC when they update the default -std option.

[steveklabnik]

Yes, the drafts I saw here said "default to the 2015 epoch"; we should fix this, IMO.

[SimonSapin]

I read through the entire RFC but missed this detail. I saw that Cargo defaults to epoch = "2015" and assumed that rustc would too.

I think this should be reconsidered. Rust-aware build systems other than Cargo can also decide to enable the latest epoch as part of their template for new projects.

[aturon]

@cuviper

One more - If a 2018+ crate wants to use a 2015 crate with catch in its API, are they just out of luck?

Another good one :-)

There are two ways I can see this going:

• We can introduce some mechanism for referring to identifiers named as keywords. It might be enough to allow use of keywords within renamed imports, e.g. use foo::catch as catch_;. We'd need to support method renaming, however.

• Worst case, you can write a shim crate that uses epoch 2015 but re-exports renamed APIs. That can get tricky with traits, though...

[aturon]

@SimonSapin

Code that compiles without warnings on the previous epoch (under the latest compiler release) will compile without warnings or errors on the next epoch

As written, I think this excludes ever adding new warnings. You may want to weaken this to "without errors on the next epoch" (but not necessarily without warnings), or "without warnings or errors in the first compiler version that supports the next epoch".

Good point. It's probably enough to say "without errors"

Even then, deprecated stuff cannot be removed or repurposed until two epochs later. This is nice for stability, but maybe a bit heavy for language evolution flexibility. If it is intended, the RFC should mention this.

I don't quite follow this point. Examples like the catch keyword involve deprecation in one epoch, then immediate removal/replacement in the next. I don't see how that violates the rule above.

[Ixrec]

@aturon A third option would be adding #[cfg(epoch >= 2018)], which would allow crate authors affected by such unfortunate corner cases to swap their catch method for do_catch or whatever else is needed to support >=2018 clients while still staying on the old epoch for the most part.

My guess is we don't actually want to do that, as I can easily imagine overuse of it in a sufficiently large dependency graph leading to compatibility nightmares, and potentially encouraging "people can just cfg it" arguments for epoch changes we would otherwise rightly reject as too breaking. But it seems worth mentioning.

[aturon]

Update: per discussion with @Manishearth, I've changed the RFC to default to the 2015 epoch when no flag is given to rustc, since that's more consistent with our general stability story. @nikomatsakis seemed to prefer the latest epoch, but I'll let him argue that point.

[cuviper]

Do macros operate with the epoch in their crate of origin? I guess they must, but there could be some hairy interactions with stuff that's only legal in the epoch that instantiated it. e.g. oldvec![new-expr]

[aturon]

@cuviper

Procedural macros, in particular, are going to be "interesting" since we can stitch together token streams potentially coming from a mixture of sources.

I haven't thought extensively about this, but I think the ideal behavior would involve some kind of "coloring" of token trees with their epoch of origin. That works well for keywords, since we make the distinction at the lexer level and don't otherwise mind if an identifier happens to have the same name as a keyword.

Other concerns, such as changes to type inference, would likely be much harder to provide a "coloring" strategy for.

I suspect the best we can do is: any time we make an epoch-dependent change, the RFC should contain a section laying out the plan for handling (procedural) macros.

[SimonSapin]

Even then, deprecated stuff cannot be removed or repurposed until two epochs later.

I don't quite follow this point. Examples like the catch keyword involve deprecation in one epoch, then immediate removal/replacement in the next. I don't see how that violates the rule above.

Let’s assume that catch didn’t make it for 2018. let catch = 4; was not a warning in epoch 2015, so the bit of RFC text quoted earlier forbids making it a warning in epoch 2018. We’d have to wait for the next one, let’s call it 2020, to deprecate catch as an identifier and then yet another cycle to be able to make it a keyword in 2022.

So I think we want to drop the "code with warning in one epoch emits no warning in the next" rule. (Reduce it to "no hard error in the next".)

[aturon]

@SimonSapin ah I see, it was based on reading the literal words I wrote, as opposed to what I had in my head :-)

I've pushed a fix.

[est31]

Just for clarity, this will fully replace the current pattern of first linting and then turning that lint into error by default then into a hard error, which is being done right now (also has a tag) for smaller breaking changes?

[Aaronepower]

How will multi epoch transitions work, especially in relation to the example of changing the Trait syntax? If for example someone writes a program for a company in current Rust, the program's code isn't touched much for a couple of years till lets say mid 2020 and the programmer decides they want upgrade the code. If they simply added epoch = "2020" to Cargo.toml couldn't they have a program that works in both 2015 Rust and 2020 Rust that behaves differently?

Similarly how will the rustfix handle transitioning from arbitrary epochs to the latest? Does it essentially run older versions of itself till it's just at the last old epoch and then run the new changes?

[scottmcm]

@konstin Fixed; thanks for noticing.

[phaylon]

Personally, I would prefer breaking changes behind feature-flags with fine-grained availability and ways to disable them, and have epochs only act as feature bundles that set the defaults. My reasons are:

• It would mean I wouldn't ever have to worry about losing features I care about, since I can always re-activate them, or deactivate the things that took it away.
• It allows me to switch to a new epoch eagerly, and only disable what is currently still in the way of a full upgrade. Then it's easy to refactor piece-by-piece.
• crates.io can be watched for crates with use-cases requiring adjustments, allowing the features to be refined.
• A large number of deactivations of a feature in crates without a specific use-case for a reason would point to ergonomics issues with the feature.
• Having epochs being just points in time where features are defaulted makes them seem more like waypoints in a moving language than a new language version.
• Constant-availability of features will make it a lot easier to refactor projects to a modern style when they are 10-20 years old, since you can work through changes individually.

[ssokolow]

It would mean I wouldn't ever have to worry about losing features I care about, since I can always re-activate them, or deactivate the things that took it away.

The problem is that taking this approach encourages a Perl-like or C++-like situation where the language is really many different personal dialects and it takes a disproportionate effort to gain the kind broad-spectrum competency that employers look for when adding new members to existing projects.

[phaylon]

If the changes are small and few, that shouldn't become a problem, since it'd still be quite close. I'd certainly assume ecosystem usage will vary a lot more and have a lot more impact on peoples experiences.

Additionally, for the full experience you'll need to know the previous epochs as well, since you'll touch multiple epochs over the course of years, and you might want to be able to read code for crates in older epochs.

[vitiral]

I love this idea. I particularily like that it will allow the language to continue to evolve over time. However, I think the proposal as-is misses the primary spirit of what I want epochs to be, and I hope others feel the same way

Spirit of Epochs

While epochs certainly give us the ability to roll out new syntax, keywords and even features I think the most important feature is it's ability to remove old cruft. As is the current proposal focuses almost exlusively on our ability to keep piling on features, but the minimalist in me really wants epoch's to primarily be for stripping out old features and unifying APIs/syntax/design patterns.

Even since 1.0, the language is evolving at a fairly rapid pace. It is probable that extern crate will be removed, the mod keyword may even eventually be removed in favor of a filesystem layout (I know this was rejected for now, but it's possible in the future). Lots of other changes are happening that improve ergonomics and readability and would cause code written in 1.0 rust to be practically unreadable to someone who learned "rust 2019".

Epoch's give us the ability to actually remove this old cruft, and I consider that their strongest selling point. We can simplify the language for newbies so that we no longer have to teach things like this. From a learning persepctive, removing things is (almost) always more valuable than adding things.

I would like this spirit to be better expressed in the RFC. At the very least, the first line item under the header "The basic idea" should be added that states:

• Deprecated APIs and syntax will now generate hard errors, making it easier to teach and read code in the new epoch

[ssokolow]

the mod keyword may even eventually be removed in favor of a filesystem layout (I know this was rejected for now, but it's possible in the future)

Hopefully not unless it's replaced by some other keyword which allows the effects of an omitted pub to take effect within the same file. The unfriendliness to non-IDEs is one of the biggest reasons I hate Java's decision to force "err on the side of too many too-tiny files" filesystem layouts based on a metric as simple as "one class per file".

Rust mods can also be ridiculously tiny when their only purpose is to provide visibility isolation for one or more private members.

[SimonSapin]

Epoch's give us the ability to actually remove this old cruft

No. The compiler still needs to support previous epochs, so stuff can only be disabled, in new epochs, not actually removed. There is no benefit to removing stuff just for the sake of removing.

Deprecated APIs and syntax will now generate hard errors

This suggests that all deprecated things should be systematically removed. I strongly disagree. Breakage should only be considered when it can make way for other benefits, like making catch a keyword to use it in a new language construct.

[vitiral]

No. The compiler still needs to support previous epochs ...

Sorry, I should have been more clear: I meant it can be removed from our heads -- which is arguably even more important :)

This suggests that all deprecated things should be systematically removed. I strongly disagree. Breakage should only be considered when it can make way for other benefits, like making catch a keyword to use it in a new language construct.

I didn't say all things should be removed (and I don't think that should necessarily be the case... although I can't really think of a counter-example), but I strongly disagree with your disagreement. Removing things that are deprecated is a benefit in itself, and a very strong one. It makes the langauge more uniform and easier to learn for newbies.

[vitiral]

@ssokolow I don't want to get into bikeshedding about removing mod -- I meant it only as an example.

[ssokolow]

@vitiral Likewise. I just wanted the point raised somewhere since I was unaware the discussion you mentioned even happened and it's possible I might miss it if it comes around again.

[aturon]

After a truly epic (epoch?) -- and remarkably civil and deep -- discussion, this RFC has been merged!

It's likely that we'll want to tweak aspects of the policy for changes outlined in this RFC over time (as some of the recent comments reflect), but this RFC gives a reasonable starting point that stakeholders have found acceptable.

Thanks, everyone, for helping make this RFC what it is!

Tracking issue.

[sgrif]

Documented for posterity

[toothbrush7777777]

@aturon The link to the rendered text of the RFC returns 404 Page Not Found. 😢 It needs to be changed to https://github.com/rust-lang/rfcs/blob/master/text/2052-epochs.md.

[carols10cents]

@toothbrush7777777 thanks! Fixed!

[sanmai-NL]

@carols10cents: Could you also add the tracking issue to the RFC please?

[carols10cents]

@sanmai-NL done!