Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add guidance about bumping the minimum required `rustc` version #123

Open
danburkert opened this issue Sep 9, 2017 · 45 comments

Comments

Projects
None yet
@danburkert
Copy link

commented Sep 9, 2017

As far as I know, this repository does not offer guidelines on when it is appropriate to bump a library crate's minimum required rustc version. I've found that authors of widely used and foundational crates do not consider bumping the minimum rustc version to be a SemVer incompatible or breaking change (1, 2). I think it would be appropriate to document what norms the Rust community expects on this issue, so that deviations from the norm can be fixed without having to re-litigate the meaning of SemVer.

My personal opinion on the matter is that bumping the required rustc version of a library is a backwards-incompatible SemVer change, since it requires applications depending on the library to follow suit. This change can be every bit as painful for application authors as introducing a breaking API change (or more). The minimum rustc version is 'viral', and a breaking change by a foundational crates forces all other downstream crates to in turn bump their minimum version. Cargo has good tools for introducing new features which may require a newer rustc version, while it has relatively poor support for resolving rustc version dependencies if the author of the library does not follow semantic versioning. Thus, unless Cargo's version resolution begins taking into account required rustc versions, it's better for library authors to put new code requiring new rustc versions behind a feature flag.

@KodrAus

This comment has been minimized.

Copy link
Collaborator

commented Sep 10, 2017

I think it's fair and simple to assume any change to the minimum rustc version is a breaking change.

What's less clear is when that happens. If you happen to have a minimum version of 1.8.0 for your library then bundle in a few ?s with other internal refactorings then you've just bumped your minimum version, possibly without even realising it.

So I think a good recommendation is to pin a rustc version in CI, serde tracks 1.13.0, when you want to stabilise your library and any changes that break your build on that pinned version need an appropriately breaking version bump, or gymnastics with flags.

@dtolnay

This comment has been minimized.

Copy link
Member

commented Sep 10, 2017

I am less of a hardliner on this, although I hear many people think I am wrong.

Serde tracks a Rust version in CI so that we don't accidentally raise the minimum, not because we never will.

Users should not have an expectation that they can selectively pin parts of their dependency graph while receiving updates to things that depend on those parts. In Serde's case, the serde and serde_derive crates are versioned in lockstep. Serde 1.0.5 and serde_derive 1.0.5 work together, and serde 1.0.10 and serde_derive 1.0.10 work together. There shouldn't be an expectation that you can pin serde to 1.0.5 and get to use serde_derive 1.0.10. Does that constitute a "breaking change" to serde_derive? Sure, technically.

Dependencies on the standard library or language should work analogously.

@sfackler

This comment has been minimized.

Copy link
Member

commented Sep 13, 2017

I'm not sure I agree that this needs to be treated as a breaking change. On one hand, you have the cost of forcing people to upgrade their rustc when cargo updateing. On the other, you have the cost of forcing the ecosystem to propagate new major versions throughout it.

Judging by the recent Rust community survey survey responses, ~95% of respondents were targeting 2 stable releases back or newer.

I am looking forward to picking up the bitflags crate's new use of associated consts. The various flags are way more discoverable attached to the relevant type rather than floating around the containing module. I can perform that upgrade in an API-compatible way with deprecated reexports, but if the rustc dependency bump is going to count as a breaking change, I'm just not going to do it for a year or so, probably. The pain of migrating the ecosystem forward across that breaking change is just too painful. rust-openssl 0.9 was released last November and people are still migrating off of the older releases. There are 1-2 issues opened per week against the rust-openssl repo that boil down to "upgrade to 0.9".

@WiSaGaN

This comment has been minimized.

Copy link

commented Sep 19, 2017

Usually I would expect bumping the minimum rustc version should not happen in bugfix release, e.g. 1.0.0 to 1.0.1, but certainly permissible when we have 1.0.0 to 1.1.0 as long as it is advertised as such.

@BurntSushi

This comment has been minimized.

Copy link
Member

commented Nov 7, 2017

I have some thoughts on this, but I think that at a minimum, we should be strongly recommending that folks include a specific version of Rust in their CI builds. This doesn't imply any particular semver policy, but does at least make bumping the minimum Rust version a conscious change. As someone who has tried to play the conservative side of this fence (semver bump for minimum rustc bump), I will say that having a pinned Rust version in a CI config makes the research task of determining the release policy of your dependencies much easier, and typically instantly clarifies the choices I actually have available to me.

I've been trying to straddle this line for a long time now myself, and it is very hard to do so. The hardest part about this problem is that there are significant reasons for choosing either side of this, and I think those reasons have already been highlighted in this thread. Another difficult aspect of this problem is that since there is no consensus on whether a minimum rustc version bump is actually a semver breaking change or not, you end up with an ecosystem where one needs to be aware of the release policy for each dependency of your code, because that policy is not necessarily communicated through version numbers. Even worse, this doesn't just apply to direct dependencies, but to all transitive dependencies too.

I'd like to collect a few of my experiences with this topic.

  • There was actually a proposed RFC on this topic which I think basically says that rust-lang crates should support the previous two stable releases, but otherwise, there's no concrete policy. The RFC contains a discussion with a variety of opinions: rust-lang/rfcs#1619
  • I asked about the release policy of clap here: clap-rs/clap#740 --- @kbknapp was incredibly receptive and opening this issue led to a policy where a minimum rustc bump was considered a minor breaking change, so that dependents could use the ~x.y syntax in their Cargo.toml and be guaranteed that a patch release will never bump the minimum Rust version.
  • Unfortunately, the new release policy of clap led to @joshtriplett opening this bug against ripgrep: BurntSushi/ripgrep#271 --- Basically, the ~x.y syntax prevents ripgrep from compiling with newer versions of clap, even if it could. The interesting point brought up in this thread by @joshtriplett is the questioning of whether ripgrep itself should require a minimum Rust version. The thinking here is that the ability to compiler ripgrep master on a distro's (potentially) ancient version of Rust isn't particular valuable in and of itself, but only if that distro enables packaging Rust applications easily. In that sense, folks should be able to just install ripgrep from their distro and not need to compile it.
  • @alexcrichton also notes issues with ~ dependencies: https://github.com/kbknapp/clap-rs#warning-about--dependencies
  • I did eventually change the clap dependency to remove the ~ dependency: https://github.com/BurntSushi/ripgrep/blob/256aeb55468d57a179447c8b2cd64ada998987c5/Cargo.toml#L35 --- As a result, I almost never run cargo update, but rather, cargo update -p {crate}.
  • Maintaining a minimum Rust version in the face of dependencies without that same policy is a maintenance burden: clap-rs/clap#967 --- and that in turn sometimes has unintended consequences: clap-rs/clap#1003
  • A more circumspect interaction with the encoding_rs crate, where the outcome was this: https://github.com/hsivonen/encoding_rs#rust-version-compatibility --- Even getting specific pinned versions in the CI config was a huge win, but the policy here is, in short, "probably like clap's, but no promises."
  • Regardless of what ideals we all want to believe, users will try and use their distro's version of Rust to compile Rust applications, and when it fails, they will get frustrated: BurntSushi/ripgrep#302

I think that's it as far as my own experiences go. It has been a struggle, but sometimes I wonder whether it is a natural struggle or not. This is why I think making the release policy of a crate discoverable is a first good concrete step we can take that will improve the situation for everyone, and I think it should be uncontroversial. (I think a minimal release policy is putting a pinned Rust version in a CI config.)

My own personal views on this have historically been a little hardline in the sense that I thought it was a semver breaking change. But I've grown increasingly soft on that particular issue. The end result is that I am intensely ambivalent. Therefore, I wind up on the conservative side of things because I'm not sure what to do. That is, all of my crates are pinned to a specific Rust version and I only ever bump the minimum Rust version with a semver version bump.

My conservative stance on this is what has been preventing me from pulling the trigger to release regex 1.0, for example. I really want to use SIMD optimizations in regex, but if I release regex 1.0 now, then under my conservative policy, I wouldn't be able to use SIMD unless I either released regex 2.0 or put the support behind a feature flag that is disabled by default on older Rust versions but enabled otherwise. (At least, I think I could do that.) Either choice is a huge pain and not ideal at all. I suppose we shouldn't necessarily be afraid of making many semver version bumps ("so what if I release regex 2.0"), but every time I do that, some portion of the crate ecosystem is going to build both regex 1 and regex 2 into their crates, which aside from being just plain weird is going to hurt compile times quite a bit!

I have been considering what I think of as the clap compromise for regex though, because I also think it's weird to be holding back from regex 1.0 because of this. In particular:

  1. regex 1.x promises API compatibility.
  2. regex 1.x.y promises Rust compiler compatibility.

Invariably, this results in the same suggestion that clap makes: if you care about Rust compiler compatibility, then use ~1.x in your Cargo.toml, which in turn creates other problems I've highlighted elsewhere. I think my own personal take is that this is a decent compromise, and that I'd prefer the problems associated with the clap compromise over the problems associated with "be conservative" and the problems associated with "move fast and break things." I guess the key problem here is that if too many folks take me up on the Rust compiler compatibility offer, then they'll use ~1.x in their Cargo.toml, which will in turn cause bad things to happen with respect to @alexcrichton's comments mentioned above.

@cuviper

This comment has been minimized.

Copy link

commented Nov 7, 2017

I still wish that a crate's minimum rustc could be expressed to cargo, as well as minimum std and cargo if we don't conflate all of these. Then cargo could just use that for dep-solving, pruning out versions from the registry that are too new for the toolchain in use. That still requires crates to be diligent about declaring their requirement, hopefully with CI to prove it, but it would be better than the status quo.

There have been a few RFCs started along these lines, and I admit I just sat back on the assumption that things were headed on the right track. Can anyone summarize what's happened with that?

@sfackler

This comment has been minimized.

Copy link
Member

commented Nov 7, 2017

@BurntSushi For things like SIMD optimizations that don't affect the public API, there is a third option. You can check the rustc version in a build script and set appropriate CFGs to enable or disable the SIMD codepaths automatically. This does mean that you now have two implementations of a bunch of stuff, but that may already be the case if you're specifically targeting AVX or whatever.

@joshtriplett

This comment has been minimized.

Copy link
Member

commented Nov 7, 2017

@cuviper Rough summary: any kind of version specification depends on new features in Cargo and Cargo.toml, so these efforts blocked on things like Cargo unstable features and versioning of the Cargo schema itself. I think rust-lang/rfcs#2182 is the next step there, and once we have that we can introduce a means of handling rustc versions.

@BurntSushi

This comment has been minimized.

Copy link
Member

commented Nov 7, 2017

For things like SIMD optimizations that don't affect the public API, there is a third option. You can check the rustc version in a build script and set appropriate CFGs to enable or disable the SIMD codepaths automatically. This does mean that you now have two implementations of a bunch of stuff, but that may already be the case if you're specifically targeting AVX or whatever.

Yeah that's what I meant by adding a feature. It's definitely a pain but yeah that might be what I end up doing.

@Ixrec

This comment has been minimized.

Copy link

commented Nov 7, 2017

It does seem like a proper solution to this isn't possible without help from cargo, but it also seems like we can all agree that bumping the minimum rustc version should at least be a minor version bump, and I doubt that's going to change even when cargo has a great, stable solution for this. So for the purposes of the API guidelines, it seems safe to recommend that everyone:

  • explicitly test a specific rustc version in their CI (on top of stable/beta/nightly)
  • make that version easy to find in your documentation
  • never bump the minimum rustc version in a patch release

Anything beyond that is probably overkill for a typical crate, but this is enough to make upgrading to a more elaborate policy easy if your crate ever becomes atypically popular, and it's enough to avoid putting an undue research burden on anyone that does need to support specific rustc versions.


Actually, I just noticed the current guidelines don't talk about CI or versioning at all. Are those considered out of scope?

@kbknapp

This comment has been minimized.

Copy link

commented Nov 7, 2017

I would personally like it if more crates were willing to release >1.0. In my opinion many crates are afraid to release 1.0 for fear of never being able to go beyond. Much of what is being discussed here could be alleviated if crates were more willing to bump the major.

I believe a major hesitation to many popular crates releasing a 1.0 is the fear of "getting something wrong" or not being able to take advantage of rustc advances by not being able to go beyond 1.0 as similarly described by @BurntSushi

I'm hoping as the "core" Rust crates start going beyond 1.0, it will become more acceptable, but don't see it happening without the "core" or very popular libraries doing it first as they generally set the tone and guidelines. Personally, I also think the, "Never a Rust 2.0" is partially contributing to this phenomenon.

This is by no means a call for crates to quickly release a 1.0 followed immediately by a 2.0. But I do wish more crates were willing to bump the major as issues like what's being discussed evolve naturally. I believe if we had more willingness to bump the major, this would probably a be a non-issue.

I'm also just as guilty, as even though clap went 2.0 quite quickly, I've been extremely hesitant to go 3.0 for fear of getting "too far ahead of the norm, and therefore something is wrong" which is admittedly a little silly. As mentioned above, I don't think the ~x.y should be the defacto solution, and perhaps create more problems than they solve. As @BurntSushi said, there are very valid reasons for both sides of the argument.

For now though, I see the above three bullet points mentioned by @Ixrec as being a better policy than none at all.

@est31

This comment has been minimized.

Copy link

commented Nov 23, 2017

A bump of the required rust version should be treated as a a breaking change of your library in the sense of semver. It breaks the cargo update workflow for anyone who is not always on latest Rust stable, and this is not good.

Is it enough if you bump the minor version? No! That means that anybody who has caret requirements anywhere inside their dependency chain might get their cargo update workflow broken, and caret requirements are definitely too popular inside the rust community.

If one day we might be getting a "minimum rust version required" field in Cargo.toml, and cargo update gets changed to not update a dependency if the newer version has a too new required rust version, I think that requirement can be loosened.

@dtolnay

This comment has been minimized.

Copy link
Member

commented Nov 23, 2017

@est31 I strongly disagree with any statements that a bump of required Rust version must always be released as a breaking change. Whatever advice we end up giving here will be primarily a discussion around evaluating tradeoffs, not a hard line.

If I have a library that miraculously compiles on Rust 1.3.0 and is used by thousands of downstream crates and I discover a performance problem that would be fixed by using Box::from_raw, my options are:

  1. Live with the poor performance for the rest of forever,
  2. Bump the minimum Rust version to 1.4.0 in a breaking change, forcing months of ongoing breakage as a thousand libraries upgrade,
  3. Write unsafe code to do it myself (a future version of Rust would be free to break my unsafe implementation and cause memory unsafety),
  4. Freely use Box::from_raw in a minor version. Zero people in the world still care about 1.3.0.

It is pretty clear to me which of these I would do and lose no sleep over.

@est31

This comment has been minimized.

Copy link

commented Nov 23, 2017

@dtolnay This is an argument of the form "if I remove this public function nothing bad will happen because who really uses it and doing a breaking change would require people to update".

I really hate how staying up to date on the current crates.io ecosystem is unusable if you have installed your rustc copy from your distro or are pinning your rust compiler for some other reason. I don't say that people should not rely on Rust 1.4.0 but at least don't break existing projects.

If you are on an older compiler you aren't even getting compilation errors like "sorry, but your Rust compiler is too old", the errors are more often of the form "function not found" or "this feature is unstable".

@est31

This comment has been minimized.

Copy link

commented Nov 23, 2017

And libc is literally inside the top two of most directly depended upon crates, so it is an exceptional example.

@dtolnay

This comment has been minimized.

Copy link
Member

commented Nov 23, 2017

It is an argument of the form "evaluate the tradeoffs and take the least bad approach rather than adhering to dogma."

@kbknapp

This comment has been minimized.

Copy link

commented Nov 24, 2017

An issue with bumping the major upon increasing the minimum required Rust that I haven't seen mentioned yet is the affects of signaling "minor breaking changes"

With Rust having a very quick 6 week release schedule, it's conceivable that a library maintainer may want to take advantage of new features every few Rust releases (or at max every release). Speaking from experience, there always seems to be that new hot feature I can't want to be stabilized. If it's advised to bump the major, it's conceivable that libraries could have very rapid major bumps.

Library consumers have no way to know if a bump of a major is a real breaking change (non additive change of API surface, major shift of how to use, etc.) from things like minimum required Rust. I believe this would lead to more uses of version = "*". This system incentives staying on old rustcs, or never releasing 1.0.

There is no way to say, "I'm using the latest stable and don't care about minimum required Rust changes, but I do care about breaking changes." You're effectively forced into the current ~x.y scenario, without the ability to opt out.

I'd be curious to know if that scenario is any different to a package maintainer (Linux, macOS Homebrew, etc.). @joshtriplett do you have any insight for such a scenario and if it'd be any different from the ~x.y issues (e.g. library releases many major bumps and applications/consumers are spread across a large range of majors)?

@joshtriplett

This comment has been minimized.

Copy link
Member

commented Nov 24, 2017

@kbknapp For packages of Rust crates within a distribution, it's helpful to get an indication when the minimum Rust version has changed, and once we have metadata for that then we can track that in package dependencies. However, any given library crate should only get uploaded once the distribution has a sufficiently new rustc and cargo, and once it does the dependency won't matter, so this is mostly a matter of helping to make sure that gets checked before uploading.

@est31

This comment has been minimized.

Copy link

commented Nov 24, 2017

Library consumers have no way to know if a bump of a major is a real breaking change (non additive change of API surface, major shift of how to use, etc.) from things like minimum required Rust.

When updating to a new major version I already now first just update it and try whether the build still works. If it does it is a good indication that nothing major changed in the API.

@kbknapp it seems that your point (as well as my previous point about cargo update breaking if you just allow minor version upgrades) will get moot once we can actually track required version information via the metadata. Because then one doesn't have to require that the required version gets bumped only with a major semver release (as cargo update can take the metadata information into account). So see my proposal only scoped to libraries that do not track the required version.

Btw, as for verifying the required version I think the best idea is to check whether the currently running compiler matches the required version at package time. We already do a compilation here to verify whether the crate compiles. A more sophisticated variant would be to optionally use rustup to possibly download the specified minimum version and then do the build with that version. It can be made really convenient and the same time really accurate (unless you upload to crates.io via CI, or you implement some other mechanism that automatically transports the build status for the minimum required version from CI to the computer where you do the packaging, CI is not really accurate in ensuring that the version specified in the metadata actually can still build the project).

@mgeisler

This comment has been minimized.

Copy link
Contributor

commented Dec 24, 2017

@dtolnay, you wrote:

If I have a library that miraculously compiles on Rust 1.3.0 and is used by thousands of downstream crates and I discover a performance problem that would be fixed by using Box::from_raw, my options are:

and listed options 1 to 4. Would there not be an option 5:

  1. Write conditionally compiled code that takes advantage of the new Box::from_raw function. Enable this code conditionally at compile time based on the version of rustc being used.

With that option, you would be respecting the semantic versioning (and thus keep the expectations of your downstream dependencies) and you would get to make things better/faster/easier for users with newer compilers. This is the same as what @sfackler suggesed doing about the SIMD support.

So to that end, maybe the focus should be on enabling such cases? Make it super easy and low-effort to opt-in to new features in the code. I don't think it's possible today to do conditional compilation with #[cfg] based on the rustc version?

That is, I'm imagining writing something like this:

#[cfg(semver_compat(rustc_version, "< 1.4.0"))]
fn make_me_a_box() -> Box<Stuff> {
    // slow way...
}

#[cfg(semver_compat(rustc_version, ">= 1.4.0"))]
fn make_me_a_box() -> Box<Stuff> {
    Box::from_raw_parts(...) // yay, the fast way
}

That is, make it easy to swap out the implementation of a function

  • without having to declare new Cargo features
  • without having to use a build script

This way the overhead of supporting new APIs requires less ceremony — you pay with some internal conditional logic. Importantly, downstream users won't have to see or deal with with this complexity, they only get the benefit. Their code won't suddenly stop compiling — and if they use a newer compiler, they'll see the code become faster.

@BurntSushi

This comment has been minimized.

Copy link
Member

commented Dec 24, 2017

@mgeisler That would be nice. I believe you can achieve something similar today with build.rs shenanigans. IIRC, there was pushback on adding version specific cfgs (they have been brought up before), but I can't remember the specific reasoning.

@nagisa

This comment has been minimized.

Copy link

commented Dec 24, 2017

Back-referencing my thread describing real-life breakage that was reported to me.

All I’d like to point out is that CI won't help with detecting breakage for crates which aren’t routinely running CI (i.e. maintainership mode, rare commits, etc.).

Other than that it seems to me that some ability to specify rustc versions supported by the crate in Cargo.toml will be necessary. The rustc_version-like cfg could be implemented as a procedural macro thing and doesn't really need explicit compiler support.

@mgeisler

This comment has been minimized.

Copy link
Contributor

commented Dec 24, 2017

@mgeisler That would be nice. I believe you can achieve something similar today with build.rs shenanigans. IIRC, there was pushback on adding version specific cfgs (they have been brought up before), but I can't remember the specific reasoning.

@BurntSushi Yeah, I cannot imagine that it hasn't been suggested before :-) — this kind of inline feature detection is basically what people have been doing for decades with Autoconf for languages like C. The Python ecosystem also makes liberal use of such "seamless degradation" by importing missing libraries for older versions of Python. In my opinion, it works quite well.

I haven't published many Rust libraries, but I've already noticed that my code breaks from time to time, even when I don't make changes. It breaks because of my dependencies which bump the required rustc version. It's not very satisfying to see Travis fail from one run to another with no changes to my code and no changes to the compiler version. A community that prides itself on using semantic versioning should not accept this.

With a Rust release every 6 weeks, having support for the two most recent stable versions means having support for ~3 months. That seems extremely short to me. Compare this with a language like Python where good Python libraries also support at least the 2-3 most recent recent releases — but Python releases are roughly one per year.

@kbknapp

This comment has been minimized.

Copy link

commented Jan 11, 2018

How does everyone think epoch's will play into this guidance? I haven't followed the epoch discussion too closely, so it's a genuine question.

Edit: or does anyone know if the official Rust crates will have guidance on supporting different epochs? (similar to the current stable-2 guidance).

@est31

This comment has been minimized.

Copy link

commented Jan 11, 2018

@kbknapp from the epoch RFC, they are designed to be private choices of the crate. AKA crates of different epochs can use each other. A change in the epoch should have no effect on users in most cases, unless you maybe need to avoid using keywords as identifiers or something... idk. So the way epochs affect this question seems to be solely from the standpoint of any other feature of rustc: if you use that feature and the current minimum rustc version doesn't support it, you have to bump the minimum rustc version.

So epochs don't change anything about the need to track the minimum rustc version in Cargo.toml.

@justinturpin

This comment has been minimized.

Copy link

commented Mar 17, 2018

I have two cents and I'd like to put them in.

I believe that requiring a new minimum rust version should not be considered a breaking change. My reasoning is twofold:

  • I believe that Semver, which Rust and its libraries have decided to go with, is only concerned with public API changes, and that the compiler version does not constitute affecting the public API. It's possible that you could interpret Semver to include compiler versions as the public api, but I believe the spirit is that it's only concerned with the API, ie. the code.
  • Consider if it was a breaking change - major version bumps now mean two very different things to the developer. In one case, an actual API change occurred and they must change their code accordingly, and in the other they simply need to update their Rustc version and should not change any code.

This is a bit of a pain point for Rust because its a young language that's being rapidly iterated on, but I think the dust will settle and it doesn't change what a breaking change is meant to signify. I agree the rustc compiler version requirement should be described somewhere, just not here. Python's setup.py has a spot for minimum required version, I'm sure lots of other package managers do too.

@gnzlbg

This comment has been minimized.

Copy link

commented May 11, 2018

The simplest way to fix that is, for msrv part of CI, to use lockfile even for libraries.

This sounds like the key guideline to me. Is the lockfile distributed as part of cargo publish ?

@matklad

This comment has been minimized.

Copy link

commented May 11, 2018

This sounds like the key guideline to me. Is the lockfile distributed as part of cargo publish ?

The lockfile of a library has no effect on the downstream crates. The guideline was specifically about CI of the library itself, and it would be obsolete when --minimal-versions flag is stable.

At present, we don't include lockfiles in published crates, however there's an unstable feature to do this, as it is useful for binaries to make sure cargo install foo always picks the same dependencies.

@anderejd

This comment has been minimized.

Copy link

commented May 11, 2018

I find the resistance to upgrading ones stable rustc version very interesting since it's very different from my own perspective. Would it help guide this situation if a list of reasons to resist upgrading rustc was compiled?

I have a hard time figuring out why anyone would avoid upgrading when it's just a rustup upgrade away.

@gnzlbg

This comment has been minimized.

Copy link

commented May 11, 2018

@anderejd

Would it help guide this situation if a list of reasons to resist upgrading rustc was compiled?

I have a hard time figuring out why anyone would avoid upgrading when it's just a rustup upgrade away.

TL;DR: upgrading Rust bumps the std library minor version and that is a breaking change that users cannot avoid while upgrading.

The RFC "semver for the std library" says that changes that can be fixed by forcing users to disambiguate are non-breaking, unless too many people complain soon enough (e.g. before it hits stable). That is anything but reassuring, and if you are worried about this, there aren't currently any ways to protect yourself beyond not using any of the libraries shipped by Rust.

For example, if we add a trait method from Itertools to Iterator, every crate using the Itertool method will break in the next Rust upgrade unless they are using UFCS to disambiguate, which is something that nobody does. Fixing this requires releasing a new version of Itertools and upgrading the whole ecosystem to that version, for that particular Rust toolchain at least. [0]

And that's the "best" case in which upgrading Rust means that your code doesn't compile. There are worse cases. For example, adding inherent methods to std library types can silently changes the behavior of Rust programs. [1]

Dependency on crates.io crates have this problem too, but it is really easy to avoid it by either using a Cargo.lock or fixing an exact version of a dependency in the Cargo.toml.


[0]: there are many ways to help with this from the itertools point-of-view, like releasing new fix versions (x.y.fix) of some or all itertools releases that don't have the trait method if the Rust compiler has some particular version, which can be checked with a build script. This will only work for those not specifying a particular fix version of itertools though.

[1]: these aren't the only issues, std changes can also affect type inference, amongst other things, and whether these changes result in compilation failures or silent behavior changes depends on the particular change.

@BurntSushi

This comment has been minimized.

Copy link
Member

commented May 11, 2018

Note that the reasons for keeping tabs on Rust versions aren't just technical. They are also social:

  • Some environments strictly control what version of software that is used. This doesn't in and of itself imply a certain semver strategy, but I think it does at least help to have a coherent story about what version of Rust your code requires. Even being able to know that requires some work.
  • Some folks use distros with older versions of the Rust compiler. If you want to cater to those folks and let them compile your code, then you need to manage the version of Rust that your code requires.
  • In general, as a maintainer, I want to have a clear and unambiguous statement about which version of the Rust compiler my code requires to compile.

The various semver heuristics are one way of managing this, but there are other ways (like making Cargo.toml aware of the Rust version). Others have pointed out potentially serious flaws with the semver approach. So there's no obvious path forward here, but the comment about around minimal versions sounds promising!

@anderejd

This comment has been minimized.

Copy link

commented May 11, 2018

I don't know much about working in environments with strict rules on what software are allowed so I can't really contribute to that part of the discussion.

For crates.io I think it would be doable to run a big batch job of cargo build tests using all rustc versions and tagging each crate + crate version with compatible rustc versions. Maybe it would and should require some reasonable limit on rustc versions to test with, starting with the latest and going back a couple of stable releases.

@newpavlov

This comment has been minimized.

Copy link

commented May 14, 2018

In my opinion bumping the minimum required rustc version currently should be considered a breaking change (arguments already listed in the discussion above), but it should be only a temporary solution and we really should push for rust/rustc field in the Cargo.toml, which will be used by cargo to select an appropriate crate versions. After we will get such capability, I think it will make sense to recommend for post-1.0 crates to bump minor version after changing the minimum rustc version. This will make it easier to publish fixes for serious bugs for crate versions which target older compilers.

@dtolnay

This comment has been minimized.

Copy link
Member

commented May 20, 2018

Here is a crazy/wonderful approach that requires rust-lang/cargo#5499.

Suppose Serde 1.0.x wants to bump the minimum required compiler version from rustc 1.13 to 1.29.

  1. We publish Serde 2.0.0 with an implementation that requires rustc 1.29 (bear with me).
  2. We go back to the code of Serde 1.0.x and add a semver-trick* dependency on serde = { version = "2.0", optional = true }. This is unlike the typical semver trick dependency in that it is an optional dependency! *you need to read this link or else nothing I write will make sense.
  3. Still in Serde 1.0.x, we add a build.rs script which uses version_check to detect when we are being built by rustc >= 1.29. If so, print the magic line that turns on the optional dependency on 2.0.
  4. In lib.rs of Serde 1.0.x we detect #[cfg(feature = "serde")] and if so, do the usual semver trick re-export of all the contents of 2.0.
  5. If #[cfg(not(feature = "serde"))] then we provide everything as usual, everything as it exists in the 1.0.x series with a compiler requirement of 1.13.
  6. Publish as the next release of 1.0.x series.

There are many steps but this is a modest amount of work for a significant library. The end state is:

  • Anyone on old compilers observes no change. Dependencies on serde = "1.0" continue to provide all the same APIs. You cannot have 2.0 in your dependency graph because your compiler does not meet its minimum version requirement.
  • On new compilers you get a semver-trick relationship where serde = "1.0" and serde = "2.0" are synonymous.

People who argue rustc requirement bumps are a breaking change are happy because their serde = "1.0" dependency can never pull in code that does not compile on their compiler version.

People who argue rustc requirement bumps are not a breaking change are happy because they get to use new compiler features without forcing an upgrade treadmill of major versions on their downstream users.

@retep998

This comment has been minimized.

Copy link

commented May 20, 2018

Still in Serde 1.0.x, we add a build.rs script which uses version_check to detect when we are being built by rustc >= 1.29. If so, print the magic line that turns on the optional dependency on 2.0.

This shouldn't work as far as I know because whether a given dependency is enabled or disabled is resolved before build scripts are run.

@dtolnay

This comment has been minimized.

Copy link
Member

commented May 20, 2018

Slight modification that no longer requires rust-lang/cargo#5499: we could have 1.x unconditionally depend on 2.0 and add a build script to 2.0 as well, which sets a feature on old compilers that turns the entire 2.0 crate into an empty crate.

@luser

This comment has been minimized.

Copy link

commented Jul 12, 2018

I don't have strong feelings about the semver question here, but as someone maintaining a decent-sized Rust application I frequently run into situations where I update a dependency or add a new dependency and need to bump the minimum Rust version I support (which I test against in CI), but I find it virtually impossible to figure out what the actual required version is! It's easy to see that a PR breaks compiling against the pinned Rust version in CI, and often the error message will even name a specific feature that's unstable in that version, but there's no good way (AFAIK) to figure out what version stabilized that feature. (In other cases the error is not as clear and it's even harder.)

@BurntSushi

This comment has been minimized.

Copy link
Member

commented Jul 12, 2018

@luser Yeah, I run into the same problem. This is why I've been advocating that everyone put a specific Rust version in their CI config so that others can go and look at it and know what version the project intends to be able to compile on. I think that is an uncontroversial and unambiguously good step forward, and in particular, does not require everyone to agree on the semver ramifications---folks are free to bump the minimum Rust version as much as they want, they just have to do so explicitly.

I think more and more projects are doing this, but unfortunately, many still aren't. At that point, you're right, it's a research task to figure out the minimum Rust version. There's probably a script someone could write that does a binary search for you, but this requires having all intermediate versions of Rust installed. (rustup makes this easy, though, may take up a bit of disk space!)

@dwijnand

This comment has been minimized.

Copy link
Contributor

commented Jul 12, 2018

(See also japaric/trust#103 about running CI with a rust-toolchain file present)

@newpavlov

This comment has been minimized.

Copy link

commented Jul 12, 2018

See rust-lang/rfcs#2495 for a proposal which will allow authors to specify minimum supported Rust version (MSRV), which in combination with additional constraints in dependency versions resolution algorithm resolution algorithm will make changing MSRV backwards compatible change.

@gnzlbg

This comment has been minimized.

Copy link

commented Jul 12, 2018

@luser same here, although I must say that while this is pretty annoying when it happens, it doesn't happen often enough to me to be a continuous annoyance. As in, when it happens, I bisect the version required, and don't have to worry about it again for a couple of months.

@matklad

This comment has been minimized.

Copy link

commented Jul 13, 2018

An interesting idea from @Eh2406 for a CI setup that should work today.

To test MSRV, one can use

cargo +nightly -Z minimal-versions generate-lockfile && cargo +MSRV test

Caveats:

  • your crate probably won't build with minimal-versions right off the bat, see 1 and 2 for the list of problems you may face.

  • -Z minimal-versions itself is unstable.

@alexheretic

This comment has been minimized.

Copy link
Member

commented Dec 21, 2018

I'm using latest stable as MSRV for the crates I maintain. Hopefully better tools will appear to handle this gracefully for both crate maintainers and old compiler users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.