Target Version

[llogiq]

This is basically a writeup of the discussion on internals. I hope I have faithfully captured the spirit of our discussion.

Edit: The above statement is no longer true. I have rewritten the RFC extensively following input from various sources.

Rendered

[steveklabnik]

Cargo should insert the current Rust version

How should cargo do this? How does it know how to access different Rust versions?

[llogiq]

I have extended the paragraph to clarify. Basically either std defines a symbol with the current version, or rustc -V could be used after some post-processing.

[steveklabnik]

You might want to word-wrap this, it's hard to comment on the exact part you want to discuss when each paragraph is a line :)

[llogiq]

Thanks for the input, @steveklabnik !

[ahmedcharles]

Just adding the thoughts I had from the thread on discuss:

I haven't read all of the comments so far, but I'm curious if using a different mechanism for avoiding warnings would be useful. The idea would be to have a targeted rust version specified for a program and have each deprecation list a rust version that it applies to and the warning would only be shown when the targeted rust version is less than or equal to the version that the deprecation applies to.

That would also allow the reverse, which is warning when using features that are newer than the desired version, though I understand that's an entirely different consideration.

[llogiq]

@ahmedcharles this is already done using #[unstable(...)] and has little to do with deprecation. Thanks for mentioning it, though – it gave me an extension idea we could use for un-deprecating an API by allowing since and removed_at to specify a version range (or even multiple ranges). The warnings could then reflect this (like "you use the foo(..) API, which is deprecated in the rust version1.1.0you specified. However, the API was undeprecated in version 1.3.0.)

However, retroactively removing the deprecation altogether is probably the saner alternative here. It also wouldn't require new syntax. Thus, unless I get some votes in favor, I'm not gonna add it.

[internetionals]

Perhaps there should be a difference between deprecation and (planned) feature removal. You want to warn new users of using deprecated functionality (because of some reason), but supporting the old function is not worth the cost of breaking compatibility.

But at some point some function might have a bigger issue than just being "wrong", "unclear" or "inconvenient". It might have security, performance or be in the way for necessary API changes. In that case a deprecation should also feature some kind of "removal" version.

Usage of a deprecated function in a crate that has a target version before that deprecation and no removal clause: No warning, all is fine.
Usage of a deprecated function in a crate that has a target version at/after that deprecation and no removal clause: Warn when it's the root crate being build, otherwise ignore
Usage of a deprecated function that has a removal, but that removal is not met: Warn when it's in a dependency crate, fail if it's in the root crate
Function should effectively be unavailable when it's removal version is reached (ideally featuring the deprecation information as a pointer if no new function with that name is declared).

[llogiq]

Perhaps there should be a difference between deprecation and (planned) feature removal.

Deprecation actually is planned future{{*}} feature removal (for some version) to allow evolving the API.

Note that this has nothing to do with removing features which are deemed inherently insecure – the RFC explicitly states that these should be removed completely (even retroactively), if it is deemed that the insecurity caused by a feature is worse than the breakage resulting from removing it. Otherwise normal deprecation (perhaps with no forward warning) will be the chosen path.

It might have security, performance or be in the way for necessary API changes.

I also don't think that performance problems are not a valid reason for removing an API as long as deprecating it is a viable choice. Being "in the way" should be carefully evaluated – in most cases, deprecation should be sufficient to evolve the API. Note that this RFC avoids talking about language changes, which need to be handled within rustc anyway.

Usage of a deprecated function that has a removal, but that removal is not met: Warn when it's in a dependency crate, fail if it's in the root crate

We have to assume that the user has no control over their build's dependencies. We can however assume that the dependencies' authors mostly follow the rules and specify their target version. Thus any error that will result from deprecation can easily be traced back to the dependency – whose author have by definition seen the same error. As cargo can hold back dependency versions, newer target versions cannot cause problems (however the machinery to effect this may be non-trivial).

{{*}} for some value of future – it may well be never if we never get around to it.

[internetionals]

Deprecation actually is planned future{{*}} feature removal (for some version) to allow evolving the API.

No it's an indication that an API should not be used any longer, like you say:

{{*}} for some value of future – it may well be never if we never get around to it.

As long as it's not actually going to get removed, it's no problem that something is using it, apart from notifying (direct) users of it's "unwanted" usage.

As long as an item is just "bad practice" and supporting it is no problem (or removal would be prohibitively problematic for the foreseeable future) usage of it would be acceptable. Usage these deprecated items by dependencies should hardly concern the developer. Like you say:

We have to assume that the user has no control over their build's dependencies. We can however assume that the dependencies' authors mostly follow the rules and specify their target version.

A developer shouldn't have to care if one of it's dependencies uses deprecated functionality that will be supported for the forseeable future. But if removal has been scheduled, there he should at least know that one of his depencies will be broken in the (near) future and that they should probably request upstream attention or prepare to move away from a possibly unmaintained dependency.

The alternative would be to not give any warning at all and all of a sudden, after a rust upgrade, his dependencies would stop building. At that point he can't upgrade dependency A because it needs a newer rust version and he can't upgrade to a new rust version for that would break dependency B.

I also don't think that performance problems are not a valid reason for removing an API as long as deprecating it is a viable choice.

This was more intended for compiler intrinsics where the compile time optimizations would be impossible as long as a certain feature was still being used. Not performance optimizations because of an old inefficient API in some crate, though developers of said crate might want something for such a use-case too.

It's about giving an option to explicitly sunset a feature in a responsible way (not making assumptions that all (indirect) users of a feature have CI enviroments that track beta and nightly rust releases for all their stuff).

[llogiq]

A developer shouldn't have to care if one of it's dependencies uses deprecated functionality that will be supported for the forseeable future.

That's why I wrote the RFC – we should not (retroactively) remove features unless absolutely necessary, or supporting it becomes too costly. Note that this even rules out future performance benefits, if those break backwards compatibility irreparably (however, it may well be that the performance benefit can be introduced on a per-crate basis for all crates that target a newer version).

Of course, sunsetting a feature once it is determined that it has no viable use anymore is still an option – though not one explored within this RFC.

The alternative would be to not give any warning at all and all of a sudden, after a rust upgrade, his dependencies would stop building. At that point he can't upgrade dependency A because it needs a newer rust version and he can't upgrade to a new rust version for that would break dependency B.

No. The alternative is that newer rust versions will continue to compile code that targets older rust versions. Though this RFC has steered clear of handling the language itself, the scheme it proposes could also be used to evolve the language itself in the face of backwards-breaking changes, as long as the old target versions can continue to compile using the old semantics and post-change code can interoperate with pre-change code. Also note the section on cargo being able to hold back dependency version, so library writers can update their target versions to introduce new features or get new performance benefits.

[llogiq]

Is there any interest in extending this RFC to language/compiler changes?

[comex]

Oi, there seems to be a lot of overlap among RFCs and internals discussions when it comes to this issue.

For the record, in this comment on RFC PR 1105, I discussed how the idea of using a version number attribute to provide full backwards compatibility could interact with language features such as trait selection, glob imports, and some trickier cases.

I personally support a similar policy for language changes, which is also covered by RFC PR 1122.

[alexcrichton]

I feel that this RFC may still be a little premature from the discussion on the internals thread as I'm not quite sure that there's been a consensus reached. I think the discussion has gotten somewhat sidetrack from the initial intention of talking about our existing thoughts on a deprecation strategy and how it might play out, unfortunately. There have been a number times that a language version marker has been discussed (either via an attribute, compiler flag, Cargo.toml field, etc), but there's always been one blocker or another, and I'm not sure that we've quite reached the point where we want to definitively say that it needs to be added.

Specifically on the topic of this RFC, though, I would personally like to see the motivation section spelled out a little more. The standard library currently has a deprecation strategy for APIs (via #[deprecated]), and I don't believe we have any intentions of outright removing APIs any time soon (that was not part of the model of stability attributes). One of the main motivations for the discuss thread was to think about the implications of simultaneously supporting multiple Rust versions in terms of UI with the compiler itself (e.g. you may be forcing yourself to see deprecation warnings on newer compilers). This topic isn't spelled out explicitly in the motivation, but it seems to be alluded to in the detailed design?

In terms of the "drawbacks" section, I feel it doesn't quite do a "require attribute in all Rust code" topic justice. I feel like it's pretty weighty decision to have Cargo warn on all invocations if a version attribute isn't specified. This seems somewhat hostile to newcomers and isn't always a problem that needs to be dealt with. Another drawback I believe needs to be mentioned is that the cross-platform story of deprecation warnings and feature detection is not that great. The fundamental drawback is that the compiler and/or Cargo can only know about one profile the code is being built with, and that profile may be slightly different (e.g. different sets of code) on different platforms). This can lead code to inadvertently believe it works on Rust 1.0 when in fact on Windows it requires Rust 1.1, for example.

[llogiq]

@alexcrichton Thank you for the detailed input and clarification. I'm certainly open to extend the RFC, though I stand by my stance on the internals discussion that we need some versioning (akin to PR #1122) for the standard APIs, too, in order to reduce the API surface and improve discoverability while keeping things backwards compatible.

I can understand that you may feel this RFC be premature, however, considering that I have determined that a per-crate attribute (I have to flesh this out a bit more, but #![target(std="1.1.0")] seems like a good starting point) is the way to go (as I don't want to require rustc read the Cargo.toml), I presume that communicating such a requirement early would give library authors more time to get accustomed to the versioning regime before we actually implement it.

I also have to re-check, but the current version of the RFC would keep warnings at a manageable level, unless a developer updates their target versions, in which case warnings regarding deprecation is to be expected – perhaps such warnings could be grouped by error, in order to reduce compiler output).

I have not yet included thoughts about different operating system profiles; I agree that this is a shortcoming of the RFC in its current form and I'm willing to address this.

[comex]

Creating a Cargo.toml file already involves specifying several metadata fields, and most .rs files at least contain a long list of boilerplate use statements, potentially in addition to metadata attributes and the like; I don't think adding one more attribute in one location or the other, required only for full-fledged Cargo crates (as opposed to throwaway standalone .rs files) is a big deal. And it is basically the only way for the compiler to provide full backwards compatibility, as opposed to having at least some probability of breakage, which is much more burdensome than the one attribute.

If the compiler warns about items a crate imports from the "future" (compared to its declared Rust version), then I don't think there is any new OS hazard; it just becomes a special case of a crate that doesn't compile on X OS. (This also provides an easier way for a crate to ensure compatibility with older versions than just keeping around an old compiler.)

[alexcrichton]

@llogiq

I have to flesh this out a bit more, but #![target(std="1.1.0")] seems like a good starting point

This is one of the fundamental aspects here to consider, though, I believe. First, there's the tradeoff between named features and version numbers. Taking the route of version numbers can lead to problems. For example if I declare I'm targeting 1.1, but I'm compiling with 1.2, then I still want to get deprecation warnings for functions which have security issues, for example (in favor of a crates.io crate probably). I didn't actually want to turn off all future deprecations, just those that I'm actually legitimately using.

If we instead take the route of named features in some form then you have the problem of a growing set of features over time, plus sometimes the header of a crate can be a little like spaghetti. The benefit here, however, is more granular control while in theory being compatible with other implementations (perhaps).

The RFC doesn't make much mention of this alternative, and there's definitely quite a design space here to explore I believe.

I also have to re-check, but the current version of the RFC would keep warnings at a manageable level

I think this is one of the points where I think a clarified motivation will help a bit. For example if I want to keep warnings manageable it's unclear why we should go with an entire versioning scheme for the libraries when #![allow(deprecated)] does the trick (with a few downsides, but a much simpler design).

[llogiq]

Thanks again! I'm fleshing out the motivation, but will take a few days to make it good.

[llogiq]

I have tried to split the Motivations section in two parts, where the first just lists the constraints in short form and the second expands on that list to add explanation. I still don't think I have fully captured all of @alexcrichton 's concerns, but it's a start.

[alexcrichton]

I definitely agree that we want to evolve the APIs that libstd provides, but I'm not quite in agreement that we want to remove deprecated APIs. I think that the overhead of keeping around unstable APIs is unlikely to become too burdensome, but this is of course tough to predict. I also think that it's possible to leave these APIs around without confusing new users through various mitigation tactics like compiler error messages, documentation, etc.

[alexcrichton]

I'm also somewhat skeptical that we'd outright remove an API because it's unsafe or insecure, for example gets is still in the C standard library, it's just plastered with endless warnings to not use it.

[sfackler]

gets was actually removed from the standard in C11, though I think most stdlibs still ship it: http://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Changes_from_C99.

[bstrie]

rustc already has a --target flag, used for a completely separate purpose (selecting target triples).

[llogiq]

Ouch. @bstrie: Care for some bikeshedding?

• --std (from C, but people could confuse it w/ nostd)
• --semver (too abbreviated)
• --source (from Java)
• --lang-version
• --target-version

What do you prefer?

[daboross]

Either this or what? Just wondering, from the wording this seems a bit ambiguous.

[llogiq]

Yeah, it should originally read 'or 1.0.0', because that would keep code without a rust dependency from breaking. However, that may also introduce spurious warnings/errors, so I removed it.

[llogiq]

I have an open discussion on internals for bikeshedding the names for the cargo attribute and the rustc option. Apart from that, it feels pretty much done.

[alexcrichton]

@llogiq I feel like I'm still somewhat confused by the motivation of this RFC. The three sections you have under motivation talk about Language/std evolution, user experience, and security considerations. When talking about evolution, I don't quite follow how this motivates --target and a rust version in Cargo.toml. This sounds a lot like I'm telling the compiler to "behave as if it's version X", but this is basically impossible to do so in a reliable manner. We might be able to imitate another version in terms of the API the standard library exports, but even that has known holes (such as trait implementations) which cannot be handled.

The motivation spelled out in the user experience section also talks about wanting to outright remove an API, but this isn't something I think we need to start worrying about at this time. There are many strategies for dealing with deprecated surface area of the standard library, and I don't think that we feel any need to start removing any of it.

Finally, in terms of "insecure APIs", this is quite a bit of machinery to be added just to get a better error message on using an insecure API, so I'm not sure it's entirely motivated to do so.

---

Overall my main hesitation on this RFC is that I think I still don't quite understand the motivation for all the machinery being added. My main points are:

• We already have a method to evolve the language and standard library through deprecation.
• It's impossible to have the compiler "behave exactly as if it's version X"
• It's unclear whether we actually want to remove deprecated APIs before 2.0

[comex]

This sounds a lot like I'm telling the compiler to "behave as if it's version X", but this is basically impossible to do so in a reliable manner. We might be able to imitate another version in terms of the API the standard library exports, but even that has known holes (such as trait implementations) which cannot be handled.

Why do you say it's impossible, and what is the issue with trait implementations? I'd say it should be easy, given some initial work, in most cases, where the compiler would just have to hide public items and trait implementations from "the future"; harder to keep compatibility in the face of minor breaking changes, but doable, and those should be avoided whenever possible anyway.

[alexcrichton]

Why do you say it's impossible, and what is the issue with trait implementations?

Right now the compiler has no ability to look back in the linting phase to determine whether a trait implementation was used or not. Pruning them out earlier from the AST may be a bit of an invasive and perhaps surprising change.

[comex]

Doing it in the linting phase is definitely too late: for compatibility, the hidden names should not have the potential to cause conflicts in method resolution.

Invasive... well, you know better than I, of course. I shouldn't have said "easy" - what I meant is that after the initial work is done to allow hiding future names and maybe deal with some edge cases, supporting old compatibility versions doesn't seem likely to require an unreasonable amount of continuing effort. But I think I misunderstood the nature of your "reliability" concern anyway.

[llogiq]

@alexcrichton: Like @comex, I think the availability logic needs to be baked in the name resolution. I agree that this would be pretty invasive, but it certainly won't be surprising, given that the crate requested exactly this behavior.

It's impossible to have the compiler "behave exactly as if it's version X"

I'd like to explore the reasoning for this. Could we at least offer a best-effort approximation of version X? How far from version X would it be?

Apart from that re-evaluating the approach it seems something is missing: Because we don't currently have a #[since="1.3.0"] attribute, we cannot remove newer APIs from older code. As the names could clash with user-defined ones, there is a distinct problem for forward-compatibility as well.

Finally, in terms of "insecure APIs", this is quite a bit of machinery to be added just to get a better error message on using an insecure API, so I'm not sure it's entirely motivated to do so.

Ok then. I'm actually fine with removing this part of the RFC.

[alexcrichton]

Could we at least offer a best-effort approximation of version X?

Yeah we could definitely get close to a different version, but unless we get all the way I don't think it's worth having a flag making it look like we're doing so. Otherwise there will be a never-ending stream of bug reports about how the 1.4 compiler doesn't behave exactly like the 1.2 compiler.

And example of how I think this is hard to do would be this PR which is just a minor bugfix to resolve. In general little bug fixes here and there will make it impossible for later compilers to behave perfectly like a previous compiler.

[llogiq]

Regarding your PR example, I believe this is strictly an extension of current behavior, i.e. if we backported it, the only things that would break are tests actively checking for the current (wrong or insufficient?) behavior. Your "never-ending stream of bug reports" would be empty in this particular case. Anyhow, I have added this conundrum to open questions (will push later today).

Yeah we could definitely get close to a different version, but unless we get all the way I don't think it's worth having a flag making it look like we're doing so.

I think there is a finer distinction to make – there are changes, e.g. bugfixes that are strictly backwards compatible. Perhaps we could even define LTS versions that get those fixes in a later release (e.g. 1.0.2 etc.).

Frankly I'm not sure how (or even if) to proceed. On one hand I believe that backwards compatibility will be vital in avoiding fragmentation in the Rust ecosystem and furthermore I believe that using a target version scheme as defined in this PR is the only way to reliably do this without imposing a huge cost on library implementers.

On the other hand I understand that implementing it correctly will have a big cost on each implemented breaking change. We will also need to define a policy to distinguish a) actual bugs that no one should rely on and b) accepted behavior that should be preserved for the given target version.

[alexcrichton]

Frankly I'm not sure how (or even if) to proceed. On one hand I believe that backwards compatibility will be vital in avoiding fragmentation in the Rust ecosystem

I totally agree with this, and I certainly want crates to be able to work on any number of Rust versions they want to work on!

I believe that using a target version scheme as defined in this PR is the only way to reliably do this without imposing a huge cost on library implementers.

I would also consider "test against the desired Rust versions" to be an acceptable threshold for reliably maintaining backwards compatibility. For example Travis makes it super easy to test against multiple Rust versions, and most crates do.

On the other hand I understand that implementing it correctly will have a big cost on each implemented breaking change

The key idea here for me is that maintaining a target scheme where a compiler can masquerade as a previous version doesn't just affect breaking changes. Any change to the compiler needs to be "backported" and/or only happy in a mode where the compiler knows it's not masquerading as an old version. Unfortunately I feel like the burden in doing this is too high to pay off.

[nagisa]

The way I see this proposal, the proposed arguments and attributes currently only can be implemented to make sure rustc is exactly the same version as specified by the attribute/argument; otherwise compilation unconditionally fails (which might be a nice convenience in build scripts?). If you want to use Rust, as it was implemented by an exact rustc version, why not use that exact version of rustc to compile your programs instead?

To make something closer to clang’s/gcc’s --std=c* happen, we need at least:

• A published standard specifying everything from syntax to stable stdlib functions, which I don’t think we are ready for yet;
• Stable ABI.

We have neither and both are pretty big undertakings.

[comex]

If you want to use Rust, as it was implemented by an exact rustc version, why not use that exact version of rustc to compile your programs instead?

You can do that - indeed, large companies that vendor everything will do that no matter what - but that prevents you from using any features or bugfixes from newer versions of Rust, or libraries that depend on them, in your entire program. It just defers the problem.

Any change to the compiler needs to be "backported" and/or only happy in a mode where the compiler knows it's not masquerading as an old version.

It would definitely be nice if rustc could (mostly) ensure that code it accepts with target version X is accepted by rustc version X, but it's not necessary to reap much of the benefit here - for mitigating upgrade pain, the reverse is sufficient, so backwards compatible changes need not pay attention to the target version.

[llogiq]

@alexcrichton

I would also consider "test against the desired Rust versions" to be an acceptable threshold for reliably maintaining backwards compatibility. For example Travis makes it super easy to test against multiple Rust versions, and most crates do.

I have one problem with this: The original maintainer of the library may no longer be around. But perhaps @nagisa is right and the best way to implement it (for those who may need it) is to use a MultiRust that can compile different crates, then link them together.

This would make the whole rustc machinery moot (because we'd simply have different rustcs), and remove much of the cost arising from the change. However, it would very probably fail at link time, unless we require that implementations of any std items stay the same between versions, which we expressly do not want.

[alexcrichton]

@comex

It would definitely be nice if rustc could (mostly) ensure that code it accepts with target version X is accepted by rustc version X, but it's not necessary to reap much of the benefit here - for mitigating upgrade pain

The problem I'd expect with this is that if 1.4 cannot perfectly emulate 1.2 then we're still in a situation where there may be upgrade pain because large codebases are likely to exploit the corner cases the compiler isn't emulating. Unless we get to 100% of being able to emulate a previous version I'm not sure that it's worth it to do the masquerade because it's unlikely to solve the problem for large codebases in the extreme.

---

@llogiq

I have one problem with this: The original maintainer of the library may no longer be around.

Yeah this is a good point, but unfortunately if the compiler doesn't have perfect emulation then if a tweak is needed to make a library compile it may fall in the category of "what the compiler doesn't emulate" so the flag proposed by this RFC wouldn't help in that case.

Overall, I still feel like the best strategy forward is not breaking things :)

[llogiq]

@alexcrichton

Overall, I still feel like the best strategy forward is not breaking things :)

Alas, we have already strayed from that path.

[llogiq]

Since this PR isn't going anywhere before 2.0 I'm closing this. I will however start a new pre-RFC to allow deprecate annotations outside of lang crates; this would be really helpful for library writers.

[golddranks]

The rendered link is broken. Here's a working one: Rendered