New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release channels take 2 #507

Merged
merged 9 commits into from Jan 2, 2015

Conversation

Projects
None yet
@brson
Contributor

brson commented Dec 8, 2014

This RFC describes changes to the Rust release process, primarily the division of Rust's time-based releases into 'release channels', following the 'release train' model used by e.g. Firefox and Chrome; as well as 'feature staging', which enables the continued development of experimental language features and libraries APIs while providing strong stability guarantees in stable releases.

This revision includes a significant simplification by merging stability attributes with feature gates, which further enables some additional useful features within Cargo related to version detection. One big implication of this change is that stability attributes are no longer suitable for use outside of std (if such a feature is desirable it is intended to be solved by Cargo).

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 9, 2014

Member

cc @wycats, this is a culmination in what feature-related work we were discussing at the work week.

Member

alexcrichton commented Dec 9, 2014

cc @wycats, this is a culmination in what feature-related work we were discussing at the work week.

@steveklabnik

This comment has been minimized.

Show comment
Hide comment
@steveklabnik
Member

steveklabnik commented Dec 9, 2014

@andrew-d

This comment has been minimized.

Show comment
Hide comment
@andrew-d

andrew-d Dec 9, 2014

Just thinking out loud: it would be interesting if we could use a Cargo feature to enable some potentially-unstable APIs in a crate. For example, a Cargo feature "use_experimental" to conditionally-compile in use of some unstable rustc feature, but still allowing stable or beta users to use the crate without that feature. Not sure how practical that is, though.

andrew-d commented Dec 9, 2014

Just thinking out loud: it would be interesting if we could use a Cargo feature to enable some potentially-unstable APIs in a crate. For example, a Cargo feature "use_experimental" to conditionally-compile in use of some unstable rustc feature, but still allowing stable or beta users to use the crate without that feature. Not sure how practical that is, though.

Stable builds are versioned and named the same as today's releases,
both with just a bare version number, e.g. '1.0.0'. They are
published at the beginning of each development cycle and once
published are never refreshed or overwritten. Provisions for stable

This comment has been minimized.

@nrc

nrc Dec 11, 2014

Member

"never" seems very strong here - we should be able to patch if there is a critical security issue, for example

@nrc

nrc Dec 11, 2014

Member

"never" seems very strong here - we should be able to patch if there is a critical security issue, for example

This comment has been minimized.

@tshepang

tshepang Dec 12, 2014

Contributor

@nick29581 Rather the version be removed from crates.io than a different version be made to take the same name. In this case, the version with a critical security issue would be 1.0.1, and 1.0.0 would be unavailable so people don't use it.

@tshepang

tshepang Dec 12, 2014

Contributor

@nick29581 Rather the version be removed from crates.io than a different version be made to take the same name. In this case, the version with a critical security issue would be 1.0.1, and 1.0.0 would be unavailable so people don't use it.

This comment has been minimized.

@alexcrichton

alexcrichton Dec 18, 2014

Member

Yes this means that 1.0.0 will never change, not that we'll never release 1.0.1

@alexcrichton

alexcrichton Dec 18, 2014

Member

Yes this means that 1.0.0 will never change, not that we'll never release 1.0.1

unstable APIs for crates (read below for the impact of feature staging
on unstable APIs).
With staged features disabled, the Rust build itself is not possible,

This comment has been minimized.

@nrc

nrc Dec 11, 2014

Member

What is the motivation here? Do we envisage building rustc with non-nightlies?

@nrc

nrc Dec 11, 2014

Member

What is the motivation here? Do we envisage building rustc with non-nightlies?

This comment has been minimized.

@alexcrichton

alexcrichton Dec 18, 2014

Member

This is just a bootstrapping problem in the sense that if we build a compiler that rejects unstable features then it itself cannot build the compiler (rustc will always be using unstable features).

@alexcrichton

alexcrichton Dec 18, 2014

Member

This is just a bootstrapping problem in the sense that if we build a compiler that rejects unstable features then it itself cannot build the compiler (rustc will always be using unstable features).

language.
To calculate this information, Cargo will compile crates just before
publishing. In this process, the Rust compiler will record all used

This comment has been minimized.

@nrc

nrc Dec 11, 2014

Member

We ought to be able to do better than compiling (although checking a crate compiles has other advantages). We could do a custom pass over the crate just checking for stability attributes and ignoring cfgs, thus solving the conditional compilation problem

@nrc

nrc Dec 11, 2014

Member

We ought to be able to do better than compiling (although checking a crate compiles has other advantages). We could do a custom pass over the crate just checking for stability attributes and ignoring cfgs, thus solving the conditional compilation problem

This comment has been minimized.

@alexcrichton

alexcrichton Dec 18, 2014

Member

We need to perform full resolution (e.g. method resolution) to determine all APIs used by a library, and this ends up amounting to basically a full compilation. It's a good smoke test regardless to compile before publishing!

Due to needing this analysis I don't think we can ignore cfg as if we enabled everything we'd probably get compilation errors due to multiple definitions of the same item in a library.

@alexcrichton

alexcrichton Dec 18, 2014

Member

We need to perform full resolution (e.g. method resolution) to determine all APIs used by a library, and this ends up amounting to basically a full compilation. It's a good smoke test regardless to compile before publishing!

Due to needing this analysis I don't think we can ignore cfg as if we enabled everything we'd probably get compilation errors due to multiple definitions of the same item in a library.

Each crate will have a "badge" indicating what version of the Rust
compiler is needed to compile it. The "badge" may indicate that the
nightly or beta channels must be used if the version required has not
yet been released (this happens when a crate is published on a

This comment has been minimized.

@nrc

nrc Dec 11, 2014

Member

I think this is an over-simplification - a crate might rely on a feature which is gated for more than one version, so even though the minimum version is now on beta (say), the crate still needs nightly for access to the feature gated feature. This doesn't affect the UI, but means the implementation must track features not just versions for calculating the channel. (I guess the permanent nightly badge solves this, but if this is the only reason for having it, it seems overly restrictive).

@nrc

nrc Dec 11, 2014

Member

I think this is an over-simplification - a crate might rely on a feature which is gated for more than one version, so even though the minimum version is now on beta (say), the crate still needs nightly for access to the feature gated feature. This doesn't affect the UI, but means the implementation must track features not just versions for calculating the channel. (I guess the permanent nightly badge solves this, but if this is the only reason for having it, it seems overly restrictive).

This comment has been minimized.

@alexcrichton

alexcrichton Dec 18, 2014

Member

Note that any crate which accesses features is a "permanent nightly" crate. Features change over time and there's no notion of stability or backwards compatibility with enabling a feature gate, so we cannot ever recommend a concrete version as none will known to be compatible.

@alexcrichton

alexcrichton Dec 18, 2014

Member

Note that any crate which accesses features is a "permanent nightly" crate. Features change over time and there's no notion of stability or backwards compatibility with enabling a feature gate, so we cannot ever recommend a concrete version as none will known to be compatible.

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Dec 13, 2014

Thank you for looking into this!

As a library author, I'm going to miss #[unstable]. I was using this to indicate that certain APIs were available for testing purposes, but that I hadn't yet committed to full SemVer guarantees, and that I reserved the right to make changes before declaring something #[stable]. This was a nice escape hatch. Without it, I'm concerned that post-1.0 libraries will be a bit too much like "quick-setting cement"—the instant something appears in an API, I need to live with it for a very long time. It would be nice to say, "Hey, this is available, and if you use it, you'll get a warning that it might change."

In fact, #[unstable] was one of my favorite minor Rust features. :-) Would it be possible to have a compiler-and-std-specific version of #[unstable], and leave the existing attribute for use by libraries? If not, no worries.

emk commented Dec 13, 2014

Thank you for looking into this!

As a library author, I'm going to miss #[unstable]. I was using this to indicate that certain APIs were available for testing purposes, but that I hadn't yet committed to full SemVer guarantees, and that I reserved the right to make changes before declaring something #[stable]. This was a nice escape hatch. Without it, I'm concerned that post-1.0 libraries will be a bit too much like "quick-setting cement"—the instant something appears in an API, I need to live with it for a very long time. It would be nice to say, "Hey, this is available, and if you use it, you'll get a warning that it might change."

In fact, #[unstable] was one of my favorite minor Rust features. :-) Would it be possible to have a compiler-and-std-specific version of #[unstable], and leave the existing attribute for use by libraries? If not, no worries.

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Dec 13, 2014

SemVer for Rust and std?

As the maintainer of heroku-buildpack-rust, I need to know exactly what compiler version should be used to build a git repository. Here are some use cases and a proposal that I posted in rust-lang/cargo#1044:

First, some use cases:

  • ACME, Inc. has a large Rust application hosted on Heroku using a Rust buildpack. The application is a cash-cow: It earns a lot of money, and it has a lot of users, but it's only updated rarely. Six months after the last update, a critical bug is discovered. A two-line patch is produced, and it needs to be pushed to production.
  • Rustalicious LLC is a boutique software consultancy specializing in Rust applications. They help maintain ACME's cash cow application. They're also working on Rust projects for 3 other clients, in varying stages of active development and maintenance.

In these examples, both ACME and Rustalicious need to control when they upgrade their Rust toolchain. ACME is trying to push an urgent fix to production, but they haven't touched the codebase in 4 Rust releases, and they don't have the time to upgrade to the lastest Rust compiler and verify that nothing is broken. They want to make a two-line fix using their old toolchain.

Rustalicious wants to use the latest and greatest Rust compiler for everything, because it's shiny. But they have multiple maintenance mode clients who depend on different versions of the Rust platform. So they potentially need to install multiple versions of Rust side-by-side, and manage them using a specialized tool.

Prior art: Bundler + buildpacks + RVM

In the Ruby world, it's possible to use Gemfile to specify the version of Ruby that should be used with an application:

`ruby '1.9.3'`

This information is read by Heroku's Ruby buildpack, and by various third-party tools like RVM that manage multiple, parallel Ruby implementations.

Proposed solution: Treat rustc and the standard libraries like any other versioned dependency

rustc and the standard library are an actual dependency for every Rust application. So it might be reasonable to specify their version numbers somewhere in Cargo.toml:

rust = "1.1.2"

This could go either in the package section, or possibily even in the dependencies section. It could potentially allow full version specifiers, and possibly even be recorded in Cargo.lock like any other dependency.

What would Cargo do with this information?

Two possibilities come to mind:

  1. Verify that all packages can be build with the specified version of Rust, and if not, give an error message explaining the constraints. This would offer significant value to ordinary Cargo users, because they'd get comprehensible messages telling them when they needed to upgrade their compiler, instead of random error spew from one of the libraries they're using.
  2. Store an exact version in Cargo.lock.

Third-party tools might also read this information to configure deployment servers (e.g., buildpacks), manage parallel Rust toolchains (e.g. RVM etc. for Rust), and decide which Rust version(s) should be used for continuous integration builds.

Alternatives

  • Trust the compiler and standard libs to never have regressions.
  • Store this information in a .rust_version file and coordinate between tool developers.

I'm not hugely attached to the details of this proposal, but I will need to address both the ACME and Rustalicious use cases in the future, with any luck. :-)

If it's useful, I'd be interested in contributing some code, but I'd need code review from the Cargo team.

I would love to be able apply SemVer rules to the "Rust platform" the same way I apply them to any other crate that I manage with Cargo.

As for the idea of automatically detecting the appropriate compiler version to use with a given source tree, my reactions are:

  1. This has a huge number of moving pieces.
  2. It doesn't satisfy any of the "ACME" or "Rustalicious" use cases described above. For example, if you've had an application running unchanged in production for 6 months (which is perfectly normal for some of my consulting clients!), and you need to make a 1-line bug fix, you really don't want your toolchain to play "guess a Rust version" for you. You want to make the 1-line change, recompile with whatever version of Rust you were using originally, and push to Heroku (or whatever).

To be fair, I can just modify heroku-buildpack-rust to use a .rust_version file, if my needs diverge too much from what everybody else needs.

emk commented Dec 13, 2014

SemVer for Rust and std?

As the maintainer of heroku-buildpack-rust, I need to know exactly what compiler version should be used to build a git repository. Here are some use cases and a proposal that I posted in rust-lang/cargo#1044:

First, some use cases:

  • ACME, Inc. has a large Rust application hosted on Heroku using a Rust buildpack. The application is a cash-cow: It earns a lot of money, and it has a lot of users, but it's only updated rarely. Six months after the last update, a critical bug is discovered. A two-line patch is produced, and it needs to be pushed to production.
  • Rustalicious LLC is a boutique software consultancy specializing in Rust applications. They help maintain ACME's cash cow application. They're also working on Rust projects for 3 other clients, in varying stages of active development and maintenance.

In these examples, both ACME and Rustalicious need to control when they upgrade their Rust toolchain. ACME is trying to push an urgent fix to production, but they haven't touched the codebase in 4 Rust releases, and they don't have the time to upgrade to the lastest Rust compiler and verify that nothing is broken. They want to make a two-line fix using their old toolchain.

Rustalicious wants to use the latest and greatest Rust compiler for everything, because it's shiny. But they have multiple maintenance mode clients who depend on different versions of the Rust platform. So they potentially need to install multiple versions of Rust side-by-side, and manage them using a specialized tool.

Prior art: Bundler + buildpacks + RVM

In the Ruby world, it's possible to use Gemfile to specify the version of Ruby that should be used with an application:

`ruby '1.9.3'`

This information is read by Heroku's Ruby buildpack, and by various third-party tools like RVM that manage multiple, parallel Ruby implementations.

Proposed solution: Treat rustc and the standard libraries like any other versioned dependency

rustc and the standard library are an actual dependency for every Rust application. So it might be reasonable to specify their version numbers somewhere in Cargo.toml:

rust = "1.1.2"

This could go either in the package section, or possibily even in the dependencies section. It could potentially allow full version specifiers, and possibly even be recorded in Cargo.lock like any other dependency.

What would Cargo do with this information?

Two possibilities come to mind:

  1. Verify that all packages can be build with the specified version of Rust, and if not, give an error message explaining the constraints. This would offer significant value to ordinary Cargo users, because they'd get comprehensible messages telling them when they needed to upgrade their compiler, instead of random error spew from one of the libraries they're using.
  2. Store an exact version in Cargo.lock.

Third-party tools might also read this information to configure deployment servers (e.g., buildpacks), manage parallel Rust toolchains (e.g. RVM etc. for Rust), and decide which Rust version(s) should be used for continuous integration builds.

Alternatives

  • Trust the compiler and standard libs to never have regressions.
  • Store this information in a .rust_version file and coordinate between tool developers.

I'm not hugely attached to the details of this proposal, but I will need to address both the ACME and Rustalicious use cases in the future, with any luck. :-)

If it's useful, I'd be interested in contributing some code, but I'd need code review from the Cargo team.

I would love to be able apply SemVer rules to the "Rust platform" the same way I apply them to any other crate that I manage with Cargo.

As for the idea of automatically detecting the appropriate compiler version to use with a given source tree, my reactions are:

  1. This has a huge number of moving pieces.
  2. It doesn't satisfy any of the "ACME" or "Rustalicious" use cases described above. For example, if you've had an application running unchanged in production for 6 months (which is perfectly normal for some of my consulting clients!), and you need to make a 1-line bug fix, you really don't want your toolchain to play "guess a Rust version" for you. You want to make the 1-line change, recompile with whatever version of Rust you were using originally, and push to Heroku (or whatever).

To be fair, I can just modify heroku-buildpack-rust to use a .rust_version file, if my needs diverge too much from what everybody else needs.

merged, and like the 'nightly' channel each published build during a
single development cycle retains the same version number, but can be
uniquely identified by the commit number. Beta artifacts are likewise
simply named 'rust-beta'.

This comment has been minimized.

@huonw

huonw Dec 14, 2014

Member

Is this meant to be 1.0.0-beta?

@huonw

huonw Dec 14, 2014

Member

Is this meant to be 1.0.0-beta?

This comment has been minimized.

@brson

brson Dec 30, 2014

Contributor

No. I intend beta and nightly artifacts to not include version numbers. It's mostly just so that old beta artifacts don't stick around forever - the beta artifacts are intended to be transient, unlike betas from a typical (non-channel-based) release process.

@brson

brson Dec 30, 2014

Contributor

No. I intend beta and nightly artifacts to not include version numbers. It's mostly just so that old beta artifacts don't stick around forever - the beta artifacts are intended to be transient, unlike betas from a typical (non-channel-based) release process.

This comment has been minimized.

@huonw

huonw Dec 30, 2014

Member

Oh, I see. ("1.0.0-nightly" is mentioned in the previous paragraph.)

@huonw

huonw Dec 30, 2014

Member

Oh, I see. ("1.0.0-nightly" is mentioned in the previous paragraph.)

This comment has been minimized.

@brson

brson Dec 30, 2014

Contributor

@huon I think maybe it will be better to have the version number in the beta artifacts. Makes it clearer what you are getting, but it does mean that 1.0.0-beta-*.tar.gz will get overwritten with updates during the release cycle. What do you prefer?

@brson

brson Dec 30, 2014

Contributor

@huon I think maybe it will be better to have the version number in the beta artifacts. Makes it clearer what you are getting, but it does mean that 1.0.0-beta-*.tar.gz will get overwritten with updates during the release cycle. What do you prefer?

@nodakai

This comment has been minimized.

Show comment
Hide comment
@nodakai

nodakai Dec 17, 2014

Which "channel" will you recommend to a newcomer?

nodakai commented Dec 17, 2014

Which "channel" will you recommend to a newcomer?

@tshepang

This comment has been minimized.

Show comment
Hide comment
@tshepang

tshepang Dec 17, 2014

Contributor

@nodakai stable

Contributor

tshepang commented Dec 17, 2014

@nodakai stable

Cargo's own support for features (distinct from Rust features) to enable
this form of feature development in a first-class method through Cargo.
At this time, however, there are no concrete plans for this design and
it is unlikely to happen soon.

This comment has been minimized.

@quantheory

quantheory Dec 18, 2014

Contributor

This is rather cold comfort. The stability attributes:

  • Are fairly simple/generic source code annotations.
  • Have benefits that are widely applicable and hard to get by any other method (affecting rustdoc output in a pretty way, rustc warnings).
  • Don't assume that a project's stabilization policies or processes remotely resemble those of the standard library.
  • Don't assume that Cargo is being used for the build system/package manager in the first place.

So this is a useful thing that will be dropped, with a vague hope that something similar (but possibly less generic and more presumptuous) will be designed for Cargo in the hazy future. I'm wondering if it's really necessary to confine the stability attributes to the compiler and standard library (as opposed to adding this system in a way that doesn't affect the existing stability attributes, or at least unstable/stable/deprecated).

@quantheory

quantheory Dec 18, 2014

Contributor

This is rather cold comfort. The stability attributes:

  • Are fairly simple/generic source code annotations.
  • Have benefits that are widely applicable and hard to get by any other method (affecting rustdoc output in a pretty way, rustc warnings).
  • Don't assume that a project's stabilization policies or processes remotely resemble those of the standard library.
  • Don't assume that Cargo is being used for the build system/package manager in the first place.

So this is a useful thing that will be dropped, with a vague hope that something similar (but possibly less generic and more presumptuous) will be designed for Cargo in the hazy future. I'm wondering if it's really necessary to confine the stability attributes to the compiler and standard library (as opposed to adding this system in a way that doesn't affect the existing stability attributes, or at least unstable/stable/deprecated).

This comment has been minimized.

@emk

emk Dec 18, 2014

Yes, the loss of #[unstable] for use by ordinary Rust libraries feels like the deliberate removal of a useful language feature. I don't recall any indications that #[unstable] was originally intended for use only by the standard library.

@emk

emk Dec 18, 2014

Yes, the loss of #[unstable] for use by ordinary Rust libraries feels like the deliberate removal of a useful language feature. I don't recall any indications that #[unstable] was originally intended for use only by the standard library.

This comment has been minimized.

@alexcrichton

alexcrichton Dec 18, 2014

Member

@quantheory it's not clear how the stability attributes can possibly interact with other libraries to us. The major purpose of this is to prevent access entirely to #[unstable] APIs, not to just warn-on-use. With this form of model, how would the compiler allow or disallow access to unstable APIs in arbitrary libraries? The compiler itself has very little knowledge of versioning, especially of libraries themselves, and almost all of the benefits of the stability attributes here are gained through libraries declaring semver-requirement dependencies on one another themselves.

To expand on the design for Cargo, I envision the ability to have unstable features in Cargo manifest (note I mean Cargo features, not stability-attribute features). Packages are then required to opt-in to these features, and packages published to crates.io will not have any experimental features. This is largely what I think external libraries would desired with respect to stability attributes because:

  1. Experimental features can be developed in the upstream source repository. Users of stable versions of a library do not have access to them.
  2. Applications can opt-in to using experimental features by depending on the repository.
  3. All features are tracked by name and there's a clear path to accepting features for libraries themselves (and a clear place to put the list of features).

@emk I'm curious on your thoughts of what I've said above. The stability attributes were never really tasked with being a general purpose mechanism for all libraries and would require significant redesign to work as-is. All of this infrastructure is very specific to the language version which does not apply to external libraries.

Also note that all libraries other than the standard library have a huge advantage which is that they are published through crates.io. Management of multiple versions of a library is super easy, as well as declaring dependencies.

All in all, we have personally been able to come up with a concrete use case where stability attributes provide guarantees for external libraries that Cargo does not already do so. With this in mind, we've architected the current iteration towards more favoring the compiler and language and planning for similar functionality through Cargo externally if necessary

@alexcrichton

alexcrichton Dec 18, 2014

Member

@quantheory it's not clear how the stability attributes can possibly interact with other libraries to us. The major purpose of this is to prevent access entirely to #[unstable] APIs, not to just warn-on-use. With this form of model, how would the compiler allow or disallow access to unstable APIs in arbitrary libraries? The compiler itself has very little knowledge of versioning, especially of libraries themselves, and almost all of the benefits of the stability attributes here are gained through libraries declaring semver-requirement dependencies on one another themselves.

To expand on the design for Cargo, I envision the ability to have unstable features in Cargo manifest (note I mean Cargo features, not stability-attribute features). Packages are then required to opt-in to these features, and packages published to crates.io will not have any experimental features. This is largely what I think external libraries would desired with respect to stability attributes because:

  1. Experimental features can be developed in the upstream source repository. Users of stable versions of a library do not have access to them.
  2. Applications can opt-in to using experimental features by depending on the repository.
  3. All features are tracked by name and there's a clear path to accepting features for libraries themselves (and a clear place to put the list of features).

@emk I'm curious on your thoughts of what I've said above. The stability attributes were never really tasked with being a general purpose mechanism for all libraries and would require significant redesign to work as-is. All of this infrastructure is very specific to the language version which does not apply to external libraries.

Also note that all libraries other than the standard library have a huge advantage which is that they are published through crates.io. Management of multiple versions of a library is super easy, as well as declaring dependencies.

All in all, we have personally been able to come up with a concrete use case where stability attributes provide guarantees for external libraries that Cargo does not already do so. With this in mind, we've architected the current iteration towards more favoring the compiler and language and planning for similar functionality through Cargo externally if necessary

This comment has been minimized.

@quantheory

quantheory Dec 19, 2014

Contributor

The major purpose of this is to prevent access entirely to #[unstable] APIs, not to just warn-on-use. [... A]lmost all of the benefits of the stability attributes here are gained through libraries declaring semver-requirement dependencies on one another themselves.

Maybe a better way to put it is this: In my view, this is not extending the existing attributes. It's more like this proposal simply removes the existing attributes, and then reuses some of the names for a very different set of attributes that have only a little functionality in common with the old ones.

It would not be totally crazy, I think, for the old and new versions to coexist without having the same spelling. Of course, maybe it's not considered worthwhile to maintain the current attributes in this way, but I still would be happier if some way of annotating unstable/stable/deprecated remained for third-party libraries.

I envision the ability to have unstable features in Cargo manifest (note I mean Cargo features, not stability-attribute features).

I'm still a bit confused. For me, the point is that some projects may, for whatever reason, have a significant number of users developing against versions of their libraries with unstable features. Maybe the library is still on version 0.y.z, or maybe they just have popular nightly/alpha/beta builds. In this case, you want to (a) communicate which features are still considered unstable in documentation, and (b) give users an way to automatically check which unstable features they are using. This may even apply to developers preparing releases, who want to be sure that they haven't accidentally left in unstable features that they thought were already rejected and removed.

For stable versions, there are similar issues for items in the API that are deprecated (as you mention in the RFC already).

For these uses, I think it's best to be able to annotate the source code directly (especially because that's also where the rustdoc documentation is). Listing unstable features in the Cargo manifest is useful, but I think that it solves a different set of problems. (Unless I'm still totally misunderstanding your suggestion.)

[P]ackages published to crates.io will not have any experimental features.

Note that this means that package versions that have to have a mix of newly-stable and unstable features (e.g. 0.y.z packages trying to get early adopters to try things out) really have to live mostly outside the crates.io ecosystem, which means that they may not have so much of an advantage over the standard library.

@quantheory

quantheory Dec 19, 2014

Contributor

The major purpose of this is to prevent access entirely to #[unstable] APIs, not to just warn-on-use. [... A]lmost all of the benefits of the stability attributes here are gained through libraries declaring semver-requirement dependencies on one another themselves.

Maybe a better way to put it is this: In my view, this is not extending the existing attributes. It's more like this proposal simply removes the existing attributes, and then reuses some of the names for a very different set of attributes that have only a little functionality in common with the old ones.

It would not be totally crazy, I think, for the old and new versions to coexist without having the same spelling. Of course, maybe it's not considered worthwhile to maintain the current attributes in this way, but I still would be happier if some way of annotating unstable/stable/deprecated remained for third-party libraries.

I envision the ability to have unstable features in Cargo manifest (note I mean Cargo features, not stability-attribute features).

I'm still a bit confused. For me, the point is that some projects may, for whatever reason, have a significant number of users developing against versions of their libraries with unstable features. Maybe the library is still on version 0.y.z, or maybe they just have popular nightly/alpha/beta builds. In this case, you want to (a) communicate which features are still considered unstable in documentation, and (b) give users an way to automatically check which unstable features they are using. This may even apply to developers preparing releases, who want to be sure that they haven't accidentally left in unstable features that they thought were already rejected and removed.

For stable versions, there are similar issues for items in the API that are deprecated (as you mention in the RFC already).

For these uses, I think it's best to be able to annotate the source code directly (especially because that's also where the rustdoc documentation is). Listing unstable features in the Cargo manifest is useful, but I think that it solves a different set of problems. (Unless I'm still totally misunderstanding your suggestion.)

[P]ackages published to crates.io will not have any experimental features.

Note that this means that package versions that have to have a mix of newly-stable and unstable features (e.g. 0.y.z packages trying to get early adopters to try things out) really have to live mostly outside the crates.io ecosystem, which means that they may not have so much of an advantage over the standard library.

This comment has been minimized.

@alexcrichton

alexcrichton Dec 21, 2014

Member

For these uses, I think it's best to be able to annotate the source code directly (especially because that's also where the rustdoc documentation is). Listing unstable features in the Cargo manifest is useful, but I think that it solves a different set of problems. (Unless I'm still totally misunderstanding your suggestion.)

Concretely speaking, I'm thinking of:

# Cargo.toml
[features.new-api]
experimental = "true"
// src/lib.rs
#[cfg(feature = "new-api")]
pub fn my_new_api() { /* ... */ }

In this case you still annotate features in source code (via cfg attributes), and then the manifest allows Cargo to know what to compile and what's available for dependencies.

Note that this means that package versions that have to have a mix of newly-stable and unstable features (e.g. 0.y.z packages trying to get early adopters to try things out) really have to live mostly outside the crates.io ecosystem, which means that they may not have so much of an advantage over the standard library.

This largely seems ok to me as it's precisely the design of the standard library in a sense. Having to go out of your way a bit to get access to experimental features which can break rapidly seems not the end of the world to me!

@alexcrichton

alexcrichton Dec 21, 2014

Member

For these uses, I think it's best to be able to annotate the source code directly (especially because that's also where the rustdoc documentation is). Listing unstable features in the Cargo manifest is useful, but I think that it solves a different set of problems. (Unless I'm still totally misunderstanding your suggestion.)

Concretely speaking, I'm thinking of:

# Cargo.toml
[features.new-api]
experimental = "true"
// src/lib.rs
#[cfg(feature = "new-api")]
pub fn my_new_api() { /* ... */ }

In this case you still annotate features in source code (via cfg attributes), and then the manifest allows Cargo to know what to compile and what's available for dependencies.

Note that this means that package versions that have to have a mix of newly-stable and unstable features (e.g. 0.y.z packages trying to get early adopters to try things out) really have to live mostly outside the crates.io ecosystem, which means that they may not have so much of an advantage over the standard library.

This largely seems ok to me as it's precisely the design of the standard library in a sense. Having to go out of your way a bit to get access to experimental features which can break rapidly seems not the end of the world to me!

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 18, 2014

Member

@nodakai yes, as @tshepang said we'd recommend the stable channel.

Member

alexcrichton commented Dec 18, 2014

@nodakai yes, as @tshepang said we'd recommend the stable channel.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 18, 2014

Member

@emk To me it sounds like your scenario actually greatly benefits from this infrastructure rather than having it hurt them. For example:

  • The users that want to use an older compiler can continue to do so. Your "ACME" group would simply not upgrade their compiler, and they could continue to make bugfixes quite efficiently. Not only that, but they could even perform cargo upgrade despite their dependencies having moved on to requiring newer versions of Rust. Cargo won't actually update any dependencies to a version which requires a newer version of rust! The key part is here is that Cargo is not actually selecting a rustc version, it is using it as a heuristic to drive the resolution phase of the package graph.
  • The users that want the latest and greatest can also continue to do so. They can upgrade the compiler whenever a new one is release, and all of their code is still compatible with older versions. When they publish a crate with rustc 1.5 but only uses features from 1.1, then this is detected and their libraries are still usable by users with older compilers. They can also publish in confidence versions of crates which require 1.5 features knowing that users on 1.1 won't have to migrate as Cargo won't allow locking to the new package version.

Does that help clear things up?

Member

alexcrichton commented Dec 18, 2014

@emk To me it sounds like your scenario actually greatly benefits from this infrastructure rather than having it hurt them. For example:

  • The users that want to use an older compiler can continue to do so. Your "ACME" group would simply not upgrade their compiler, and they could continue to make bugfixes quite efficiently. Not only that, but they could even perform cargo upgrade despite their dependencies having moved on to requiring newer versions of Rust. Cargo won't actually update any dependencies to a version which requires a newer version of rust! The key part is here is that Cargo is not actually selecting a rustc version, it is using it as a heuristic to drive the resolution phase of the package graph.
  • The users that want the latest and greatest can also continue to do so. They can upgrade the compiler whenever a new one is release, and all of their code is still compatible with older versions. When they publish a crate with rustc 1.5 but only uses features from 1.1, then this is detected and their libraries are still usable by users with older compilers. They can also publish in confidence versions of crates which require 1.5 features knowing that users on 1.1 won't have to migrate as Cargo won't allow locking to the new package version.

Does that help clear things up?

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Dec 19, 2014

@alexcrichton:

The stability attributes were never really tasked with being a general purpose mechanism for all libraries and would require significant redesign to work as-is.

:-( Well, I would really love to have some way to mark a specific library API as unstable, and have users of the library receive a warning (which they could ignore, disable, etc.) like any other warning. This would make it a lot easier to try out experimental features after 1.0. Without something like unstable, every API I release is set in stone until the next x.0.0 version, as per SemVer rules. That makes library design a lot less fun, because every decision has to be correct the very first time. Library design is hard, and frankly, I would like an escape hatch.

I suppose I could just add documentation saying, "This function is NOT STABLE UNDER SEMVER RULES" and let the user deal with the fallout.

I don't think it would require a huge amount of work to have a library-level unstable attribute that issued a compilation warning. Surely that's less than a couple dozen lines of code? But if there's not a lot of demand for a library-level unstable, I'll just go with a documentation-level convention, and implement my own unstable_api (or whatever) attribute as a plugin later on. For library use, it really doesn't need to be complex at all. I don't need a heavy-duty, Cargo-based solution, at least not personally.

The users that want to use an older compiler can continue to do so. Your "ACME" group would simply not upgrade their compiler, and they could continue to make bugfixes quite efficiently.

Hmm. Well, my "ACME" example was inspired a real-world Rails company, and in the real-world case, they actually had two products: One running on Rails 1.8, and one on Rails 1.9. So they needed to maintain multiple Ruby installations using tools like RVM, and keep very careful track of the exact versions of Ruby used by each project. Originally, before Bundler and RVM, this was an absolute nightmare. But then Bundler, RVM, Heroku, etc., all agreed on a common way to specify exactly which version of Ruby a project should use, and everything worked beautifully:

  • RVM could switch to the appropriate Ruby version easily.
  • Heroku could choose the correct Ruby version when deploying.
  • Humans could just look a Gemfile and see what version was used.

This was just an absolutely lovely system, and it made life very easy for everybody. I think there's a lot to be said for having a standard file that says, "This project has been verified to work with Rust 1.0.2." Node's NPM has something like this:

{
   // ...
  "engine": "node 0.4.1"
}

Ruby's Bundler has something like this, too:

source "https://rubygems.org"
ruby "1.9.3"

…and so on. It's a pretty widespread feature of most Cargo-like tools.

heroku-buildpack-rust needs something like this: When a user asks it to deploy a project on Heroku, it actually needs to download a appropriate version of rustc and cargo, and use those to build the code. I can't just say, "Oh, use the latest compiler version. What could go wrong?"

If Cargo doesn't want to imitate Bundler and NPM and provide an optional "Rust version" attribute for bin packages, then heroku-buildpack-rust will probably check a top-level .rustc_version file. I need something like this somewhere, and I personally think it makes sense to have a centralized location.

Thank you for listening to my meandering design comments! :-)

emk commented Dec 19, 2014

@alexcrichton:

The stability attributes were never really tasked with being a general purpose mechanism for all libraries and would require significant redesign to work as-is.

:-( Well, I would really love to have some way to mark a specific library API as unstable, and have users of the library receive a warning (which they could ignore, disable, etc.) like any other warning. This would make it a lot easier to try out experimental features after 1.0. Without something like unstable, every API I release is set in stone until the next x.0.0 version, as per SemVer rules. That makes library design a lot less fun, because every decision has to be correct the very first time. Library design is hard, and frankly, I would like an escape hatch.

I suppose I could just add documentation saying, "This function is NOT STABLE UNDER SEMVER RULES" and let the user deal with the fallout.

I don't think it would require a huge amount of work to have a library-level unstable attribute that issued a compilation warning. Surely that's less than a couple dozen lines of code? But if there's not a lot of demand for a library-level unstable, I'll just go with a documentation-level convention, and implement my own unstable_api (or whatever) attribute as a plugin later on. For library use, it really doesn't need to be complex at all. I don't need a heavy-duty, Cargo-based solution, at least not personally.

The users that want to use an older compiler can continue to do so. Your "ACME" group would simply not upgrade their compiler, and they could continue to make bugfixes quite efficiently.

Hmm. Well, my "ACME" example was inspired a real-world Rails company, and in the real-world case, they actually had two products: One running on Rails 1.8, and one on Rails 1.9. So they needed to maintain multiple Ruby installations using tools like RVM, and keep very careful track of the exact versions of Ruby used by each project. Originally, before Bundler and RVM, this was an absolute nightmare. But then Bundler, RVM, Heroku, etc., all agreed on a common way to specify exactly which version of Ruby a project should use, and everything worked beautifully:

  • RVM could switch to the appropriate Ruby version easily.
  • Heroku could choose the correct Ruby version when deploying.
  • Humans could just look a Gemfile and see what version was used.

This was just an absolutely lovely system, and it made life very easy for everybody. I think there's a lot to be said for having a standard file that says, "This project has been verified to work with Rust 1.0.2." Node's NPM has something like this:

{
   // ...
  "engine": "node 0.4.1"
}

Ruby's Bundler has something like this, too:

source "https://rubygems.org"
ruby "1.9.3"

…and so on. It's a pretty widespread feature of most Cargo-like tools.

heroku-buildpack-rust needs something like this: When a user asks it to deploy a project on Heroku, it actually needs to download a appropriate version of rustc and cargo, and use those to build the code. I can't just say, "Oh, use the latest compiler version. What could go wrong?"

If Cargo doesn't want to imitate Bundler and NPM and provide an optional "Rust version" attribute for bin packages, then heroku-buildpack-rust will probably check a top-level .rustc_version file. I need something like this somewhere, and I personally think it makes sense to have a centralized location.

Thank you for listening to my meandering design comments! :-)

@quantheory

This comment has been minimized.

Show comment
Hide comment
@quantheory

quantheory Dec 19, 2014

Contributor

Sorry for the huge line comment (and if I sounded snippy initially; I was being kept up and probably a bit cranky last night). Just one more thing:

@emk says: "I'll just go with a documentation-level convention, and implement my own unstable_api (or whatever) attribute as a plugin later on." If a lot of people agree that the current behavior of #[unstable] is useful, I imagine that this become a typical practice. It's just unfortunate to have something implemented and give it up.

Contributor

quantheory commented Dec 19, 2014

Sorry for the huge line comment (and if I sounded snippy initially; I was being kept up and probably a bit cranky last night). Just one more thing:

@emk says: "I'll just go with a documentation-level convention, and implement my own unstable_api (or whatever) attribute as a plugin later on." If a lot of people agree that the current behavior of #[unstable] is useful, I imagine that this become a typical practice. It's just unfortunate to have something implemented and give it up.

@SimonSapin

This comment has been minimized.

Show comment
Hide comment
@SimonSapin

SimonSapin Dec 19, 2014

Contributor

Hmm. Well, my "ACME" example was inspired a real-world Rails company, and in the real-world case, they actually had two products: One running on Rails 1.8, and one on Rails 1.9. So they needed to maintain multiple Ruby installations using tools like RVM, and keep very careful track of the exact versions of Ruby used by each project.

Yes. I don’t know about the details of how to achieve it, but dealing with multiple versions is very useful. See rust-lang/rust#19263 and rust-lang/cargo#967. (I don’t know if this is in the scope of this RFC, though.)

Contributor

SimonSapin commented Dec 19, 2014

Hmm. Well, my "ACME" example was inspired a real-world Rails company, and in the real-world case, they actually had two products: One running on Rails 1.8, and one on Rails 1.9. So they needed to maintain multiple Ruby installations using tools like RVM, and keep very careful track of the exact versions of Ruby used by each project.

Yes. I don’t know about the details of how to achieve it, but dealing with multiple versions is very useful. See rust-lang/rust#19263 and rust-lang/cargo#967. (I don’t know if this is in the scope of this RFC, though.)

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Dec 19, 2014

Yes. I don’t know about the details of how to achieve it, but dealing with multiple versions is very useful.

Well, I mostly just care about having a single, standard place to declare "This application has been tested with Rust 1.0.3." This is how Bundler, NPM, etc. work. The process of actually finding Rust 1.0.3 is the responsibility of other tools.

But this is a fairly minor point. If the Cargo maintainers feel that having some way to declare a tested/official Rust version is a bad idea, then I'll try to establish a separate community standard among tools like heroku-buildpack-rust.

emk commented Dec 19, 2014

Yes. I don’t know about the details of how to achieve it, but dealing with multiple versions is very useful.

Well, I mostly just care about having a single, standard place to declare "This application has been tested with Rust 1.0.3." This is how Bundler, NPM, etc. work. The process of actually finding Rust 1.0.3 is the responsibility of other tools.

But this is a fairly minor point. If the Cargo maintainers feel that having some way to declare a tested/official Rust version is a bad idea, then I'll try to establish a separate community standard among tools like heroku-buildpack-rust.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Dec 19, 2014

Member

@emk

Well, I would really love to have some way to mark a specific library API as unstable, and have users of the library receive a warning (which they could ignore, disable, etc.) like any other warning.

This is actually one of the key points about #[unstable] which we don't think will work for library developers. As part of the standard distribution this is a hard error by default, and you have to get on nightly to make it only a warning. If you want this sort of behavior for library design, then the experimental features in Cargo I was talking about are probably what you want (completely gating access on stable channels).

That being said, this form of attribute could always come back in the form of a plugin! The external lint infrastructure is ripe for these sorts of extensions to the compiler.

One running on Rails 1.8, and one on Rails 1.9. So they needed to maintain multiple Ruby installations using tools like RVM, and keep very careful track of the exact versions of Ruby used by each project

Hm, that is a good point! I suspect that the issues that @SimonSapin pointed to are definitely related to this. While slightly out of scope for this RFC, I'd love to explore how Cargo can help with the channels of Rust and versions over time, but it's not super clear to me precisely what that is (need a release first!)

Member

alexcrichton commented Dec 19, 2014

@emk

Well, I would really love to have some way to mark a specific library API as unstable, and have users of the library receive a warning (which they could ignore, disable, etc.) like any other warning.

This is actually one of the key points about #[unstable] which we don't think will work for library developers. As part of the standard distribution this is a hard error by default, and you have to get on nightly to make it only a warning. If you want this sort of behavior for library design, then the experimental features in Cargo I was talking about are probably what you want (completely gating access on stable channels).

That being said, this form of attribute could always come back in the form of a plugin! The external lint infrastructure is ripe for these sorts of extensions to the compiler.

One running on Rails 1.8, and one on Rails 1.9. So they needed to maintain multiple Ruby installations using tools like RVM, and keep very careful track of the exact versions of Ruby used by each project

Hm, that is a good point! I suspect that the issues that @SimonSapin pointed to are definitely related to this. While slightly out of scope for this RFC, I'd love to explore how Cargo can help with the channels of Rust and versions over time, but it's not super clear to me precisely what that is (need a release first!)

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Dec 19, 2014

@alexcrichton Thank you for your response! My apologies for being so stubborn about this issue, too.

This is actually one of the key points about #[unstable] which we don't think will work for library developers. As part of the standard distribution this is a hard error by default, and you have to get on nightly to make it only a warning.

I agree that a hard error #[unstable] makes no sense for library authors.

My proposal is to have two versions of #[unstable], with different names:

  1. Rustc/std could use something like #[rustc_unstable] or #[std_unstable], with hard error semantics, as proposed here.
  2. Third-party libraries could use something like #[unstable] which defaulted to a warning: "Hey, you know, this API may still change." I'd guess this would be a very short lint plugin, and I'd be happy to try to contribute it.

Here's a use-case for for something like #[unstable] in a library:

#[deriving(Show, Default)]
pub struct Hints<'a> {
    /// A value from an HTTP Content-Language header.  The value "fr,en"
    /// will bias the decoder towards French and English.
    pub content_language: Option<&'a str>,

    /// The top-level domain associated with this text.  The value "fr"
    /// will bias the decoder towards French.
    pub tld: Option<&'a str>,

    /// The original encoding of the text, before it was converted to
    /// UTF-8.  See `Encoding` for legal values.
    #[experimental]
    pub encoding: Option<Encoding>,

    /// An extra language hint.
    pub language: Option<Lang>
}

Here, I've finalized 3 out of 4 fields in a struct, but the encoding field is tricky. Right now, it's represented by a low-level Encoding enum taken directly from the C headers, but it should be a high-level type from rust-encoding. Unfortunately, there wasn't any way to depend on the rust-encoding library without dragging in some pretty huge data tables. So I marked this one field #[experimental] until we can sort this out. I don't want to block the release of an entire library over one minor design decision.

Basically, I feel that #[unstable] is far to useful a name to be reserved for something that's only used internally by std. I agree that the standard library needs a complicated and specialized mechanism for handling these issues. I just feel that library writers should have a simple and straightforward way to mark certain features as falling outside of the usual SemVer rules.

Needless to say, everything I argue here goes equally for #[deprecated]. Third-party libraries really do need some way to deprecate APIs, too.

Anyway, thank you for patiently listening to this long argument in favor of library-level #[unstable] and #[deprecated]. :-) I know it must get tedious to read lots of posts about, "Hey! Where did my feature go?" And I really appreciate the work the core team is putting in to make this release happen.

emk commented Dec 19, 2014

@alexcrichton Thank you for your response! My apologies for being so stubborn about this issue, too.

This is actually one of the key points about #[unstable] which we don't think will work for library developers. As part of the standard distribution this is a hard error by default, and you have to get on nightly to make it only a warning.

I agree that a hard error #[unstable] makes no sense for library authors.

My proposal is to have two versions of #[unstable], with different names:

  1. Rustc/std could use something like #[rustc_unstable] or #[std_unstable], with hard error semantics, as proposed here.
  2. Third-party libraries could use something like #[unstable] which defaulted to a warning: "Hey, you know, this API may still change." I'd guess this would be a very short lint plugin, and I'd be happy to try to contribute it.

Here's a use-case for for something like #[unstable] in a library:

#[deriving(Show, Default)]
pub struct Hints<'a> {
    /// A value from an HTTP Content-Language header.  The value "fr,en"
    /// will bias the decoder towards French and English.
    pub content_language: Option<&'a str>,

    /// The top-level domain associated with this text.  The value "fr"
    /// will bias the decoder towards French.
    pub tld: Option<&'a str>,

    /// The original encoding of the text, before it was converted to
    /// UTF-8.  See `Encoding` for legal values.
    #[experimental]
    pub encoding: Option<Encoding>,

    /// An extra language hint.
    pub language: Option<Lang>
}

Here, I've finalized 3 out of 4 fields in a struct, but the encoding field is tricky. Right now, it's represented by a low-level Encoding enum taken directly from the C headers, but it should be a high-level type from rust-encoding. Unfortunately, there wasn't any way to depend on the rust-encoding library without dragging in some pretty huge data tables. So I marked this one field #[experimental] until we can sort this out. I don't want to block the release of an entire library over one minor design decision.

Basically, I feel that #[unstable] is far to useful a name to be reserved for something that's only used internally by std. I agree that the standard library needs a complicated and specialized mechanism for handling these issues. I just feel that library writers should have a simple and straightforward way to mark certain features as falling outside of the usual SemVer rules.

Needless to say, everything I argue here goes equally for #[deprecated]. Third-party libraries really do need some way to deprecate APIs, too.

Anyway, thank you for patiently listening to this long argument in favor of library-level #[unstable] and #[deprecated]. :-) I know it must get tedious to read lots of posts about, "Hey! Where did my feature go?" And I really appreciate the work the core team is putting in to make this release happen.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Dec 19, 2014

Member

@emk

Thanks for your clear articulation of what you'd like to see as a library author. I definitely see where you're coming from -- I am very sympathetic to wanting stability attributes in the Cargo ecosystem.

It's a feature we definitely want to provide at some point. But we don't necessarily want to just keep the old stability system alongside the release channel system for Rust -- that would give two very different meanings to stability attributes. We also want to gain some experience using release channels for Rust itself before moving forward with something for the libraries.

In short, I think we're likely to provide these facilities for external libraries in some form in the future, but we want to take our time to get the design right. In the interim, once plugins are stabilized it should be possible to build something like our existing stability system externally.

I hope you can understand where we're coming from here -- as with so many feature requests, we'd really like to offer it, but we need time before we can commit to a design, and there is a lot of other stuff in flight right now.

Member

aturon commented Dec 19, 2014

@emk

Thanks for your clear articulation of what you'd like to see as a library author. I definitely see where you're coming from -- I am very sympathetic to wanting stability attributes in the Cargo ecosystem.

It's a feature we definitely want to provide at some point. But we don't necessarily want to just keep the old stability system alongside the release channel system for Rust -- that would give two very different meanings to stability attributes. We also want to gain some experience using release channels for Rust itself before moving forward with something for the libraries.

In short, I think we're likely to provide these facilities for external libraries in some form in the future, but we want to take our time to get the design right. In the interim, once plugins are stabilized it should be possible to build something like our existing stability system externally.

I hope you can understand where we're coming from here -- as with so many feature requests, we'd really like to offer it, but we need time before we can commit to a design, and there is a lot of other stuff in flight right now.

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Dec 19, 2014

Thank you for your kind response, @aturon! I'll try to figure out the design disconnect one last time, offer to prepare a PR, and then drop this issue. :-)

But we don't necessarily want to just keep the old stability system alongside the release channel system for Rust -- that would give two very different meanings to stability attributes. We also want to gain some experience using release channels for Rust itself before moving forward with something for the libraries.

I think this is where the disconnect is happening. The std crate needs a pretty sophisticated design:

  1. A fine-grained way to enable some experimental features without enabling all of them.
  2. Release channels.
  3. Some way to automatically promote crates from a nightly compiler to a stable compiler when features are accepted.

As a library author, I need:

  1. Some way to say, "Hey, be careful with this, it's not stable yet."
  2. Some way to say, "Hey, this is going away at the next x.0.0 release."

I don't need fine-grained control over multiple unstable features, or release channels, or anything else like that. A really simple system with nothing but #[unstable], #[deprecated] and a couple of lint warnings would cover at least 95% of the use cases for libraries in the crates ecosystem. At least in theory, this could be implemented today: It's distinct from what std is doing; it's ultra-simple to design; and it's probably not much code.

Now, given the realities of getting 1.0 out the door, I understand that it might be necessary to remove the simple versions of #[unstable] and #[deprecated], and not provide them to crate authors until a more complex replacement system can be designed and thoroughly tested in the real world. But I would personally prefer to keep a simple-but-stupid system for library authors that covers the 95% case, instead of waiting for a really mature design that handles all kinds of exotic (for library writers) use cases.

Anyway, I'd be happy to contribute code, if that's the primary thing needed. But if there's no consensus in favor of an ultra-simple #[unstable] for libraries, I can just resort to using notes in the documentation for the time being, and hack up a standalone lint if that ever becomes possible on the stable channel.

Anyway, thank you to everybody for listening to this plea. And I'll leave this alone from here on, unless somebody asks for a PR or something. :-)

emk commented Dec 19, 2014

Thank you for your kind response, @aturon! I'll try to figure out the design disconnect one last time, offer to prepare a PR, and then drop this issue. :-)

But we don't necessarily want to just keep the old stability system alongside the release channel system for Rust -- that would give two very different meanings to stability attributes. We also want to gain some experience using release channels for Rust itself before moving forward with something for the libraries.

I think this is where the disconnect is happening. The std crate needs a pretty sophisticated design:

  1. A fine-grained way to enable some experimental features without enabling all of them.
  2. Release channels.
  3. Some way to automatically promote crates from a nightly compiler to a stable compiler when features are accepted.

As a library author, I need:

  1. Some way to say, "Hey, be careful with this, it's not stable yet."
  2. Some way to say, "Hey, this is going away at the next x.0.0 release."

I don't need fine-grained control over multiple unstable features, or release channels, or anything else like that. A really simple system with nothing but #[unstable], #[deprecated] and a couple of lint warnings would cover at least 95% of the use cases for libraries in the crates ecosystem. At least in theory, this could be implemented today: It's distinct from what std is doing; it's ultra-simple to design; and it's probably not much code.

Now, given the realities of getting 1.0 out the door, I understand that it might be necessary to remove the simple versions of #[unstable] and #[deprecated], and not provide them to crate authors until a more complex replacement system can be designed and thoroughly tested in the real world. But I would personally prefer to keep a simple-but-stupid system for library authors that covers the 95% case, instead of waiting for a really mature design that handles all kinds of exotic (for library writers) use cases.

Anyway, I'd be happy to contribute code, if that's the primary thing needed. But if there's no consensus in favor of an ultra-simple #[unstable] for libraries, I can just resort to using notes in the documentation for the time being, and hack up a standalone lint if that ever becomes possible on the stable channel.

Anyway, thank you to everybody for listening to this plea. And I'll leave this alone from here on, unless somebody asks for a PR or something. :-)

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Dec 31, 2014

Contributor

I've pushed a new revision that does a few minor things:

  • It renames the attributes to 'staged_' to reserve the 'nice' names for future use, and explicitly mentions that the current names will have no semantic meaning (but will not be warned about). The ugly names are fine since they're just for in-tree use.
  • It emphasizes that the automatic version detection scheme is fairly speculative.
Contributor

brson commented Dec 31, 2014

I've pushed a new revision that does a few minor things:

  • It renames the attributes to 'staged_' to reserve the 'nice' names for future use, and explicitly mentions that the current names will have no semantic meaning (but will not be warned about). The ugly names are fine since they're just for in-tree use.
  • It emphasizes that the automatic version detection scheme is fairly speculative.
@Valloric

This comment has been minimized.

Show comment
Hide comment
@Valloric

Valloric Dec 31, 2014

It emphasizes that the automatic version detection scheme is fairly speculative.

I appreciate the recent changes made here, but the proposed system still seems like lots of unnecessary complexity that will end up shooting users in the foot. I've asked this before without receiving an answer, so I must ask again: the language itself is a versioned dependency much like any other library, so why is it being treated differently? Why aren't we asking the user to always explicitly state which version of the language their library requires? There should be no "shoot yourself in the foot" option where cargo guesses for you. I'm fine with cargo verifying that the version you stated as a minimum actually works (if you say stable but are using feature directives, cargo should complain) and even cargo recommending a version based on its analysis (while stating that the recommendation is a best-effort guess and shouldn't be taken as gospel), but it really should always be leaving the final decision to you.

This is the simplest approach and the one that's least likely to lead to nasty surprises.

Valloric commented Dec 31, 2014

It emphasizes that the automatic version detection scheme is fairly speculative.

I appreciate the recent changes made here, but the proposed system still seems like lots of unnecessary complexity that will end up shooting users in the foot. I've asked this before without receiving an answer, so I must ask again: the language itself is a versioned dependency much like any other library, so why is it being treated differently? Why aren't we asking the user to always explicitly state which version of the language their library requires? There should be no "shoot yourself in the foot" option where cargo guesses for you. I'm fine with cargo verifying that the version you stated as a minimum actually works (if you say stable but are using feature directives, cargo should complain) and even cargo recommending a version based on its analysis (while stating that the recommendation is a best-effort guess and shouldn't be taken as gospel), but it really should always be leaving the final decision to you.

This is the simplest approach and the one that's least likely to lead to nasty surprises.

@wycats

This comment has been minimized.

Show comment
Hide comment
@wycats

wycats Dec 31, 2014

Contributor

Why aren't we asking the user to always explicitly state which version of the language their library requires? There should be no "shoot yourself in the foot" option where cargo guesses for you. I'm fine with cargo verifying that the version you stated as a minimum actually works (if you say stable but are using feature directives, cargo should complain) and even cargo recommending a version based on its analysis (while stating that the recommendation is a best-effort guess and shouldn't be taken as gospel), but it really should always be leaving the final decision to you.

The approach I had in mind wasn't that Cargo says "this will definitely work on Rust 1.3", but rather "this will definitely not work before Rust 1.3". The idea is that if people leave out the version number, at least we can give a useful error message to people using the library on versions of Rust we know for sure won't work.

Contributor

wycats commented Dec 31, 2014

Why aren't we asking the user to always explicitly state which version of the language their library requires? There should be no "shoot yourself in the foot" option where cargo guesses for you. I'm fine with cargo verifying that the version you stated as a minimum actually works (if you say stable but are using feature directives, cargo should complain) and even cargo recommending a version based on its analysis (while stating that the recommendation is a best-effort guess and shouldn't be taken as gospel), but it really should always be leaving the final decision to you.

The approach I had in mind wasn't that Cargo says "this will definitely work on Rust 1.3", but rather "this will definitely not work before Rust 1.3". The idea is that if people leave out the version number, at least we can give a useful error message to people using the library on versions of Rust we know for sure won't work.

@Valloric

This comment has been minimized.

Show comment
Hide comment
@Valloric

Valloric Dec 31, 2014

@wycats Thank you for the explanation. That's a lot more reasonable.

The idea is that if people leave out the version number

What I'm saying is don't make that an option. You must state the language version number when you upload a library to crates.io. No confusion possible. If you leave out the version number, cargo complains saying you must, and tells you it has detected a certain minimum version based on what you used in the code. Say, it detects 1.2.1 as a min version. You on the other hand know that that version has a bug in a std library that affects your code, and you pick 1.2.2 as the min language version.

If you pick 1.2.0, cargo complains that the version you picked can't compile your code (based on its analysis; it should even tell you why).

Valloric commented Dec 31, 2014

@wycats Thank you for the explanation. That's a lot more reasonable.

The idea is that if people leave out the version number

What I'm saying is don't make that an option. You must state the language version number when you upload a library to crates.io. No confusion possible. If you leave out the version number, cargo complains saying you must, and tells you it has detected a certain minimum version based on what you used in the code. Say, it detects 1.2.1 as a min version. You on the other hand know that that version has a bug in a std library that affects your code, and you pick 1.2.2 as the min language version.

If you pick 1.2.0, cargo complains that the version you picked can't compile your code (based on its analysis; it should even tell you why).

@wycats

This comment has been minimized.

Show comment
Hide comment
@wycats

wycats Dec 31, 2014

Contributor

What I'm saying is don't make that an option. You must state the language version number when you upload a library to crates.io. No confusion possible. If you leave out the version number, cargo complains saying you must, and tells you it has detected a certain minimum version based on what you used in the code. Say, it detects 1.2.1 as a min version. You on the other hand know that that version has a bug in std library that affects your code, and you pick 1.2.2 as the min language version.

We could do this, but I'm not sure how this would be an improvement. Most of the time, people would just blindly copy in what Cargo tells you to use, so why not just have us insert it automatically?

Contributor

wycats commented Dec 31, 2014

What I'm saying is don't make that an option. You must state the language version number when you upload a library to crates.io. No confusion possible. If you leave out the version number, cargo complains saying you must, and tells you it has detected a certain minimum version based on what you used in the code. Say, it detects 1.2.1 as a min version. You on the other hand know that that version has a bug in std library that affects your code, and you pick 1.2.2 as the min language version.

We could do this, but I'm not sure how this would be an improvement. Most of the time, people would just blindly copy in what Cargo tells you to use, so why not just have us insert it automatically?

@Valloric

This comment has been minimized.

Show comment
Hide comment
@Valloric

Valloric Dec 31, 2014

We could do this, but I'm not sure how this would be an improvement. Most of the time, people would just blindly copy in what Cargo tells you to use, so why not just have us insert it automatically?

Because it forces you to think about it. By doing it automatically, you're teaching people to blindly trust the version cargo spits out. By not doing it automatically, you make it explicit that cargo is making an (educated) guess.

If you're aiming for your library to be compatible with 1.1, but cargo tells you it detects 1.2 as a minimum, you know you messed something up and you know it before publishing the library.

Valloric commented Dec 31, 2014

We could do this, but I'm not sure how this would be an improvement. Most of the time, people would just blindly copy in what Cargo tells you to use, so why not just have us insert it automatically?

Because it forces you to think about it. By doing it automatically, you're teaching people to blindly trust the version cargo spits out. By not doing it automatically, you make it explicit that cargo is making an (educated) guess.

If you're aiming for your library to be compatible with 1.1, but cargo tells you it detects 1.2 as a minimum, you know you messed something up and you know it before publishing the library.

@reem

This comment has been minimized.

Show comment
Hide comment
@reem

reem Jan 1, 2015

I agree completely with @emk and would be very sad if the complicated and very niche needs of the standard library for versioning ruined the system for the rest of us. The std library should just use another set of attributes which could optionally be exposed to library authors and there should remain a simple set of attributes for setting stability of library APIs.

reem commented Jan 1, 2015

I agree completely with @emk and would be very sad if the complicated and very niche needs of the standard library for versioning ruined the system for the rest of us. The std library should just use another set of attributes which could optionally be exposed to library authors and there should remain a simple set of attributes for setting stability of library APIs.

@aturon

This comment has been minimized.

Show comment
Hide comment
@aturon

aturon Jan 1, 2015

Member

@reem To be clear, everybody wants the stability attribute system to be useable in the crates.io ecosystem. The problem is that the current system has an extremely coarse-grained "opt in" where you have to accept unstable APIs from any sources, rather than saying only that certain unstable APIs are OK with you. We'd like to change this before committing to the design.

I'm curious whether you agree that this is a problem with the design.

It may be that using a feature naming system in crates.io is the right way to go; or maybe an item-based opt-in. It's not clear yet. But we plan to look into this ASAP.

Also, we discussed the possibility of letting the attributes be used but not yet linting against them, which would at least retain a way to signal stability without committing to the blanket opt-in.

Member

aturon commented Jan 1, 2015

@reem To be clear, everybody wants the stability attribute system to be useable in the crates.io ecosystem. The problem is that the current system has an extremely coarse-grained "opt in" where you have to accept unstable APIs from any sources, rather than saying only that certain unstable APIs are OK with you. We'd like to change this before committing to the design.

I'm curious whether you agree that this is a problem with the design.

It may be that using a feature naming system in crates.io is the right way to go; or maybe an item-based opt-in. It's not clear yet. But we plan to look into this ASAP.

Also, we discussed the possibility of letting the attributes be used but not yet linting against them, which would at least retain a way to signal stability without committing to the blanket opt-in.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Jan 2, 2015

Member

This RFC has had some pushback against removing the stability attributes for this RFC, but I believe that @brson has addressed many of these concerns by renaming the attributes to their less appealing counterparts (prefixed with staged_). One of the primary reasons for removing them as-is for now is that the opt-in mechanism isn't something that we'd like to stabilize as part of the language today, and we certainly plan on pursuing other vectors for a stability-attribute-like system, perhaps through Cargo. The removal of stability attributes as-is is likely to be quite temporary!

Additionally, @Valloric has pointed out that the "Cargo version detection" mechanisms may be too ambitious, but with @wycats's clarification and @brson's update the wording around this has been toned down significantly. As @wycats points out, the precise implementation in Cargo can develop over time.

As a result, we've decided to merge this RFC as-is. Thanks again for the discussion everyone!

Member

alexcrichton commented Jan 2, 2015

This RFC has had some pushback against removing the stability attributes for this RFC, but I believe that @brson has addressed many of these concerns by renaming the attributes to their less appealing counterparts (prefixed with staged_). One of the primary reasons for removing them as-is for now is that the opt-in mechanism isn't something that we'd like to stabilize as part of the language today, and we certainly plan on pursuing other vectors for a stability-attribute-like system, perhaps through Cargo. The removal of stability attributes as-is is likely to be quite temporary!

Additionally, @Valloric has pointed out that the "Cargo version detection" mechanisms may be too ambitious, but with @wycats's clarification and @brson's update the wording around this has been toned down significantly. As @wycats points out, the precise implementation in Cargo can develop over time.

As a result, we've decided to merge this RFC as-is. Thanks again for the discussion everyone!

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton
Member

alexcrichton commented Jan 2, 2015

Tracking issue: rust-lang/rust#20445

@alexcrichton alexcrichton merged commit ecb78f3 into rust-lang:master Jan 2, 2015

@l0kod

This comment has been minimized.

Show comment
Hide comment
@l0kod

l0kod Jan 3, 2015

Maybe out of scope but rust-lang/rust#18815 should be considered too.

l0kod commented Jan 3, 2015

Maybe out of scope but rust-lang/rust#18815 should be considered too.

@quantheory

This comment has been minimized.

Show comment
Hide comment
@quantheory

quantheory Jan 5, 2015

Contributor

Though this was merged while I was on vacation, I spilled a lot of words before that and want to point out a couple of final things:

  • The revisions of @brson and explanation from @wycats were very helpful; I am pretty much on board with the "detection" proposal now (especially since further developments are now up to Cargo).

  • I'm somewhat reassured by @alexcrichton's explanation. I'm also happier that the more ergonomic/mneumonic "unstable"/"stable"/"deprecated" names have been freed up for either a future version or plugin use. But when we get closer to stabilizing some kind of interface for plugins, this needs to be revisited, I think.

    In particular, it would be very nice to be able to tweak some aspects of rustdoc styling/formatting with directives. (An obvious example is the current color-coding, where stable is green and deprecated is purple, but regular/italic text or crosshatching, for the visually impaired or colorblind or for B/W printing, would be neat too.)

Contributor

quantheory commented Jan 5, 2015

Though this was merged while I was on vacation, I spilled a lot of words before that and want to point out a couple of final things:

  • The revisions of @brson and explanation from @wycats were very helpful; I am pretty much on board with the "detection" proposal now (especially since further developments are now up to Cargo).

  • I'm somewhat reassured by @alexcrichton's explanation. I'm also happier that the more ergonomic/mneumonic "unstable"/"stable"/"deprecated" names have been freed up for either a future version or plugin use. But when we get closer to stabilizing some kind of interface for plugins, this needs to be revisited, I think.

    In particular, it would be very nice to be able to tweak some aspects of rustdoc styling/formatting with directives. (An obvious example is the current color-coding, where stable is green and deprecated is purple, but regular/italic text or crosshatching, for the visually impaired or colorblind or for B/W printing, would be neat too.)

of `#[staged_unstable(feature = "foo")]` APIs unless the current crate
declares `#![feature(foo)]`. This enables crates to declare what API
features of the standard library they rely on without opting in to all
unstable API features.

This comment has been minimized.

@daira

daira Jan 19, 2015

I like this a lot, I think I'll use it in my own language, Noether (not necessarily tied to the "train" release model).

@daira

daira Jan 19, 2015

I like this a lot, I think I'll use it in my own language, Noether (not necessarily tied to the "train" release model).

well recording the version in which the deprecation was performed with
the `since` parameter.
(Occassionally unstable APIs may be deprecated for the sake of easing

This comment has been minimized.

@daira

daira Jan 19, 2015

Typo (occasionally)

@daira

daira Jan 19, 2015

Typo (occasionally)

will generate an error in this case unconditionally.
These steps ensure that the `#[feature]` attribute is used exhaustively
and will check unstable language and library features.

This comment has been minimized.

@daira

daira Jan 19, 2015

I may be reading this wrong, but doesn't it mean that when a feature becomes stable, code that used it will have to be changed to remove the #[feature] attribute? What's the motivation for that?

@daira

daira Jan 19, 2015

I may be reading this wrong, but doesn't it mean that when a feature becomes stable, code that used it will have to be changed to remove the #[feature] attribute? What's the motivation for that?

@chriskrycho chriskrycho referenced this pull request Feb 10, 2017

Closed

Document all features in the reference #38643

0 of 17 tasks complete

@chriskrycho chriskrycho referenced this pull request Mar 11, 2017

Closed

Document all features #9

18 of 48 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment