Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the experimental_keywords ability #2919

Closed
wants to merge 11 commits into from
74 changes: 74 additions & 0 deletions text/0000-experimental_keywords.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
- Feature Name: `experimental_keywords`
- Start Date: 2020-04-30
- RFC PR: [rust-lang/rfcs#0000](https://github.com/rust-lang/rfcs/pull/0000)
- Rust Issue: [rust-lang/rust#0000](https://github.com/rust-lang/rust/issues/0000)

# Summary
[summary]: #summary

This RFC introduce a new way to specify "experimental" keywords. An experimental keyword exposes a fundamental operation of the language and/or compiler _without_ giving that operation its final intended syntax. This allows for user experimentation with the capability introduced by the feature, even on Stable, while allowing the final syntax for the capability to be decided on and stabilized separately from the initial implementation.

An experimental keyword is given with `r#$keyword`, such as:
* `r#$yeet`
* `r#$assembly_block`
* `r#$raw_ref`

# Motivation
[motivation]: #motivation

It is often the case that we know we want a new ability to be available for users to try out as soon as possible, even before we know what the final syntax for that ability should look like. In the past the compiler has had special macros for this purpose (`try!` and `await!`). This RFC is a continuation of that idea, while also trying to improve the understanding to general users that the ability they're using isn't in its final form.
Lokathor marked this conversation as resolved.
Show resolved Hide resolved

Previously, introducing _any_ new keyword at all had to be done only on an edition change because of the compatibility hazard. Since `r # $ token` is not currently a valid token sequence for anything at all, we can use the `r#$` prefix as a way to "namespace" the experimental keywords away from the main language.

That said, it will be possible (on a case by case basis) for an experimental keyword available in Nightly to go through the stabilization steps and then be used with the Stable branch. If this happens, the keyword's meaning and usage becomes as fixed as any other Stable part of the language.
* When the final "real" syntax for an experimental feature is stabilized (assuming that it is) the experimental keyword for that feature would continue to work, and code would continue to compile (example: `try!` became the `?` operator).
* It's also possible that the "real" usage of an experimental keyword will be decided to be a proc-macro in the standard library, rather than some bit of syntax (example: the primary way to access inline assembly is not decided on yet, but it's likely to be a proc-macro).

It is probable that usage of an experimental keyword after the final syntax has been stabilized will fire a warning that you should move to the final syntax, but this can be decided on a case by case basis for each language feature.

# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation

From time to time you may see things like `r#$name` in Rust code. These are "experimental keywords".

An experimental keyword represents accessing the "raw operations" of the compiler and language. They have an intentionally funny syntax because experimental keywords are not intended to be used directly in the long term. Instead, they're a way to add features to the language before we've decided on the final syntax for that feature.

Most experimental keywords are given a final syntax intended for general, long-term use, some time after the feature itself becomes available in the Stable compiler. This way the Rust community has an opportunity to try it out and see how it feels, how they use it in practice, and so on. Some experimental keywords aren't ever given a direct syntax of their own, instead they're simply there so that proc-macros can expand to code that uses the keyword.

[And here we'd ideally we'd add at least one or two real examples to the guide once we have some examples of this in practice]

# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

This RFC alters the language so that `r # $ token` is an accepted token sequence.
* It names an experimental keyword named `token`.
* The experimental keyword named _must_ be known to the compiler. If you attempt to name an experimental keyword that the compiler doesn't recognize it is an error.
* The exact meaning and usage for each experimental keyword depends on the language feature it's associated with, and is not specified here.

# Drawbacks
[drawbacks]: #drawbacks

* It can slightly hurt Rust's stability story to deliberately introduce new keywords that are intended to become deprecated within a relatively short period of time.
* Mitigation 1: Experimental keywords that do become Stable would continue to work even once the final syntax is decided upon and also stabilized. Stable code would not break.
* Mitigation 2: The rather unusual `r#$` prefix stands out as "clearly unusual"

# Rationale and alternatives
[rationale-and-alternatives]: #rationale-and-alternatives

* Alternative: We could select _some other_ sequence of invalid tokens to use as a prefix for experimental keywords.
* `r#$` is used because it is close enough to the "raw literal" and "raw string" syntaxes that people are very likely to understand what's going on even if they're not familiar with a specific experimental keyword when they first see it.
Copy link
Member

@scottmcm scottmcm May 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs a justification for $ in particular. Why is r#$ better than r#@, say? (Not that I think that one's necessarily superior, just that it's not clearly worse either.)

Historically macros have zealously guarded the use of $ for themselves, and it's not impossible to imagine someone trying to do ($mystring:literal) => { r#$mystring }...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Speaking as the RFC author: the current text simply reflects the "first thing that came to mind" during the discussion that lead to this proposal, with not much bike-shedding during that discussion.

Speaking as a user: either way is fine and I have no opinion.

So I'll happily put in whatever people want to decide on for this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r#$mystring doesn't work today, and I don't think it will ever work in the future. You can't just stick a r# next to a lexed token to turn it into a raw string. Macro's use of $ shouldn't really matter here.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kennytm Sure, but macro authors use $ so commonly that it can cause confusion (they might think that r#$foo can work). So why not reduce that confusion by choosing a different character?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well for sure there are plenty of alternatives we could choose.

spellings which currently are tokenization error:

  • r##keyword
  • r#[keyword]
  • r#@keyword / r#!keyword / etc
  • r#805209
  • br#haha_lang_design_go

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, br#keyword was also mentioned in #2151.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only question I have while reading this thread is "why?".
Why come up with these contrived monstrosities messing with lexer if pretty much any combination of two existing keywords would work (do catch style)?

If you want a generic tool for this (which is, again, excessive in my opinion), make __experimental a keyword and use __experimental other_keyword as the combination.


# Prior art
[prior-art]: #prior-art
Lokathor marked this conversation as resolved.
Show resolved Hide resolved

* The Ember framework for javascript places experimental framework abilities into a special module that is clearly experimental so that users can try new things and become familiar with new features even before the feature is fully stable.
Lokathor marked this conversation as resolved.
Show resolved Hide resolved

# Unresolved questions
[unresolved-questions]: #unresolved-questions

None at this time.

# Future possibilities
[future-possibilities]: #future-possibilities

* It might be desirable to also make an `experiments` crate which can provide various proc-macros that use the experimental keywords. We would also "whitelist" this crate so that it's assured to be available on the Rust Playground's list of crates, and then people could share example proposed usages of the experimental proc-macros.