RFC: Start working on a Rust specification #3355
Conversation
m-ou-se
added
T-lang
m-ou-se
added
the
I-lang-nominated
|
What are the stability guarantees of the spec expected to be? That is, say that after the spec is official, we want to make a change to the spec that is "breaking" in the sense that it permits Rust programs to have behaviors that they could not have before (are now a compiler error instead of compiling successfully, print 1 instead of 2, etc.). What is the expected way for this to be handled? I am specifically interested in the case where this change is not accompanied by a breaking change in the behavior of rustc |
|
I think a specification is a great idea, and I'd like to talk about one point in particular
As a developer for one of these alternative compilers, I also believe that "helping other compilers" should not be one of the specification's goals. In fact, I believe alternative compilers could strive to help with that specification. One of my personal goal for gccrs has always been for it to have a positive impact on the language, and help with any standardization/specification/detailing effort. I know that other team members also feel that way :) I'd love for us to be able to provide a fresh set of eyes on the specification, as I'm sure we could raise questions that have caused us trouble when implementing parts of gccrs. Things, behaviors, details that might be obvious to the rustc team, but not necessarily to others. I believe this might help detail some of the spec and hopefully improve it in a meaningful way. Long story short, we'd love to help where we can and I personally appreciate the effort for a spec. |
I am curious about what this means exactly. I would naively assume that one of the useful aspects of a specification is indeed for other implementations of Rust to have a document that they can reference for what the behavior of their implementation ought to be, if they are trying to match the behavior of the Rust compiler. Is it a wording thing? An intellectual property technicality thing? Or does it have an impact on the actual content that does or doesn't go into the specification? (I do not mean to debate this goal, but rather, ask about what it means and perhaps its motivation.) |
|
My understanding of that paragraph is: If a specification has as primary goal to help with development of alternative compilers, then in case of conflicts between compiler and specification the specification should win. For the rust specification as suggested by this rfc however rustc would win and not the specification or any other compiler. |
|
That interpretation would seem to make specification useless. If the final truth is still "whatever rustc does", what's the point of having a spec in the first place? |
|
I think the point is to know what is being done by rustc without having to dig through all the code and to specify where behavior is unspecified and as such may change between rustc versions. For example whether code is UB or if you are allowed to assume that for example the sort method will always make the exact same comparisons in the same order or not. |
|
I wouldn't say that rustc wins by default. I would say that if rustc differs from the spec, that is a bug in either one of them, and which one it is needs to be determined on a case-by-case basis. |
text/0000-rust-spec.md
Outdated
| - The specification will be written in English and will be freely available under a permissive license | ||
| that allows for translations and other derived works, just like all our existing documentation and code. |
There was a problem hiding this comment.
I hope this is not intended to exclude parts of the spec being written in formal languages. :) (e.g., like a-mir-formality or MiniRust)
There was a problem hiding this comment.
No, that's not my intention at all. See also this point later in the RFC:
How certain topics will be specified.
Certain parts of the specification might use a formal language for specifying behavior or syntax. For example, the grammar might be specified as EBNF, and parts of the borrow checker or memory model might be specified by a more formal definition that the document refers to.
It's essentially a scoping question. A spec meant for implementing multiple compilers is a document mainly written for agreement, a spec just for rustc is a tool to document understanding of that one specific tool. That also means that the second is a lot leaner, because there's no multi-party agreement that needs to be sought. The latter can become the former though. |
If you built a rocket and that rocket crashes, you wouldn't update the spec of the rocket to say "it is expected to crash after reaching 3000m altitude". But if you made a typo that says the rocket should crash after reaching 3000m altitude and somehow passed review, you wouldn't add a self detonation device into the rocket just because of this either. |
It means a few things: Mainly, if we were to write a specification aimed at alternative compilers, we'd most likely not only describe what—for example—type inferrence will do, but also what it won't do. (Because having one compiler accept more than another might result in less portable code.) If it's not aimed at other compilers, it might be good enough to only describe what it can do, without specifying all its limits. (You can just run rustc to find out what its limits are. Add type annotations when necessary.) For e.g. safety critical software certification, you only need a specification to explain what (compiled, running) software does. Specifying limitations (of type infererence or the borrow checker, etc.) is not very important. In general, it results in different priorities for various topics. (Things like diagnostics/lints, compiler flags, etc.) Apart from that, it also means that this spec will not have anything to do with the Rust™ trademark. This RFC is not about specifying when someone is or isn't allowed to call their software a Rust™ compiler. That'd be a separate discussion. |
That is a good example of how the choice affects the specification. Which leaves the question: why? Alternative compilers are being created, after all, and it would be beneficial if one compiler didn't accept more than another. So what is the benefit of not focusing on that aspect?
|
No, it's simply separating development of the language from development of compilers for the language. The outcome should be: Language development --[specification]--> Reference compiler ----> Compliant compilers The other compilers are legitimate compilers, but they are not legitimate places for language design questions to be answered. When working off a written specification there is always some degree of interpretation needed. |
It reflects a view that the Rust project delivers the Rust compiler, and in the future a specification describing its intended behavior. An alternative implementation will never be the Rust compiler, and the spec does not express an intent to have several implementations be considered "equally official" just because they all implement the spec. That is clearly different from C/C++. |
Note that this RFC proposes a next step, not barring any steps in the future. The RFC does not make anything legitimate or not, it proposes something achievable. A spec written for all other compilers is quite another ticket to put on the team writing it. |
|
As it's mentioned in the RFC, I'd like to add some detail around the current state of the Ferrocene spec. The Ferrocene Spec exists to solve one axis of what is presented: provide a document for good understand for outsiders to see all the components and pieces of the Rust language as implemented by the compiler version we ship, for the details that needed by our customers. It was one of our main blockers for Ferrocene. It is currently almost complete if:
You may have noticed that it follows the structure of the Ada Specification, taking the contents of the Reference as a starting point. This mainly for 4 reasons:
It is built in mind with:
We find it is very useful in its current state also beyond is current scope scope, e.g. it currently has the most extensive Glossary of Terms currently in use in the ecosystem. It is also currently in practical use, we're consuming it downstream for our requirements tracing activities. We have already indicated that we can offer as a basis for the Rust project. Our exact needs about the specification are rather thin, rather then that it exists. What the spec has on top is a team already experienced with editing the specification daily to build a larger team out of. If you have any questions, prefer pinging me on Zulip if the question is individual, to avoid spamming the thread here. |
@BurntSushi As another example of this: if you're writing a specification specifically to enable alternate implementations, you might overtly provide leeway in implementation, such as by having "minimum maximums" and "maximum minimums". But since we don't actually want variation in implementations (and @CohenArthur as an author of one alternate implementation has said as much as well), adding such ranges only provides drawbacks, not benefits. That doesn't mean we're going to go out of our way to nail down everything the compiler must not do on the first pass (as one might do if developing a specification adversarially), but we don't need to go out of our way to offer multiple incompatible alternatives either. |
|
IMO, as another person working on an alterative implementation, I think that the spec should ultimately be intended to be useful for implementing rust. I do think there are places that variation might be good (for example, in certain implementation limits, like consteval step limit), and some places variation would 100% be down to QoI (for example, precisely what diagnostics are issued for an ill-formed program, and the forms of this diagnostics), though I agree that most well-formedness things should be nailed down and there should be limited implementation-defined and unspecified behaviour. My proposal would be that we target writing a spec for implementations, but not an exhaustive/complete one at first, then develop that spec with implementation experience from rustc, gcc-rs, lccc, mrustc, and any others that pop up between now and then (which I think will certainly happen). This goal need not require us to get everything we want to nail down so nailed in the first version, nor would it conflict with the goal of writing the spec for users. However, I'd expect it would be more reasonable at that point to keep the reference for "advanced/power users" and target the spec more at people who want to know rust down to the low-level functionality, such as arround the skill-level of a certain subset of the regular residents of #dark-arts on the Community discord would be. |
Very nice description. There may be bugs in the spec, or bugs in rustc, and we'll fix the bugs wherever they are, rather than always giving one priority over the other. |
I would certainly expect it to be useful for that purpose. But that's not the same as being primarily intended for that purpose.
A spec written primarily to enable alternate implementations should not do this, and might in fact need to very precisely specify all the ways not to do this.
Writing a spec is a monumental task, and one that will hear requirements from myriad sources, the union of which would make the task intractably huge to accomplish. It's going to be necessary to limit the scope and requirements in order to successfully complete a specification that satisfies those requirements. The proposal, here, is that one of the requirements we don't target is "write a precise enough specification that we're comfortable saying anything following that specification is a Rust implementation". There are many other purposes a specification can serve, and the proposal here is to focus on some of those. |
|
(this is kinda just a +1 comment but) I'm excited to see a plan1 to make progress on creating a specification, and perhaps more importantly, figuring out what exactly that means for Rust. I'm absolutely happy to provide what help I can with the wg-ucg/wg-opsem parts of defining what behavior of Rust we want to specify, and I hope this project can serve to shine some clarifying light on the project. This project will certainly take some time, will be incremental, and there will be bugs found and fixed both in the compiler and the specification as part of the process. But I'm confident in saying that if we decide to pursue a specification with the scope outlined by this RFC, that the Rust project will benefit from the entire process. So here's to the next 342 versions and beyond 🎉 Footnotes
|
|
On Tue, Dec 13, 2022 at 01:48 Josh Triplett ***@***.***> wrote:
I think that the spec *should* ultimately be intended to be useful for
implementing rust.
I would certainly expect it to be useful for that purpose. But that's not
the same as being primarily intended for that purpose.
but may be more lenient in results that may accept slightly more or
slightly fewer programs, or that differ slightly in behaviour
A spec written primarily to enable alternate implementations should *not*
do this, and might in fact need to very precisely specify all the ways not
to do this.
I'm not saying the spec should do this, but that we should accept an
initial spec that does, pending greater precision. That is, it should be a
considered a bug but we don't need to make sure we've squashed all of them
before the initial version.
… My proposal would be that we target writing a spec for implementations
Writing a spec is a monumental task, and one that will hear requirements
from myriad sources, the union of which would make the task intractably
huge to accomplish. It's going to be *necessary* to limit the scope and
requirements in order to successfully complete a specification that
satisfies those requirements. The proposal, here, is that one of the
requirements we *don't* target is "write a precise enough specification
that we're comfortable saying anything following that specification is a
Rust implementation".
There are many other purposes a specification can serve, and the proposal
here is to focus on some of those.
—
Reply to this email directly, view it on GitHub
<#3355 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABGLD26ZTS7KZZJW7XDDMCLWNAL4TANCNFSM6AAAAAASX4UTDU>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
be useful != be a way to decide whether a compiler is a Rust compiler or not. The proposal states:
This would be useful for implementing an alternate compiler. As mentionned earlier, instead of digging through rustc's source code or trying out odd bits of code on the playground, you would get a formal description of rustc's behavior. However, this Rust grammar reference does not make any guarantees regarding whether or not your parser can be called a "Rust parser". Similarly, I don't think this specification should have a goal of being a "validation step" for a Rust compiler. I imagine it's going to be helpful for alternative implementations, but it would also be a much bigger effort to write something that enables deciding whether a compiler can be called a Rust compiler or not. |
This is by and large correct. Please note that qualification is not only the specification work, but also a certification of the organisation producing it. We'd obviously be interested in the work being as easy as possible and work closely with upstream, but that's on a different level. |
|
This mirrors a bit the internal discussion we had when naming the Ferrocene specification. We though about calling it "requirements document" or similar things to avoid the overloaded term "specification". Ultimately, we found that in a programming language, it rather confuses than helps - the requirements document for a compiler is colloquially the "specification" whichever form it takes. I am not aware of any regulatory requirements here. |
|
One reason I would argue against calling it "The Rust Specification" is
because eventually Rust *should* have a proper specification that defines
the requirements on any implementation, and calling this "The Rust
Specification" means that later document cannot be called such.
…On Thu, 22 Jun 2023 at 09:32, Florian Gilcher ***@***.***> wrote:
@m-ou-se <https://github.com/m-ou-se> @tmandry
<https://github.com/tmandry>
This mirrors a bit the internal discussion we had when naming the
Ferrocene specification. We though about calling it "requirements document"
or similar things to avoid the overloaded term "specification". Ultimately,
we found that in a programming language, it rather confuses than helps -
the requirements document for a compiler is colloquially the
"specification" whichever form it takes.
I am not aware of any regulatory requirements here.
—
Reply to this email directly, view it on GitHub
<#3355 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABGLD24C6Q235BHL3OP2KODXMRCP3ANCNFSM6AAAAAASX4UTDU>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
A specification can be broad. It can have multiple parts and even distinct contexts. This RFC is the suggested path for the beginnings of a Rust specification, but that's not to say it is or must be the end either. In my experience, specifications, particularly around programming languages, are constantly being updated with new material. That said, I agree with @m-ou-se , @skade and some other comments, and I think we can move forward with the actual work in producing the content proposed by this RFC and, if required, make a call on the naming in concert with that work. |
Well, more of the point that a Rust Specification that is the comprehensive requierments for any implementation need not be concerned about how rustc specificaly handles repr(Rust), or what have you. That said, I agree that this need not block adopting the RFC, just adding my $.02 on the naming discussion. I might suggest "The Rust Compiler Specification" as an alternative name in the bikeshed war. |
|
If the naming reference, standard, describe rustc bug for bug, .. takes more time than the decision we want to do this. How about moving the naming into a followup RFC? |
|
Is it possible that the core doc be something that other, more formal, specifications reference and refine? For example, if it is called "The Rust Specification", it can be "blessed" independently as "The ISO Rust Specification", "The NHTSA Rust Specification", "The FAA Rust Specification", etc. They can be independently versioned as well (e.g., the FAA says that Rust 1.100 is fine and it is all that is allowed until the next time the compiler is "vetted"…very unlikely to follow the 6 week release cycle for sure). These may have additional restrictions/rules (e.g., "no global initializers" à la rust-lang/rust#111921) in how Rust is "allowed" within their realms that may not make sense in the document being created here. If that is a plausible usage, I think "specification" would be fine here. Just because it gets called a specification doesn't mean it is automatically blessed for usage in whatever contexts care about things being Specifications. |
I would expect this specification to define requirements that any implementation must meet. It isn't one of the goals of the initial document to define sufficient conditions for something to be an implementation of "Rust", but I'd still expect it to define necessary conditions. |
It's going to document (at least some) implementation details of rustc, though, is it not? |
|
Mara's latest unresolved question resolves my concern. Thanks! @rfcbot resolve naming |
rfcbot
added
final-comment-period
|
🔔 This is now entering its final comment period, as per the review above. 🔔 |
|
This FCP is for approval from the language team. However, as mentioned before, this does still need approval from the council. I will add this to the agenda of the council. |
|
Doing this manually, because I'm not sure if rfcbot works yet for the council, and also because we want all checkboxes checked rather than just N-2: @rust-lang/leadership-council: This RFC was approved by the language team and is currently in FCP. However, this should get full council approval, in part because this involves an official request from the project to the Foundation to hire someone. (This has already been discussed in detail with the Foundation, see #3355 (comment), #3355 (comment), and #3355 (comment).) Approval from the council: |
|
Very excited to see progress on a Rust Specification! |
|
Happy to approve hiring someone. I should avoid questions of precisely what that hired person should do, because my company wrote the Ferrocene Language Specification. |
|
The Leadership Council has given all ✅. Excited for this! I'd like to just give a +1 to the concern that Niko brought up around naming that is now reflected in the RFC. I think we'll need another decision on what this document actually ends up being called, because that might have implications on the future of Rust evolution. |
rfcbot
added
finished-final-comment-period
|
The final comment period, with a disposition to merge, as per the review above, is now complete. As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed. This will be merged soon. |
|
This RFC has been merged! To track further discussion, subscribe to the tracking issue here: rust-lang/rust#113527 |
m-ou-se
removed
the
I-lang-nominated
Rendered