RFC: Require documentation for all new features. #1636

Merged
merged 23 commits into from Dec 2, 2016

Projects

None yet
@chriskrycho
Contributor
chriskrycho commented Jun 4, 2016 edited

Rendered

Note: I used #[deprecated] as an example in this... and I submitted a PR to update it right after I submitted this PR.

@chriskrycho chriskrycho changed the title from Document all features. to RFC: Require documentation for all new features. Jun 4, 2016
@dikaiosune

This might be out of scope for this particular RFC, but since you mentioned Ember... Just thought it was worth mentioning that a part of the feature documentation process for them appears to be a "How we teach this" section of RFCs:

https://github.com/dgeb/rfcs/blob/module-unification/text/0000-module-unification.md#how-we-teach-this

Seems to me like a great way to have documentation (and thus user onboarding) in mind as part of the RFC process.

@chriskrycho
Contributor

@dikaiosune: excellent link (I was trying to find it earlier, and @steveklabnik mentioned it elsewhere as well), and I think that actually should be included, probably as part of the Detailed Process section. I'll add it to the writeup sometime tomorrow.

@dikaiosune

I think that'd be great -- especially because (IMO) part of having good documentation is making sure that features can be sanely documented and taught. I see it come up in RFC discussions, but it'd be nice to formalize as part of the process before an implementation happens (again, IMO).

@acorncom
acorncom commented Jun 4, 2016

I'm on the Ember Documentation / Learning team and would highly recommend this. Once the documentation stopped falling further and further behind, there was actually some motivation to improve things. πŸ˜„ We're still catching up on places, but at this point, about a year later, things are finally in reasonable shape.

We'd be happy to discuss things further if it'd be useful ...

@jugglerchris

As a user not (so far) involved in the core, this gets an enthusiastic vote from me. As an example the for<'a> construct is neither discoverable in the reference, nor Googlable if you come across an example of it.

I think an appendix in the reference with links to accepted RFCs not yet incorporated into the documentation would be a move in the right direction.

@chriskrycho
Contributor

I've updated the RFC slightly to account for those pieces of feedback:

  • I added a call to add the Ember-inspired "How do we document this?" section.
  • I added a "How do we document this?" section to this RFC. That includes some thoughts on how to make sure this is a community effort, not just piling more work on @steveklabnik.
  • I added some thoughts on how a community subteam might be useful.
  • I added an addendum discussing the possible necessity of a "strike team" to get the current reference up to date as well.

@acorncom, I'd be particularly interested to hear what the Ember team has found most effective in helping get the community to step in and work on the docsβ€”it's one of those things that people hear a lot ("Contribute to the docs!"), but I imagine it's hard to actually get people engaged that way?

@chriskrycho
Contributor

@jugglerchris, I also just went back and included your suggestion of an appendix for implemented-but-not-yet-documented RFCs, as I quite like that as a useful intermediate step.

@nagisa nagisa and 1 other commented on an outdated diff Jun 4, 2016
text/0000-document_all_features.md
+
+Updating the reference could proceed stepwise:
+
+1. Begin by adding an appendix in the reference with links to all accepted RFCs which have been implemented but are not yet referenced in the documentation.
+2. As the reference material is written for each of those RFC features, it can be removed from that appendix.
+
+
+### Standard library
+[std]: #standard-library
+
+In the case of the standard library, this could conceivably be managed by setting the `#[forbid(missing_docs)]` attribute on the library roots. In lieu of that, manual code review and general discipline should continue to serve. However, if automated tools *can* be employed here, they should.
+
+
+# How do we teach this?
+
+Since this RFC promotes including this section, it includes it itself. (RFCs, unlike Rust `struct` or `enum` types, may be freely self-referential. No boxing required.)
@nagisa
nagisa Jun 4, 2016 Contributor

I guess changing the template 0000-template.md or whatever that’s called should be part of this PR.

@chriskrycho
chriskrycho Jun 4, 2016 Contributor

I plan to submit that in an "implementation" PR if this is accepted (and discuss it below).

@dikaiosune dikaiosune and 1 other commented on an outdated diff Jun 4, 2016
text/0000-document_all_features.md
+### Standard library
+[std]: #standard-library
+
+In the case of the standard library, this could conceivably be managed by setting the `#[forbid(missing_docs)]` attribute on the library roots. In lieu of that, manual code review and general discipline should continue to serve. However, if automated tools *can* be employed here, they should.
+
+
+# How do we teach this?
+
+Since this RFC promotes including this section, it includes it itself. (RFCs, unlike Rust `struct` or `enum` types, may be freely self-referential. No boxing required.)
+
+To be most effective, this will involve some changes both at a process and core-team level, and at a community level.
+
+From the process and core team side of things:
+
+1. The RFC template should be updated to include the new section for teaching.
+2. The RFC process description in the [RFCs README], specifically by including "fail to include a plan for documenting the feature" in the list of possible problems in "Submit a pull request step" in [What the process is].
@dikaiosune
dikaiosune Jun 4, 2016

I don't think this sentence has a main verb. Should it instead be "The RFC process description in the [RFCs README] should be updated to include "fail to include a plan for documenting the feature" in the list of possible problems in "Submit a pull request step" in [What the process is]." ?

@chriskrycho
chriskrycho Jun 4, 2016 Contributor

Indeed, something to that effectβ€”I'll update it shortly. Thanks! (English: how does it even work.)

@kylone
kylone commented Jun 4, 2016

It may be worthwhile to create new RFCs to update documentation for existing language features that aren't documented well in the reference, so that the process can be formalized. (Assuming hat this RFC is accepted.)

@nagisa
Contributor
nagisa commented Jun 4, 2016 edited

It may be worthwhile to create new RFCs… for existing language features

There’s nothing worthwhile to discuss about existing features other than removing (EDIT: or changing) them on the RFC scale. That should be done in forums, IRC and other less involving channels.

@chriskrycho
Contributor

@nagisa, that's true in the "RFC" sense but not the general sense: there's plenty to discuss in terms of documenting a lot of existing language features (they're certainly not all documented now; the lack is part of what prompted this RFC). @kylone, we'd want to create issues to track those rather than RFCs. See When you need to follow this process in the RFCs README.

@withoutboats
withoutboats commented Jun 4, 2016 edited

I worked abortively on an RFC along the same lines a few weeks ago, but this is much more comprehensive. Well done! 🎈🎈

I'm not sure what should be done about the reference, though. The main way language features are taught is through the book, not the reference - but the documentation in the book is more laborious than the documentation in the reference, and it might not be reasonable to gate stabilization on giving the feature a proper "book" treatment. I suppose that requiring a "reference" treatment of every feature will finally give the reference a real purpose and some attention (I personally use the reference almost exclusively for ransacking reserved words to paint a bikeshed with).

@chriskrycho
Contributor

@withoutboats: thanks! I think you're right that it should not require putting it in the book, but should require documentation in general. That's part of what I was thinking in the "How do we teach this?" section discussion: we can specify when writing new RFCs where it makes the most sense to put the new informationβ€”and that can be either or both of the reference and The Rust Programming Language, and optionally also Rust by Example (which I found enormously helpful as an introduction to the language for myself).

@azerupi
azerupi commented Jun 5, 2016

it might not be reasonable to gate stabilization on giving the feature a proper "book" treatment.

The book is the most used and most comprehensive source of information about Rust. If I need to find an explanation about something, I expect to find it in the book. Depending on the feature, it could need just a paragraph or it could need a whole chapter. Considering that a new feature will take at the very least 6 weeks to go stable and knowing that bigger features will also have longer testing phases, I don't think it's unreasonable to ask for documentation in the book.

If we don't require the book to be up to date before / while stabilizing, this means the work is mostly relegated to Steve Klabnik. The time he has to spend keeping up with new language features is time he can't spend improving the other parts of the book.

@chriskrycho
Contributor

@azerupi, FWIW, my take is that we should document a given feature wherever is appropriate for a given feature. For example, I don't think we need a list of every #[attribute] and its applicability in The Rust Programming Language, but we do need a description of attributes and how they work in general there, and we do need that exhaustive list, presumably in the reference.

I can further clarify the RFC if it's missing that emphasis; I thought it was clear that part of the RFC process, with this revision, would entail discussing where and how to document any given feature. That's what I assume would be included in "How do we teach this?" and would be discussed in the RFC along the way.

The last thing I want to do is land yet more work on @steveklabnik's plate; he has plenty as it is. But requiring that everything land in the book, or not, seems to me to be orthogonal to whether Steve is the one writing all of it. (Other people can update the book; Steve can update the reference.) Do you see a spot in the RFC where I should clarify the wording to make that clearer?

@azerupi
azerupi commented Jun 5, 2016

@chriskrycho No it's good :)

My response was more directed at @withoutboats because his proposition seemed to imply that no feature should be blocked for not being documented in the book because it could take some time to write about it.

It does indeed not make sense to document all the features in the book, but a lot of them can and should be. When saying this I am more thinking about features like specialization. I would find it normal to require a section in the book about that. Blocking stabilization while documentation is lacking will make sure that the documentation gets the treatment it deserves. Even features like #[deprecated] are good to mention in the book, even if it's only in one sentence with a link to somewhere else. Because people going through the book will at least know that it exists.

@acorncom
acorncom commented Jun 6, 2016 edited

@chriskrycho Re: your question:

I'd be particularly interested to hear what the Ember team has found most effective in helping get the community to step in and work on the docsβ€”it's one of those things that people hear a lot ("Contribute to the docs!"), but I imagine it's hard to actually get people engaged that way?

As far as getting folks to contribute, we've got contributions on three levels roughly:

  • minor tweaks to fix tiny bugs in the guides (spelling mistakes, logic issues, etc)
  • decent chunks of work (editing a section of the site, hitting one topic issue that spreads across the guides)
  • major pieces of work that involve coordinated work across a period of time

For the minor tweaks, adding our "Edit this page" icon meant that the small bugs are something we don't spend much time thinking about (people tell us if they find something unclear). It has turned each user of the docs into a beta tester who might send in feedback. We're going to need to add that to the API docs as well (which are still a bit outdated) once we have made some infrastructure changes.

For decent chunks of work, having labeled issues in the guides repo with the work laid out clearly seems to have led to people contributing the most. If people ask how they can help, we point them there. With these types of issues, we tend toward a facilitating role ("let us know if you have questions as you dig in") Although I have to be a bit careful with it (I've probably said it to you too much @chriskrycho πŸ˜‰ ) the "hey, send in a PR πŸ˜„ " comment has also led to contributions in this vein. Having it stated clearly (and regularly in different ways) that keeping the documentation up-to-date is a major undertaking and we'd love to get help has led to a number of different major contributions that we didn't have the bandwidth for.

For the really large pieces of work that require on-going discussion (and background on how we're wanting the guides to work), we tend to reserve those for the learning team to work on. Not all our issues are labeled "help wanted" and we keep some larger items on a private Trello board where we track and discuss things, as they would be very hard for outside contributors to help with. Having meetings weekly (or as close as possible to that) has also helped us work together as a team (as we regularly discuss what our top priorities should be and what things need to be put on hold due to time constraints).

For Rust, what I'm hearing as part of this discussion is that Steve Klabnick is the main guy for documentation. If that's the case (it's where Ember was a year+ ago), I would suggest putting out a "call for help" to see if you can find a few more folks who are willing to join the docs team. It was only once there were multiple people working on the Ember docs (in addition to the core team's commitment to not add new code without documentation) that they turned around. Without a team approach, keeping documentation up-to-date can turn into a major chore ...

Does that help? Would love to chat about it here (or offline) further if it's useful ...

@steveklabnik steveklabnik and 1 other commented on an outdated diff Jun 6, 2016
text/0000-document_all_features.md
+5. Information on how the features will be used is often closely coupled to information on how the features will be implemented, both in the RFCs and in the discussion threads. Again, this is as it should be, but it makes it difficult (at best!) for ordinary Rust users to read.
+
+In short, RFCs are a poor source of information about language features for the ordinary Rust user. Rust users should not need to be troubled with details of how the language is implemented works simply to learn how pieces of it work. Nor should they need to dig through tens (much less hundreds) of comments to determine what the final form of the feature is.
+
+## Precedent
+[precedent]: #precedent
+
+This exact idea has been adopted by the Ember community after their somewhat bumpy transitions at the end of their 1.x cycle and leading into their 2.x transition. As one commenter there [put it][@davidgoli]:
+
+> The fact that 1.13 was released without updated guides is really discouraging to me as an Ember adopter. It may be much faster, the features may be much cooler, but to me, they don't exist unless I can learn how to use them from documentation. Documentation IS feature work. ([@davidgoli])
+
+[@davidgoli]: https://github.com/emberjs/rfcs/pull/56#issuecomment-114635962
+
+The Ember core team agreed, and embraced the principle outlined in [this comment][guarav0]:
+
+> No version shall be released until guides and versioned API documentation is ready. This will allow newcomers the ability to understand the latest release. ([@guarav0])
@steveklabnik
steveklabnik Jun 6, 2016 Contributor

this link is broken

@chriskrycho
chriskrycho Jun 6, 2016 Contributor

Fixed! Thanks. πŸ‘

@steveklabnik
Contributor

Hi @chriskrycho ! Thanks for this RFC. And thanks to everyone who's commented so far.

I have... complicated feels about this. I do love, love, love the "how will we teach this" part. Regardless of the rest of this RFC, I would like to see that proposed.

I should also mention that we have been working on spinning up a docs team. We have weekly meetings, and we're sort of in the final stages of making it official. If you'd like to help, please come by! The next meeting will be on Wednesday 8th of June 2016 at 20:00 GMT on #rust-docs channel in irc.mozilla.org.

In general, I am pro "it's not a real feature until it's documented" as a concept, but am extremely worried about becoming even more overburdened than I am already. This RFC does have some suggestions, and maybe with it as a forcing function, more will be interested in helping out with docs. I dunno.

I am torn about the reference. For some fun/sad history here: rust-lang/rust#20137

At various times, I have suggested that maybe the book should get more reference-y, and subsume it entirely. That said, I think this is probably the wrong choice; the book shouldn't explain every last detail of everything. But it would be nice if the reference saw more love; as you've said, it's accurate, but not complete. Most of my time writing docs these days is being spent on the second edition of the book. It was going a bit slow for a while, but has been going faster lately; it's going to be great, but it also takes time. The weird thing about it is that the beginning is the hardest part; I took way longer on the first chapters than I expect to on the later ones, because the later ones mean you already know most of Rust! So it's a bit easier.

There's also been a discussion over the years of "how much like a spec should the reference be?" Which is an interesting and tough question. In general, I see the layout of the docs like this:

  • Book: "How do I learn Rust?"
  • API docs: "I know Rust, but I want to know how to use the standard library"
  • Reference: "I know Rust, but I want to learn about a particular thing in great detail."

In other words, I see the reference as being just as big of a job as the book, but with a different focus/voice. In some ways, it's a bigger job, as it should have more details in it! At the same time, it doesn't need tons of examples, etc. So it might be literally smaller, text-wise.

Anyway, I think that's what's been in my brain over the weekend. In general, regardless of this RFC, if anyone wants to get more involved with docs, we'd love to have your help. Please drop by IRC and ping me.

@llogiq
Contributor
llogiq commented Jun 7, 2016

I was a bit surprised to see my deprecation RFC as the bad example, considering it explicitly states that the language reference is to be updated. Sadly, this particular change has been lost in tracking.

@chriskrycho
Contributor

@llogiq, interesting, I didn't realize that.

  1. Good call stating that it needed to be in the docs update.
  2. I think that highlights that we need something like more generally.
  3. I think it also highlights the fact that even having a "requirement" per the RFC will only go as far as the actual shift in community and core team practice goes.
@chriskrycho
Contributor

@steveklabnik, I made a number of tweaks to the RFC, taking onboard some of the considerations you outlined above as well as some of those from our conversation in IRC.

One of the things your comments and @azerupi's have me thinking about (and which probably warrant at least follow-up discussion if this were accepted; perhaps they should also go in the Detailed Design section?) is what goes where. My current sense is:

  • Every language item and feature should be covered in the reference. A great many probably only need a sentence or two.
  • All major language features should be included in The Rust Programming Language. Whether a feature is "major" should be discussed in the How do we teach this? section.
  • All public API items should continue to be documented.
  • Only the highlights should appear in Rust by Example. I think the balance between the two books today is about right. (More links to the main book in Rust by Example might be good? I'll think on that.)

On each of those bits of documentation and the core team member's (Steve's) role relative to the docs team, fully granted that I don't know what you all have in mind (but I'm planning on observing the meeting tomorrow on IRC):

  • Perhaps different team members can take the lead on different parts of the documentation?
  • Perhaps those core team members can also take the lead on recruiting help, in line with some of @acorncom's comments above?

Last but not least: I definitely see the same problem you do with the reference. I think that either a "strike team" approach or a continuous development with the appendix (as suggested above) could work to help resolve that. I'm mulling over some other thoughts on that which I may integrate at a later point as well.

@llogiq
Contributor
llogiq commented Jun 8, 2016

I think having a checklist on tracking issues that includes the necessary doc updates will go a long way. The deprecation issue was, in a way, closed prematurely.

@chriskrycho
Contributor
@jan-hudec jan-hudec commented on an outdated diff Jun 13, 2016
text/0000-document_all_features.md
+5. Specific to the suggestion to [**Add an "edit" link**][edit-link]:
+
+ - If the specific page is in flux (e.g. being rewritten, broken into pieces, etc.), then a link to edit `master` will be confusing.
+ - In addition, when users *have* made edits, it may take some time before it appears, and thus users may be confused when attempting to make edits and finding that the relevant editss have already been made.
+ - Some pages users attempt to edit are *likely* to have different documentation in them than the existing pages, to account for inbound changes for feature additions to the language!
+
+ Two notes, however:
+
+ 1. Even facing the same issues, the Ember team has found it useful to have the link, as it enables basically any user of a sufficient comfort level with GitHub to fix basic typos or logic errors.
+ 2. This concern primarily impacts _The Rust Programming Language_. Both in its current state and in the event of an eventual revamp (at least: after such a revamp finished), the Rust Reference is far less likely to see pages removed or moved.
+
+ Finally, while infrastructure changes could be made in support of a more "targeted" editing experience, doing so would substantially increase the triage work required for the docs. It would also entail extra work "porting" the changes back to `master`. Additionally, because the language itself does not currently "bugfix" releases, this would substantially alter the workflow for dealing with releases in general.
+
+6. Specific to the suggestion to [**Visually Distinguish Nightly**][distinguish-nightly]:
+
+ This requires at least some infrastructure investment. Making the change apply to the Reference as well as to the two books would entail the maintenance of further CSS. This might be acceptable if documentation teams are sufficiently motivated and engaged, but it means that if not very carefully designed up front, any changes to the documentation theme will basically require double CSS changes; they will also require double the *design* efforts.
@jan-hudec
jan-hudec Jun 13, 2016 edited

Perhaps this could be handled by just adding a style-sheet. Stylesheets combine just fine and additional stylesheet defining a watermark background image and perhaps a couple of icons with :before and :after would be plenty conspicuous with just a few rules and shouldn't need too much maintenance.

@nrc nrc added the T-core label Jul 7, 2016
@chriskrycho
Contributor

I've updated the outline of this a little bit (and added an outline to it!) and clarified wording in a few spots after some helpful feedback from @jeremy-w elsewhere.

@jeremy-w
jeremy-w commented Aug 9, 2016

@chriskrycho Links in outline are not working. Looks like in the rendered document, all the header ids are prefixed with user-content- vs what the outline expects to find.

@chriskrycho
Contributor

@jeremy-w – hmm, maybe a browser extension you're running? I can't reproduce (tried Safari 10, Opera, Chrome Canary).

@chriskrycho
Contributor

Addendum: looks like it's a GitHub mobile thing or a mobile Safari thingβ€”not sure which. I'll try to fix that later this week if I can.

@aturon
Contributor
aturon commented Aug 31, 2016

Ack, this seems to've fallen off the radar a bit! @rust-lang/core, can everyone please take a look at this? It'd be good to move to FCP soon.

@llogiq
Contributor
llogiq commented Aug 31, 2016

Meanwhile @chriskrycho has agreed to write docs for the #1623 implementation. πŸ‘

@aturon
Contributor
aturon commented Sep 12, 2016

Sorry, this got lost in the RustConf shuffle. Let's try out the fancy new RFC bot!

@rfcbot fcp merge

@rfcbot
rfcbot commented Sep 12, 2016

FCP proposed with disposition to merge. Review requested from:

No concerns currently listed.

@aturon
Contributor
aturon commented Sep 12, 2016

Woo, looks like the bot worked, but it doesn't know what the core team is πŸ˜†

@chriskrycho
Contributor
@aturon
Contributor
aturon commented Sep 14, 2016

Let's try it again!

@rfcbot: fcp merge

@rfcbot
rfcbot commented Sep 14, 2016 edited

Team member @aturon has proposed to merge this. The next step is review by the rest of the tagged teams:

Concerns:

Once these reviewers reach consensus, this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

@aturon
Contributor
aturon commented Sep 14, 2016

I'm broadly in favor of this RFC. In my mind, there are two core ideas here:

  • Require teaching to be part of the RFC discussion. I cannot πŸ‘ this enough. It keeps a strong explicit focus on learnability during design (which often coincides with usability), which neatly fits into our 2017 planning discussion as well.
  • Gate stabilization on some form of docs. We know from experience that anything we don't gate on will regress -- that applies to docs as much as to engineering artifacts. I strongly agree with the spirit of the RFC, that docs are every bit as important as code, and that if it's not documented, it doesn't exist.

I'm going to file a separate comment with a formal concern, however, about the precise details.

@aturon
Contributor
aturon commented Sep 14, 2016

@rfcbot concern The state of the books

I'm a little worried about a couple aspects of the detailed design for lang features:

  • It cements three separate books as requirements for documentation, which feels like a very ambitious starting point. I hadn't quite realized that Rust by Example had reached this stature -- though, it's also of the three the easiest to update, I suspect.
  • Both the main book and the reference are in problematic shape. For the book, we're in the middle of a re-write, and I'm not sure how this RFC should interact with that. For the reference, as the RFC itself mentions, things are already quite out of date. I want to have very concrete plans about how the requirements of this RFC are going to be fulfilled in practice.
  • Of the books mentioned, the main book is the one that gets the most care in terms of having a steady voice, consistent style and approach. The new book, in particular, is being very carefully crafted. I worry that it won't work well to have major contributions not coming from its original authors -- that it will lose its voice and careful construction. The other resources, on the other hand, already have a more "crowd-sourced" flavor.

I'm not so worried about the gating pushing back features. Small features are quick to document. Large features require a long stabilization period anyway -- this RFC just ensures that we work out the logistics to document them prior to stabilization. In other words, I think the basic idea is sound, I just want to get greater clarity on the precise mechanics given the current state of play. And @steveklabnik, I'm particularly interested to hear your take on the "voice" of the book and related issues.

@aturon aturon referenced this pull request in dikaiosune/rust-dashboard Sep 14, 2016
Closed

Team nagboard #61

@chriskrycho
Contributor
  • FWIW, I'd be perfectly happy to drop the Rust by Example materials from the text of the RFC. I included it primarily for completeness.
  • I agree about the other concerns, and have a few thoughts on the reference process in particularβ€”but the "voice" issue on the book proper is tough. That one might fall to @steveklabnik and @carols10cents to "ride herd" on at a minimum.
@steveklabnik
Contributor
steveklabnik commented Sep 14, 2016 edited

And @steveklabnik, I'm particularly interested to hear your take on the "voice" of the book and related issues.

Even though we joke that "Steve doesn't scale", and part of this whole thing is about making sure that everyone chips in with docs, I assume that changes to the book will be basically gated on me/@carols10cents for this reason. That being said, someone coming up with a lot of the text that I can then work from is still very helpful. I'm actually more worried about the decisions about what goes in the book vs not: the book isn't truly comprehensive, but the reference is. Maybe we should figure out how to word this better, like, the reference is required, and a decision about the book/RBE should be made, if not actually committed.

Related,

I hadn't quite realized that Rust by Example had reached this stature

I read this as a way to get people to care about RBE at all; a lot of people love it, but it's one of the most neglected Rust doc projects. I try to keep on top of PRs, and we have some great people doing great work on it, but not nearly as much as everything else. It could use a lot of work overall, it's kinda closer to the reference than the book or API docs in this sense.

@ticki
Contributor
ticki commented Sep 14, 2016

When people ask questions over at IRC, I've found referencing to RBE incredibly useful. It is faster for those who understand the concept, but needs to Rust angle on it (syntax, library, etc.). Heck, it even contains stuff which is barely documented otherwhere.

@jimmycuadra

Re: voicing in writing. I've learned from my partner @leahscampoli, who is a technical writer, that keeping a consistent voice even when multiple people are working on a piece of writing is simply one of the skills that a technical writer must develop. For example, at Salesforce, they have some particular guidelines on how to keep tone and voicing consistent across their documentation, and the writers use this to help shape the tone they use when writing. To that end, perhaps doing a bit of analysis to figure out what exactly makes the book sound like the book could produce a set of guidelines that people could use, both for guiding their writing while working on the book (or other docs), and for docs reviewers to use to look for content in docs PRs that sound at odds with this consistent voice.

@brson
Contributor
brson commented Sep 14, 2016 edited

Let's see how this works...

@rfcbot concern Unrelated changes to documentation infrastructure.

Both "add an 'edit' link to the documentation pages" and "visually distinguish nightly vs. stable build docs" seem like wishlist items unrelated to the thrust of the RFC. The RFC says we "might" do these things. I suggest removing them.

There's a whole section "add an 'edit' link". This section instead of saying "might" says "should". Again, I think this is simply a feature request and largely unrelated to this RFC. Likewise the "support with infrastructure change" and "visually distinguish nightly" sections.

@rfcbot concern Unclear language about change reviews

"Changes will now be reviewed for changes to the documentation prior to being merged". This is an awkward sentence and unclear. Of course "changes" are reviewed for "changes". Probably this should be something like "prior to approving pull requests that stabilize features reviewers must verify that the feature is properly documented in the following places".

@rfcbot concern Write a blog post

This RFC demands that a blog post be written about the RFC: "Write a blog post discussing the new process should be written discussing why we are making this change to the process, and especially explaining both the current problems and the benefits of making the change". Seems nice, but I wouldn't require it as criteria for accepting the RFC.

@rfcbot concern Strikeouts in "how do we teach this" section

There are two sentences with strikouts. A final RFC doesn't need to show these edits from the in-process RFC.

@rfcbot concern Incorrect mentions of core team

There are many references to "the core team" in this document that are overly precise, referring to functions that are performed by more than the core team. In many cases "the Rust team" would be more appropriate.

  • "When the core team discusses whether to stabilize a feature". Not the core team's responsibility. Individual subteams.
  • "From the process and core team side of things". A few of the bullets here may be the responsibility of the core team, but it's not obvious. Updating the RFC template and description certainly has to be approved by the core team, but others can do it. Writing a blog post about process changes would be best done by the core team. This sentence is kinda casual and could be reworded to avoid the problem.
  • "Core team members are already very good at this". Lots of people are involved in the RFC process and good at this.
  • "under the leadership of the relevant core team representative". Not particularly relevant to mention the core team here.
  • "This is also an opportunity to allow/enable non-core-team members ... to contribute". non-core team members can and do contribute to all aspects of documentation today.
  • "rather than it falling entirely on the shoulders of a single core team member". I think this whole paragraph is superfluous really. It's just saying "it would be great if other people felt responsible for docs".

It's good to not invoke the core team when possible since some people have a strong negative reaction to appeals to authority.

@rfcbot concern Release notes

This mentions release notes in passing. It would be great if you are not allowed to stabilize a feature without writing a line for the changelog and a section for the long-form release announcement. Doing this after the fact is tedious and results in poor quality.

@rfcbot concern Mandating changes to the book

This is fairly unclear about the requirements for updating the book. It says "We will document all language features in the Rust Reference, as well as making some updates to The Rust Programming Language and Rust by Example as necessary". It's not clear though when updating the book and rbe is "necessary".

I'm reluctant to mandate updates to the book. The book is a cohesive work with an editorial vision, and it is not appropriate to cram everything about Rust into it. That is what the reference is for. On the other hand, I don't see much downside to mandating updates to RBE.

Needs to be clarified what the criteria are for making these decisions.

@rfcbot concern Validation of feature coverage and cross-referencing

Besides suggesting that the reference contain an appendix containing links to uncovered RFCs this doesn't talk much about the mechanics of ensuring feature coverage. If we're going to to put policy in place demanding such coverage then I would like it to be enforced automatically. It's easy to imagine a system that validates that every stable feature (which we track) is mentioned in the reference, and fails the test suite if not. And while we're adding a validator we might as well validate that those features are cross-referenced, in the manual, with their RFC. It may be reasonable to leave this to future work.

@rfcbot concern State of the reference

The RFC acknowledges that the reference is in a bad state and suggests how we "might" fix that while also acknowledging that "it is probably a necessity". This section is all "might"s and "could"s - basically has no effect on the RFC. I'm not sure if it should be removed or strengthened.

@brson
Contributor
brson commented Sep 14, 2016

OK, I should have said some good things as well. Sorry, I was very focused on rfcbotting concerns. This is a useful RFC and I do think we should require documentation for all features. Thank you for writing it @chriskrycho.

@aturon
Contributor
aturon commented Sep 14, 2016

cc @rust-lang/docs -- while this is a core team issue (affecting the basic RFC process), the docs team angle is definitely important here as well!

@GuillaumeGomez
Member

I totally approve this PR! The time where a feature comes out and the time where it gets documentation are just too far away from each other.

@chriskrycho
Contributor
chriskrycho commented Sep 15, 2016 edited

I'll make a bunch of changes for those points tomorrow, @brson; all totally fair.

… I do really appreciate the follow-up note, because that first response had me feeling a mite hammered. πŸ”¨ πŸ˜€

@withoutboats

It looks like rfcbot only reads 1 concern per message.

@carols10cents
Member
carols10cents commented Sep 15, 2016 edited

General thoughts

First off, I want to say that this RFC is incredible. I love the subject matter, I think this idea is important, and it was quite enjoyable to read. My favorite part was the self-referential bit ;)

I think a lot of @brson's concerns boil down to going through this RFC again keeping IETF RFC 2119 in mind (that is another of my favorite RFCs!). Especially since there's now been a decent amount of discussion about what should be recommended or optional versus what should be required.

This mentions release notes in passing. It would be great if you are not allowed to stabilize a feature without writing a line for the changelog and a section for the long-form release announcement. Doing this after the fact is tedious and results in poor quality.

πŸ’― Release notes SHOULD be in the MUST category!

Book specific thoughts

I've got conflicting feelings on this too. The current focus on the new book is with an immutable print version in mind, while this RFC embraces the easily-changeable nature of the web version. The print version has a much higher bar for a cohesive narrative and consistent voice.

Idea: what if, post-print or without intent to incorporate into the printed version, we add to the book a chapter or appendix for sections on "newest features"? I'd be totally fine with anyone contributing docs of any style to be smaller vignettes on various features in this chapter or appendix. This way, we at least have some documentation available in with the book where people would look for it, but we wouldn't be gating on @steveklabnik or I or whoever else to ensure narrative/voice. As features stabilize, become more widely used, and integrate more tightly with the language as a whole, we could work this content into the main book.

But again, I don't really want to do much of that while we're working on the print version, and I don't think a book section should be required for every RFC.

Also, we do have an issue in the new book repo regarding edit links that I am in favor of, but that we have to put some thought into regarding the trains, as you addressed in the RFC, and again that @steveklabnik and I are unlikely to work on until the print book is done.

@cbreeden

In the language features section:

New features do not need to be documented to be merged into master/nightly, and in many cases should not, since the features may change substantially before landing on stable, at which point the reference material would need to be rewritten.

Instead, the documentation process should immediately precede the move to stabilize. Once the feature has been deemed ready for stabilization, either the author or a community volunteer should write the reference material for the feature.

I was wondering if we could clarify this? Some of us were talking on IRC about what this exactly means, we we came to believe that it means:

  1. Final decision to stabilize feature (decision should be independent of documentation)
  2. Add proper documentation
  3. Merge into rust stable.

This would mean that a tracking issue would remain open even after a final decision to stabilize. Is this a correct interpretation @chriskrycho?

@chriskrycho
Contributor

All, a quick note: I'm seeing and reading carefully every comment left here, but am very busy right now. I'll respond, make edits, etc. to all of these (very helpful) comments over the weekend, probably on Sunday.

@chriskrycho
Contributor
chriskrycho commented Sep 21, 2016 edited

One note, to @brson's otherwise very much appreciated set of concerns:

There's a whole section "add an 'edit' link". This section instead of saying "might" says "should". Again, I think this is simply a feature request and largely unrelated to this RFC.

Although I'm fine with removing this from the RFC, I actually think these kinds of infrastructure-level changes are a necessary set of associated changes to make this RFC work in practice. I didn't include this as a "wouldn't it be nice?" kind of thing (though I also didn't distinguish it well enough from the other sections which are in that category).

To put it another way: I think that making the change is, even if not necessary for this RFC in principle, absolutely necessary for its implementation.

(In that sense this is something of a weird RFCβ€”it's not a change to the language, it's a change to the process. And tools are part of the process.)

If it gets this across the finish line and we all recognize that these kinds of infrastructural changes are important in making this change, I'm totally good with cutting that piece. I just want it noted that I really do think it's that important to make some changes (even if not that one) to make it easier to engage on this front.

@chriskrycho
Contributor

All right, I've made a start on addressing concerns. I'd love to see further feedback and discussion in the meantime; I'll come back to this again later in the week (Thursday or Friday evenings, perhaps). I have a bunch of other writing to do yet this evening, and need to go get working on that.

@nikomatsakis
Contributor

@carols10cents

Idea: what if, post-print or without intent to incorporate into the printed version, we add to the book a chapter or appendix for sections on "newest features"?

This is a great idea!

text/0000-document_all_features.md
+- Given that the reference is out of date, does it need to be brought up to date before beginning enforcement of this policy?
+- Given that the book is in the process of a rewrite for print publication, how shall we apply this requirement to RFCs merged during its development?
+- For the standard library, once it migrates to a crates structure, should it simply include the `#[forbid(missing_docs)]` attribute on all crates to set this as a build error?
+- Is a documentation subteam, _a la_ the one used by Ember, worth creating?
@carols10cents
carols10cents Sep 21, 2016 Member

I believe this question can be removed from the unresolved questions list since we have a documentation subteam now :)

@chriskrycho
Contributor
text/0000-document_all_features.md
+ - in _The Rust Programming Language_
+ - in _Rust by Example_
+- the standard library: in the `std` API docs
+
@cbreeden
cbreeden Sep 23, 2016 edited

Could we also add the originating RFC to this list of things that need to be updated? Take for example the closure RFC. This RFC references things that doesn't seem to have stabilized like FnShare and receiver specifiers (which to be fair were removed from a different RFC).

Edit: I felt that it might be worth discussing the purpose of RFCs before making such a decision here.

@cbreeden

I thought it might be worth while taking a moment to clarify what purpose an RFC has after a feature has been stabilized (but before the feature is released). I questioned if it would be worth requiring:

  • RFCs must be updated prior to stabilization to ensure accuracy and completeness
  • RFCs must not have unresolved questions prior to stabilization

Perhaps this topic is not within the scope of this RFC, but depending on whether RFCs should essentially be kept for historical purposes or as something that can be "official referenced" may impact how the suggestions in this RFC are executed.

For instance, RFCs should contain motivation and numerous examples illustrating the problem that the RFC is proposing to solve. While discussing the scope of "The Reference" it seemed that motivation and illustrating examples, while they should be present, should probably be limited in a minimal way to what is required to describe a feature in some complete manner. Provided that RFCs are still accurate and complete, the Reference can direct readers to an RFC for further information and motivational examples. The alternative is to place this motivation and examples in RustByExample.com or The Book. While this may be fine, I just also feel that some motivational examples details may conflict with the target audience at times.

I started a discussion along these lines in the internals forum here, which in hindsight was probably not a good idea. Thoughts? Maybe do nothing and just see how it looks after awhile and see if we need to tweak it?

@chriskrycho
Contributor

@cbreeden, I think that's a hugely important point, actually. I'd love to hear feedback from others before attempting to integrate any changes (and I have several other changes I still need to integrate! Hopefully tomorrow!), but my own sense is that the RFC should probably have a section added if there are differences between what it proposed and what was finally stabilized. And I think you're quite right that it affects how this RFC is carried out if accepted.

@steveklabnik
Contributor

Historically, RFCs are more of a "this is the decision at the time it was made" rather than a "should be updated to be current." The thinking was that the Reference should be the go-to explanation for anything in any detail, and the RFCs were there to capture the history.

@chriskrycho
Contributor

Back at it; it's been a nutty month, but I'm going to see if I can make a bunch of the needed updates this afternoon. πŸ˜€

@chriskrycho
Contributor

All right, I have pushed a bunch of changes here to address the other concerns raised, especially by @brson, as well as to clean up the text a bit further. I also tried as much as possible to switch to must/should/may language throughout, dropped unnecessary bits, etc.

As part of that, I've substantially strengthened the bits around how to handle the reference (we should make a strike team, and make this happen), and clarified how to handle the book per @steveklabnik's and @carols10cents' suggestions.


A couple things beyond those changes, then:

  1. On this bit from @brson:

    If we're going to to put policy in place demanding such coverage then I would like it to be enforced automatically. It's easy to imagine a system that validates that every stable feature (which we track) is mentioned in the reference, and fails the test suite if not. And while we're adding a validator we might as well validate that those features are cross-referenced, in the manual, with their RFC. It may be reasonable to leave this to future work.

    I'd like this too, but have no idea what that would entail. It strikes me in any case as much more ambitious than the small suggestions I originally had for an Edit link etc., but I might be missing something obvious there. I'm happy to add a comment describing Future work or something if that's desired, though.

  2. Re: @cbreeden's question here:

    I was wondering if we could clarify this? Some of us were talking on IRC about what this exactly means, we we came to believe that it means:

    1. Final decision to stabilize feature (decision should be independent of documentation)
    2. Add proper documentation
    3. Merge into rust stable.

    This would mean that a tracking issue would remain open even after a final decision to stabilize. Is this a correct interpretation @chriskrycho?

    I think that's one viable approach. Maybe this suggests adding another stage (formal or otherwise?) to the stabilization process: a feature can be in decided to stabilize or ready to stabilize, where the former means "time to make sure docs are ready!" and the latter means "docs are ready, too!"

    I don't have strong feelings on what that should look like from a process-level, though. Presumably the folks actually responsible for making those kinds of changes would have more thoughts on how that part should go.


I think that's everything that was brought up?

@brson
Contributor
brson commented Oct 31, 2016

Thanks for all the updates @chriskrycho. This is a really good RFC now. Though there are still some things I'm not thrilled with, I think it is in spirit good enough to push through.

Here's what I still have qualms with:

  • "a longer summary for the long-form release announcement.". Where does this go? It doesn't seem to be specified. Should authors stabilizing features literally be posting patches to blog.rust-lang.org?
  • "the current state of the reference". The language here still reads as an unenforceable wishlist to me: "should create a strike team". It sounds to me like we aren't going to do what's in this section. Certainly this RFC doesn't itself create a "strike team". In general, I'm skeptical that anybody is going to step up and take ownership of the reference just because the RFC says somebody "should".
  • "This is also an opportunity to allow/enable non-core-team members with less experience to contribute more actively to The Rust Programming Language, Rust by Example, and the Rust Reference.". I don't think this sentence should mention the core team. More like "less experienced community members". It's not just the core team that writes docs today.

I went ahead and clicked the fcp button though, because these are relatively minor, and this is otherwise a really nice RFC.

@brson
Contributor
brson commented Oct 31, 2016

Oh, I have one other technical issue with updating the changelog.

As a requirement for marking a feature as stable one must update the changelog (RELEASES.md). But the format of RELEASES.md is such that it contains links to some PR, usually the PR doing the stabilization. There's a bit of a chicken and egg problem there, though one could imagine the author submits a PR then adds its link by amending the commits.

@chriskrycho
Contributor
chriskrycho commented Oct 31, 2016 edited

(GAH. GitHub just crashed my browser; that was fun.)

Thanks, @brson!

A couple quick responses

I don't think this sentence should mention the core team. More like "less experienced community members". It's not just the core team that writes docs today.

Thought I got all those! Will patch.

Where does this go? It doesn't seem to be specified. Should authors stabilizing features literally be posting patches to blog.rust-lang.org?

Great question! You actually suggested that bit, so do you have thoughts? I liked the idea but didn't know where it should go.

The language here still reads as an unenforceable wishlist to me: "should create a strike team". It sounds to me like we aren't going to do what's in this section. Certainly this RFC doesn't itself create a "strike team". In general, I'm skeptical that anybody is going to step up and take ownership of the reference just because the RFC says somebody "should".

Fair. I guess my question is: what would it take to prioritize this? (Note as well that I used should instead of must there advisedly.) I'm happy to bring it up on the 2017 RFC thread, for exampleβ€”as I think it actually does make a big difference in approachability and learnability for this stuff actually to be documented.

Also, as I stated way up-thread (though, this being an RFC, it's not a surprise to me if someone hasn't read every comment in the thread πŸ˜‚), I'm happy to contribute a bit of time every week to help with that. Even just 15–30 minutes every week will add up. The main thing I need is someone to figure out what needs changing.

As a requirement for marking a feature as stable one must update the changelog (RELEASES.md). But the format of RELEASES.md is such that it contains links to some PR, usually the PR doing the stabilization. There's a bit of a chicken and egg problem there, though one could imagine the author submits a PR then adds its link by amending the commits.

Ah, procedural info I didn't realize. Happy to amend this however makes the most sense process-wise.

@brson
Contributor
brson commented Nov 3, 2016

Great question! You actually suggested that bit, so do you have thoughts? I liked the idea but didn't know where it should go.

I'm inclined to drop the requirement for the long-form description, and just go for the changelog line, simply because we don't have a place to collect this stuff, and because if we actually had such paragraphs it would imply changes to the structure of our release announcements we haven't thought through. But it would be amazing if every new feature had a paragraph explaining and demonstrating it... Maybe @steveklabnik has opinions. I guess we could for now just say to write it up in the PR.

what would it take to prioritize this?

Probably just some individual with the motivation to keep pressing on it.

I'm happy to contribute a bit of time every week to help with that.

Like you...

Ah, procedural info I didn't realize. Happy to amend this however makes the most sense process-wise.

Sigh. I don't know. I feel like let's just whatever, ya' know. It's been long enough.

@brson
Contributor
brson commented Nov 3, 2016

Waiting on FCP approval from @erickt and @pcwalton (click the checkboxes)

@erickt
erickt commented Nov 4, 2016

@brson checked!

@steveklabnik
Contributor

But it would be amazing if every new feature had a paragraph explaining and demonstrating it... Maybe @steveklabnik has opinions.

In projects I've worked on in the past, after a release, we'd create a space in the changelog for the next release, and people would have to modify it with their PRs. This is a recipe for merge conflicts though...

Frankly, the best option, imho, is to be a bit more stringent about commit messages, and put that info in there. Then, when writing the release notes, I can look at the PR directly.

Well, that's my best idea this early in the morning anyway πŸ˜‰

@chriskrycho
Contributor

@brson:

...it would be amazing if every new feature had a paragraph explaining and demonstrating it...

You know, I just realized... if we actually have documentation in the reference, we have that! πŸ˜‚ The changelog can just link to the API docs for the stdlib or to the reference for language features.

@aturon aturon referenced this pull request in rust-lang/rust-roadmap Nov 7, 2016
Open

Rust should have a lower learning curve #3

@carols10cents
Member

Waiting on FCP approval from @erickt and @pcwalton (click the checkboxes)

It looks like they checked the check boxes, does this go to FCP now? Does someone have to manually add the label? (idk how this works really, poking in case this got missed, i'm so excited for this RFC!)

@dikaiosune
dikaiosune commented Nov 10, 2016 edited

@carols10cents rfcbot is having a bad day -- I'm trying to find time to diagnose and fix

EDIT: Looks like the 2 concerns listed weren't ever resolved. They're currently checked independently from individual team members' reviews.

@brson
Contributor
brson commented Nov 10, 2016

This RFC is in FCP.

@carols10cents
Member

Hiiii I thought FCP was supposed to be 1 week, this looks like it's been in FCP for 3? Am I misunderstanding something, or did it get lost in election/holiday shuffle?

@aturon
Contributor
aturon commented Dec 2, 2016

@carols10cents Oy, good catch! It looks like the FCP label was applied manually, and we've come to rely on the bot adding a comment when FCP is over.

In any case, given that we've long since reached consensus here, I'm going to go ahead and merge. @chriskrycho, thanks again for sticking with it!

@aturon aturon merged commit 3d882f9 into rust-lang:master Dec 2, 2016
@chriskrycho
Contributor

Huzzah! πŸŽ‰

@llogiq
Contributor
llogiq commented Dec 3, 2016

Yay! Thanks @chriskrycho for following through.

@carols10cents
Member

YAYAYAYAY!! @chriskrycho, are you going to send a PR to update the RFC template to have the "How do we teach this?" section? I can do it if you're busy, but didn't want to steal the glory from your hard work :)

@chriskrycho
Contributor
@chriskrycho chriskrycho deleted the chriskrycho:document_all_features branch Dec 7, 2016
@bluss
bluss commented Dec 7, 2016

Please edit the "Rendered" link in the message thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment