Spec table design: Add "Initial publication"?

[Elchi3]

We've designed a more simplified spec table for MDN's next generation platform. There are a few things we want to stop doing as it turned out to be bad for maintenance and bad for readers. In particular:

• Links to historical specs (even implementors look up specs via MDN and accidentally reading outdated specs can be so so bad! Oftentimes these links are also useless given they redirect to the newest draft, because spec authors want to avoid people reading old specs, too).
• Spec status (this outdates so fast, and adds little benefits). Also, things being in "draft" often says so little about the stability of the feature and is a bad signal to people wanting to use a feature.

For illustration, here is how we envision this to work in the future:

New spec table specification for future markdown based MDN:

Specifications are added to markdown based MDN docs by using specifications in the front matter of reference pages. Examples:

Feature defined in a single spec:

---
title: "grid-column"
mdn_url: https://developer.mozilla.org/en-US/docs/Web/CSS/grid-column
+ specifications: https://drafts.csswg.org/css-grid/#propdef-grid-column
---

Feature defined in multiple specs:

---
title: "Date.prototype.toLocaleString()"
mdn_url: Web/JavaScript/Reference/Global_Objects/Date/toLocaleString
+ specifications: 
+   - https://tc39.es/ecma262/#sec-date.prototype.tolocalestring
+   - https://tc39.es/ecma402/#sup-date.prototype.tolocalestring
---

specifications is defined as "An optional URL or array of URLs, each of which is for a specific part of a specification in which this feature is defined. Each URL must contain a fragment identifier."
(the pattern is "^http(s)?://[^#]+#.+").

Historical specs aren't added. To control which specification domains are allowed (those that qualify as relevant and non-historical) and to add human readable titles for the specs, a spec data definition file is added. It looks like this:

drafts.csswg.org/css-grid": 'CSS Grid'
tc39.es/ecma262: 'ECMAScript 262 Language Specification'
tc39.es/ecma402: 'ECMAScript 402 Internationalization API'
html.spec.whatwg.org: 'WHATWG HTML Living Standard'
w3c.github.io/webappsec-cspee: 'Content Security Policy: Embedded Enforcement'
w3c.github.io/webappsec-feature-policy: 'Feature Policy'
wicg.github.io/priority-hints: 'Priority Hints'

Non-standard features

If a feature has no spec, you explicitly state so like this:

---
specifications: non-standard
---

---

For now, I've rolled this out to JavaScript pages to test this design in the current wiki based MDN already and to see if we have feedback. Random pages:

• Single spec: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/slice#Specifications (before: https://wiki.developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/slice$revision/1597030#Specifications)
• Multiple specs: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/toLocaleString#Specifications (before: https://wiki.developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/toLocaleString$revision/1165291#Specifications)

I think it is a good change and now presents readers the relevant specs. However, we're loosing a piece of information here that some people are interested in: The initial publication of a feature. You were able to figure this one out in the old tables, because it showed the outdated specs, so you knew in which spec a feature appeared first. We haven't thought about this, because this is more of an advanced bit information, but MDN is also for advanced readers :) Some use cases by @ljharb https://twitter.com/ljharb/status/1220779709832630272. I agree that it does tell you something if a feature is in ES3 or in ES 2019.

So, for sure we don't want to get back to adding outdated spec links, but maybe we should add "Initial publication" to features to our spec table design? Example pages where I've added this now:

• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array#Specifications
• https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/from#Specifications

Thoughts? @chrisdavidmills @wbamberg @sideshowbarker @ljharb
Is "initial publication" relevant for other areas, say CSS, as well? (@rachelandrew?)

[ljharb]

I think anything not in the spec, should link to https://github.com/tc39/proposals, and to the proposal repo; i think the stage doesn’t update that fast and so is fine to inline.

Anything in the spec should imo include the publication year, which is listed on the tc39/proposals repo as well, and won’t change.

Please do only link to the living spec on github, as that’s the most up to date.

[chrisdavidmills]

@Elchi3 fwiw, I like this idea, although I do wonder if it is worth keeping a complete separate row for the spec the feature was initially published in, as well as the latest spec, so people can check back to see if the feature changed since it was first published?

Maybe you can just keep a single table row, and turn the spec name in the "Initial publication" column into a link to that spec?

[Elchi3]

I do wonder if it is worth keeping a complete separate row for the spec the feature was initially published in, as well as the latest spec, so people can check back to see if the feature changed since it was first published?

I'm not aware that "checking if a feature changed since it was first published" is a common use case for MDN readers. It would then provide a link to an outdated spec, which I think is an anti goal here.

[ljharb]

I think it’s mostly about giving the information one needs to configure Babel and eslint: the minimum spec year.

[chrisdavidmills]

I'm not aware that "checking if a feature changed since it was first published" is a common use case for MDN readers. It would then provide a link to an outdated spec, which I think is an anti goal here.

I'm certainly not saying that it is a common goal, or that I'm in favour of outdated spec links ;-)

But to me it would feel a bit strange to cite a spec name without linking to it. However, if that is all that's needed (e.g. for the use case @ljharb mentions), then fair enough.

[sideshowbarker]

So, for sure we don't want to get back to adding outdated spec links, but maybe we should add "Initial publication" to features to our spec table design?

I think in the case of most features, the "Initial publication" spec data isn’t useful, so I would much prefer we don’t carry it forward any longer. It’s hard for me to imagine anybody actually missing it in practice once it were gone.

There are also a significant number of cases I’ve come across in MDN articles where the "Initial publication" information is just outright wrong. (I’ve corrected some of those, but there are lots more that are still incorrect.)

All that said, I recognize the “giving the information one needs to configure Babel and eslint: the minimum spec year” case that @ljharb cites for JavaScript features as something that merits addressing if we can. But IMHO the best way to address that would be by adding data for it to BCD, and maintaining it there, rather directly in BCD. That way, it would also be exposed to Can I Use and other downstream BCD consumers in a way that they could then in turn expose to their users.

And of course it could then also be exposed in MDN articles by using a mechanism similar to what’s used for the MDN browser-compat tables to pull it from BCD.

In that regard, the JavaScript-feature minimum-spec-year data (and use cases for it) seems quite similar, as a class of data, to the existing BCD compat/support data.

[sideshowbarker]

There are a few things we want to stop doing as it turned out to be bad for maintenance and bad for readers. In particular:

• Links to historical specs (even implementors look up specs via MDN and accidentally reading outdated specs can be so so bad!

Yes. Massive +1 to dropping historical specs.

Oftentimes these links are also useless given they redirect to the newest draft, because spec authors want to avoid people reading old specs, too).

Yes, those kind of redirects get set up quite regularly. I know, because I’m the person who sets up a lot of them for W3C and WICG specs.

• Spec status (this outdates so fast, and adds little benefits).

Yes. And since that status is generally set by using the Spec2 macro, it’s only as up-to-date as the source for that macro is in the kumascript repo. And if a contributor notices that the status is wrong in some particular case, they need to submit a PR to the kumascript repo to get it updated.

Also, things being in "draft" often says so little about the stability of the feature and is a bad signal to people wanting to use a feature.

Yes — the data that actually gives a useful indication of the stability of a feature is the browser-compat/support data in BCD and in the corresponding tables in MDN articles.

[sideshowbarker]

For illustration, here is how we envision this to work in the future:

New spec table specification for future markdown based MDN:

Specifications are added to markdown based MDN docs by using specifications in the front matter of reference pages.

I’d like to make a counter-proposal: Stop maintaining spec data directly in MDN articles and instead, let’s plan together to move it all into BCD maintain it there going forward.

The basic rationale for doing that is: It’s data.

…So let’s start maintaining that spec data as actual data — in the same way we maintain other such data in BCD, instead of directly in MDN — so that it can actually be programatically consumed as data, particularly to known downstream BCD consumers such as Can I Use.

That is, it’s the same rationale as what I mentioned in my comment at #2735 (comment) about @ljharb’s use case — but just more broadly applied to the spec data overall.

---

Longer rationale:

There are extremely compelling use cases for exposing feature spec data outside MDN articles. Can I Use is one case, but for me the one that makes it an actual necessity is the case I have for adding “MDN” annotations to all W3C and WHATWG specs — in-spec point-of-use annotations with data and links to BCD compat/support data and to MDN articles.

The WHATWG HTML spec already has such MDN/BCD annotations, and within the next few months, all WHATWG specs will have them, and all the CSS specs too, and many others.

By the end of this year, I expect that basically every spec that defines web-platform features that are documented in MDN and that have data in BCD will have those MDN annotations. That includes the ECMAScript spec. (Of course pending agreement from the ECMAScript spec maintainers; I plan on eventually submitting a patch for enabling MDN annotations there.)

But in order to generate those MDN annotations in specs, I need the spec data in a form that I can programatically consume. Currently I get the spec data into that form by running a script that regularly scrapes (thousands of) MDN articles and then adds that spec data back into BCD.

https://github.com/w3c/browser-compat-data is the BCD fork I maintain for the sole purpose of being able to programmatically expose the spec data. The data in that fork differs from the upstream BCD data in only one way: The data for each feature has an additional spec_url key that gives the spec URL (scraped from the corresponding MDN article.)

For my purposes, I could indefinitely continue scraping MDN and maintaining that fork. But I’d really rather not have to do that.

And anyway, while that fork solves my particular information use case, it doesn’t solve the problem for downstream consumers in general. It’s imaginable that consumers such as Can I Use and the MDN Browser Compatibility Data Explorer might also want to expose the spec data to their users.

(And in fact for those particular two I’ve already opened a feature request (for Can I Use) and a PR/patch (for MDN Browser Compatibility Data Explorer) to get support added for consuming and using the spec data.)

So anyway, if we’re anyway going to be embarking on making changes to how we handle specifications in MDN, I hope we can take some time to strongly consider the possibility of moving all the spec data over to BCD.

[wbamberg]

@sideshowbarker , yes, we should treat spec data as data. We are working on what Florian calls "MDN's next generation platform", and there we plan to represent spec data as data. See mdn/stumptown-content#126 for some background on that.

From that perspective this conversation is really about what data exactly we should include.

About adding "initial publication": I can see the value of that for the use cases listed here, like configuring eslint. But it's worth keeping in mind: at the moment "spec data" is a generic construct across JS, HTML, CSS, etc. However we decide to satisfy this use case for JavaScript it would be nice to do it in a way that makes sense for other parts of the web platform. For example, if we represent it as "initial publication year", that doesn't do anything for CSS items, AFAIK.

[sideshowbarker]

@sideshowbarker , yes, we should treat spec data as data. We are working on what Florian calls "MDN's next generation platform", and there we plan to represent spec data as data.

ah, cool — looking back at the issue description, I see now that I should have realized it myself from reading that @Elchi3 had written. Anyway, given that’s the case, I’m really looking forward to seeing this get launched.

See mdn/stumptown-content#126 for some background on that.

Thanks, I glanced through that I and see y’all have put a lot of thought into this together already.

About adding "initial publication": I can see the value of that for the use cases listed here, like configuring eslint. But it's worth keeping in mind: at the moment "spec data" is a generic construct across JS, HTML, CSS, etc. However we decide to satisfy this use case for JavaScript it would be nice to do it in a way that makes sense for other parts of the web platform. For example, if we represent it as "initial publication year", that doesn't do anything for CSS items, AFAIK.

Agreed on that point. That said, I wonder if for CSS and HTML we can think of use cases for that information/data that have specific utility as clearly as the configuring-eslint case does — especially for tooling.

For CSS, I can say that I know of at least one case of an existing tool that uses CSS-version information: The W3C CSS validator at https://jigsaw.w3.org/css-validator/. If you go to https://jigsaw.w3.org/css-validator/#validate_by_uri+with_options and look at the Profile select menu, it gives you option for checking against "CSS Level 3", "CSS Level 2", and "CSS Level 1". I personally don’t understand why users would actually ever want to check against anything other than "CSS Level 3", but maybe there’s some case — and maybe there are other CSS linting tools that want to provide an option for varying their behavior based on level.

For HTML though, the current tooling I’m aware of is all version-agnostic. Or maybe rather I should say it all just follows the current HTML spec, which is version-less.

Certainly that’s the case for the current W3C HTML Checker at https://validator.w3.org/nu (which I maintain); it offers no options to check against any specific HTML version. The legacy W3C Markup Validator https://validator.w3.org/ does of course still offer options for checking against older HTML versions, though.

[amc1804]

Yes please! For any language I want to be able to say "in this codebase you are free to use language features from version x or earlier". Having an easy way to check whether a given feature was introduced before or after x is very convenient for following such a rule. Until recently, for Javascript, MDN was that easy way. I don't want to have to say "in this codebase you are free to use language features supported by version b of browser B and version c of browser C and..." for a dozen browsers. That's hard to say and hard to check.

[cpcallen]

This does not appear to be complete at all. For example, Promise.any still has no "initial publication" line.