Is partial a long term maintainability problem? If so, how do we fix it?

[dbaron]

I wanted to capture an issue I previously raised in w3ctag/design-reviews#221 (comment) as its own issue. I think partial has a risk of being a long-term maintainability problem for Web specifications, and I think we should consider offering advice about fixing it. In particular, when people use partial interface or partial dictionary in a specification, they're essentially adding to another specification without putting any notation there. (@bzbarsky likes to call this a "come from" statement.)

This leads to a few risks, which include:

• specification authors might define (or implementors might implement) conflicting definitions of a member with the same name
• implementors might implement an interface while unaware of things that extend it
• any readers of the specification might be aware of a feature by name, but unaware of how to find it from the specification for the interface

There are some different approaches the web community could take to mitigate these risks:

• we could encourage that specifications using partial get rolled back into the primary specification defining the interface or dictionary when they become stable parts of the Web platform (for some definition of stable)
• we could encourage the development and/or use of tools that view interfaces in their "complete" form (though still annotated by specification source and perhaps stability)

I think this is worth considering and perhaps making having the design-principles document make recommendations on.

[annevk]

I think another problem is that the iteration order of members from different partials is not defined.

[dbaron]

Another question is whether mixins share any of these disadvantages, or whether it's worth recommending mixins as one of the options to mitigate these issues.

[annevk]

The risks you identified and the problem I identified also apply to mixins.

[dbaron]

This also came up in w3ctag/design-reviews#344 (see particularly w3ctag/design-reviews#344 (comment)).

I think the guidance that I'd like to be able to give is that partial is appropriate for use in early-stage proposals, but around the point where such proposals are being implemented, things should be properly merged into the underlying specifications and partial should go away.

[annevk]

@Ms2ger @littledan @domenic you should probably give some feedback on the above, as maintainers of IDL.

[marcoscaceres]

This would mean folding significant parts of one spec into, for instance, HTML... like GamePad API has:

partial interface Navigator {
    sequence<Gamepad?> getGamepads();
};

so, maybe not a terrible idea... Then the Gamepad would be defined in Gamepad API itself. It does make things a little disjointed tho because the GamePad spec would still need to say something about where to find getGamepads().

[dbaron]

It does make things a little disjointed tho because the GamePad spec would still need to say something about where to find getGamepads().

That seems like a pretty straightforward thing to do with a sentence or two and a link, which seems like the right way to handle this (and also probably some other things that these days don't appear at all in specs).

[marcoscaceres]

@annevk, @domenic, wdty? Should we upstream .getGamepads() to HTML. If you agree, I can get that organized.

[domenic]

Personally I think partials are reasonable.

[annevk]

I'd be supportive of that per rationale upthread.

[domenic]

I guess I particularly don't want to see this done piecemeal. I'd like to see some kind of commitment that all non-incubation groups are willing to roll their Navigator partials into HTML before accepting a single one like gamepads.

Personally I am much more bothered by the extensions to HTML elements than to Navigator, see e.g. w3c/media-playback-quality#11 (comment) and the conversation stemming from there.

[dbaron]

Window and Navigator are probably the places I'm most worried about two separate specs trying to use the same name (and worse, both being implemented in different engines).

I also think Window and Navigator partials could be rolled in to the HTML spec in a very small way, where the HTML spec only has the IDL edit and a stub definition that points to the other spec.

---

(I'd also note there's a bit of relevant discussion in whatwg/webidl#184 as well.)

[dbaron]

@hober is planning to file another issue about "monkeypatching" other specs more generally (per today's teleconference), and this is probably a sub-case of that, although I think a bunch of the discussion about principles here applies to that.

[a2sheppy]

It's worth noting that in the end, the documentation on MDN is already structured with the docs for the contents of partial interfaces integrated directly into the docs for the interface. The specification table, on the other hand, still lists these properties and methods as being part of the spec in which they're defined.

So if you look on MDN, you'll find that the main page for the [navigator](https://developer.mozilla.org/en-US/docs/Web/API/Navigator) object includes the [navigator.xr](https://developer.mozilla.org/en-US/docs/Web/API/Navigator/xr) property, which is defined in a partial interface in the WebXR Device API. If you go to the document for navigator.xr, you'll see that its specifications section lists the WebXR Device API specification.

This has worked well; however, I agree that merging the partial interfaces into the main interface once stable is a good way to go, as long as the content is updated to reflect the transition. Fortunately, only the spec information should need to chnage.

[dbaron]

So in a little bit more detail, here's the guidance I'd like to give; I'd be interested in thoughts on whether other agree or disagree:

partial is appropriate and generally encouraged for early-stage specifications (roughly, specs that haven't been shipped in a release-channel browser or aren't about to be so shipped).

partial is not appropriate for later-stage specifications, because it means that the specifications fail to serve their role as documenting the features of the platform. This can cause developers to fail to understand what is available on an interface, can cause implementers to miss things that they should implement, and could even cause different features with the same name to ship in different implementations.

To avoid having partials in later-stage specifications, things that are initially developed as partials should be merged/rolled into the spec that contains the primary definition of the interface. However, the amount of material that should be merged into the primary spec should be determined by what makes sense for that specific case. In some cases, a large part of the specification should be merged in. In some cases, only the IDL and a one-line definition pointing to another specification should be merged in. (This is particularly likely to be true of common integration points like Window and Navigator.) Other cases are likely in-between. (Here we could probably cite guidance on avoiding monkeypatching from #184 once it's written.)

[domenic]

I don't really agree that HTML should contain definitions for every API that lives on Navigator or Window, even if that is just pointers to other specs. It creates a minor burden on the HTML editors reviewing a lot of PRs, and I doubt that this will ever be uniformly applied enough to give an actual benefit of making the HTML spec an "index of all APIs on Window". (For just one example, it won't include the APIs from the JS spec.)

And, as mentioned above, we haven't even succeeded at this for core things like the definitions of HTML elements and attributes (mostly due to media specs).

I think name collisions are more easily detected at implementation time when they will be a compile error.

I don't feel too strongly on the above, but that's my gut reaction.

[domenic]

For just one example, it won't include the APIs from the JS spec.

Taken another way: the position advanced above seems to be against layering, of the type where e.g. one spec layers new globals on top of the the HTML spec which layers new globals on top of the JS spec. It seems to be advocating for rolling the definitions of all globals into the JS spec, not really the HTML spec, since the HTML spec's definition of Window is basically a "partial" onto the JS spec's definition of the global object.

[dbaron]

I think the layering of Window in the HTML spec onto the definitions in the JS spec is an exception to the rule, and one that's justified by the JS spec having a significant community of users that are not also using the HTML spec (e.g., Node.js). But I don't think that holds for the things that are using partial, which are intended additions to the web platform.

[marcoscaceres]

I'm also leaning towards disagreeing a little bit with the recommendations... Where things act as namespaces, like navigator mostly does, I think it's ok to have a partial. It is, nevertheless, prudent to talk to the people who specified the original namespace.

For non-namespace interfaces, or dictionaries, and other "partial" things, I agree that it can lead to maintainability problems.

[annevk]

I think the recommendations make sense, but it might require some restructuring of how things are managed to keep things manageable. E.g., perhaps the IDL block could be in HTML, but directly link to definitions in other standards. In general it's problematic that when looking at an interface you cannot see its entirety without having a global view of all standards.

In Linked Data there's a follow your nose principle which makes a lot of sense to me in the abstract. If a standard defines X, it should define or reference all of X. You should not also have to know about the existence of another standard that also defines a part of X in order to get the complete picture. That makes it harder to learn things and makes it harder to spot problems.

[plinss]

On thought here is to address this via tooling. The Shepherd database has all the IDL definitions from the specs it tracks, (thought it may need some enhancement for this), it also maintains a list of links out to other specifications in addition to all the anchor points.

It may be possible for Bikeshed, Respec, Wattsi, et al, to use the Shepherd db to be aware of the graph of all the partials in other specs when generating the spec containing the base interface definition. It could then do something like provide a list of all the partials (with links), or possibly auto-generate a fully combined interface (annotating the partials as such, etc) in place of, or in addition to, the base definition. (It could also then do a better job of looking for conflicts, etc)

We're interested in thoughts from the maintainers of those tools, as well as what a possible output of that could look like.

cc @tabatkins @sideshowbarker

[sideshowbarker]

It may be possible for Bikeshed, Respec, Wattsi, et al, to use the Shepherd db

Wattsi doesn’t currently use the Shephard db at all, so adding any kind of interaction with the Shepard db would be all new

to be aware of the graph of all the partials in other specs when generating the spec containing the base interface definition. It could then do something like provide a list of all the partials (with links), or possibly auto-generate a fully combined interface (annotating the partials as such, etc) in place of, or in addition to, the base definition

From that description, I think the level of effort involved with adding that to the Wattsi code would be relatively high.

[annevk]

@plinss I think that would be an improvement on the status quo. It wouldn't necessarily solve ordering however.

[tabatkins]

Hm, thinking on this a bit. As @plinss says, we can tell from Shepherd's data where the original versions of certain interfaces are defined, and where partials are defined, so we could link these up from either side.

What if we required the original interface version to include annotations (a comment, I presume?) linking to each partial? We can warn on both sides when this link doesn't exist, to encourage it to get inserted. Then we can format the original version to automatically include the partial's contents, possibly collapsed by a details by default so it doesn't get too noisy?

This would eliminate the COMEFROM issues, and give us a definitive ordering as well. There's some details that need more thought on this (like the partial probably needs an opt-in so experimental things don't annoy core-spec authors, but we want a check on this so the opt-in actually does get added at some point...), but overall I think this could work?

[dbaron]

Discussed in the TAG's breakout A today. It sounds like we don't have consensus to make a recommendation about reducing use of partial in stable specs. It seems like we agree it would be good to have better tooling for cross-linking partials. But we probably don't want to be putting requests to tool authors in the design principles document since they're not applicable to the vast majority of the readers of the document.

[alvestrand]

webrtc-pc uses partial liberally within a single specification in order to put the WebIDL for a feature next to the explanation of that feature. This doesn't have the cross-reference issue that cross-spec usage of partial has.

(We've also used partial liberally to document extensions in extension documents; the intent for many of these, but not all, is for them to be merged to the main spec eventually.)

We haven't seen that as much of a problem, given that the reviews of all those docs have been in a single WG.

[annevk]

The problem is that ordering between the partials is web-exposed.

I'd recommend coalescing as much as possible and using links to help people out.