Enumeration order of interface members

[tobie]

Spec both says:

The order that members appear [on an interface] has significance for property enumeration in the ECMAScript binding.

and

The order of appearance of an interface definition and any of its partial interface definitions does not matter.

This seems contradictory. Which one is it? And more precisely, how is enumeration order defined across partials if at all.

[annevk]

It's important to some extent for compatibility. E.g., we cannot change the order of some members on the 2D API as that would break sites. How much order is important in general is unknown and proposals have come up for randomizing it and seeing what breaks...

[tobie]

So enumeration order matters between some members defined within the same interface or partial interface declaration. I see.

[tobie]

So if we want to define this precisely, here's a proposal so there's some basis for discussion:

1. Let |members| be a new [=ordered set=].
2. Let |stack| be the result of invoking [=create an inheritance stack=] for [=interface=] |I|.
3. [=iteration/While=] |stack| is not [=stack/empty=]:
   1. Let |interface| be the result of [=stack/popping=] from |stack|.
   2. [=set/Append=] [=For each|each=] |member| of |interface|'s
      [=interface member|members=] to |members|.
   3. [=For each=] |partialInterface| of |interface| declared in the same specification as |interface|,
      in the order in which it appears in the specification:
      1. [=set/Append=] [=For each|each=] |member| of |partialInterface|'s
         [=interface member|members=] to |members|.
   4. [=For each=] |specification| that declares a [=partial interface=] of |interface|,
      lexicographically ordered by |specification|'s short name:
      1. [=For each=] |partialInterface| of |interface| declared in |specification|,
         in the order in which it appears in |specification|:
         1. [=set/Append=] [=For each|each=] |member| of |partialInterface|'s
            [=interface member|members=] to |members|.
   5. [=For each=] |mixin| [=included=] by |interface|, declared in the same specification as |interface|,
      in the order in which it appears in the specification:
      1. [=set/Append=] [=For each|each=] |member| of |mixin|'s
         [=mixin member|members=] to |members|.
      2. [=For each=] |partialMixin| of |mixin| declared in the same specification as |mixin|,
         in the order in which it appears in the specification:
         1. [=set/Append=] [=For each|each=] |member| of |partialMixin|'s
            [=mixin member|members=] to |members|.
      3. [=For each=] |specification| that declares a [=partial mixin=] of |mixin|,
         lexicographically ordered by |specification|'s short name:
         1. [=For each=] |partialMixin| of |mixin| declared in |specification|,
            in the order in which it appears in |specification|:
            1. [=set/Append=] [=For each|each=] |member| of |partialMixin|'s
               [=mixin member|members=] to |members|.
   6. [=For each=] |specification| that declares a [=mixin=] [=included=] by |interface|,
      lexicographically ordered by |specification|'s short name:
      1. [=For each=] |mixin| [=included=] by |interface| in |specification|,
         in the order in which it appears in |specification|:
         1. [=For each=] |member| of |mixin|'s [=mixin member|members=]:
            1. [=set/Append=] |member| to |members|.
         2. [=For each=] |partialMixin| of |mixin| declared in the same specification as |mixin|,
            in the order in which it appears in the specification:
            1. [=set/Append=] [=For each|each=] |member| of |mixin|'s
               [=mixin member|members=] to |members|.
         3. [=For each=] |specification| that declares a [=partial mixin=] of |mixin|,
            lexicographically ordered by |specification|'s short name:
            1. [=For each=] |partialMixin| of |mixin| declared in |specification|,
               in the order in which it appears in |specification|:
               1. [=set/Append=] [=For each|each=] |member| of |partialMixin|'s
                  [=mixin member|members=] to |members|.
4. Return |members|.

[bzbarsky]

Interfaces don't really have a concept of which specification they're declared in attached to them. And in particular:

1. On a text level which specification is mutable, and sometimes there are multiple specifications that do it at once. Right now that's all OK as long as they all agree.
2. Right now, just concatenating all IDL together is a reasonable (and possibly desirable) thing to do.

Likewise "order within a specification" may not be a very well defined concept...

It's not clear to me that "|specification|'s short name" is either stable or unique....

In general, other than the arc/arcTo thing I'm not obviously aware of people depending on the order, fwiw.

[tobie]

So are you suggesting we should just no spec this for now?

[TimothyGu]

Or we can just enforce the relative order of members declared in the definition of an interface in a single IDL fragment, though that does cause questions like this:

interface A {
  DOMString a();
  DOMString b();
  DOMString a(DOMString arg);
};

[tobie]

Or we can just enforce the relative order of members declared in the definition of an interface in a single IDL fragment, though that does cause questions like this:

That doesn't define clearly what "in order" means in step 2 of collect attribute values.

[TimothyGu]

@tobie Well, can't we leave inter-IDL fragment order undefined/implementation-defined? Like isn't relative order within the IDL fragment all we really care about in real life?

[tobie]

Well, can't we leave inter-IDL fragment order undefined/implementation-defined?

I don't really have a strong opinion on this beyond wanting to resolve the current contradiction in the spec and needing to specify mixin member order in host interfaces.

Like isn't relative order within the IDL fragment all we really care about in real life?

For testing JSON order, that's sort of painful, but maybe nobody really cares about JSON order.

[bzbarsky]

For the case @TimothyGu raises (overloads not next to each other), all hope is lost, kinda, because they could in fact be in different IDL fragments too. We could order them by "which one appears first in the global ordering we haven't figured out how to establish"....

I would probably be OK with ensuring things in a given IDL fragment stay in order. As for JSON... people shouldn't care about order there, but who knows whether they do. :(

[annevk]

Couple ideas:

1. A simple global ordering could be lexicographical with the key being the contents of each of the fragments that end up being combined into one. It's definitely not great and if we organize specifications differently there might be some observable side effects.
2. Another thing we could do is require random order between fragments, but specified order for a given fragment. That complicates testing and implementation more than the former strategy I think.
3. We could continue to not care and hope the problem doesn't spread.

[bzbarsky]

So in practice, how do web browsers get these IDL fragments into their IDL parser?

Presumably, they put them into files somehow. Is this a manual or automated process? Is the "fragment identity" preserved in any way during this process?

That is, having ordering depend on the concept of "IDL fragment" assumes that this concept even exists in practice in implementations. If it doesn't and would have to be painstakingly maintained by hand, then chances are it wouldn't get maintained....

[tobie]

Here's what I think Web developers might reasonably expect order to be:

1. members are ordered alphabetically (case insensitive), just like in dev tools,
2. members are ordered lexicographically, and
3. members are ordered as listed in the spec (without really thinking that some APIs are split-up through partials, mixins, and different documents).

I have no idea what the perf cost of (1) and (2) would be. I find them sort of attractive if they're not too expensive.

The problem with (3) is that we're unclear what it means precisely. Could we settle on something were we say:

• order within a single definition is respected,
• order in between definition is vendor-specific,
• overloads across multiple definitions might show up with any of those in a vendor specific way.

[bzbarsky]

I have no idea what the perf cost of (1) and (2) would be.

I can't speak to other implementations, but in Gecko this would be a compile-time cost, not a runtime cost. This is also how dictionary members are already ordered.

Though, actually, that's not quite true. Gecko sets up properties on prototypes in "batches", operations separately from attributes, for various reasons. So right now in Gecko ordering is IDL order (ignoring the multiple fragment thing) within each of operations and attributes, but operations always come before attributes. That said, ordering within each of those lexicographically or so would be easy.

The problem is that the arc/arcTo thing we ran into in https://bugzilla.mozilla.org/show_bug.cgi?id=623437 needed arcTo to come before arc.

[littledan]

What are the next steps to make progress on this issue? The question came up again in #675. @bzbarsky The issues you point out with the above proposals seem pretty significant; it doesn't sound like WebIDL fragments are preserved contiguous in general, or that we have well-defined short names for specifications. Any more ideas?

How should we proceed with defining new "partial" constructs in WebIDL? Should we hold off, due to the ambiguity, until we have a solution to this issue?

[annevk]

I'm leaning toward:

1. Order within a block (interface / partial interface) is preserved.
2. We compute an order key for each block (e.g., concatenate all member names?).
3. We order the blocks and then combine them in order.

[littledan]

@annevk That plan sounds good to me. I like how it's insensitive to implementation-specific extended attributes, whitespace differences, etc., and how it's easy to run in your head. I don't think people will run into the theoretical ambiguity (when static members may share names with non-static members).

To spell it out a bit further, the order could be:

1. The main interface (or namespace or module)
2. Partial interfaces (sorted by the concatenation of names)
3. Included interface mixins (sorted by the name of the interface mixin), closing #473

Seems like we have room to change things here, as browsers disagree with each other in enumeration order already. For example, in Document.prototype, Chromium and Webkit put implementation first, whereas Gecko puts getElementsByTagName first--and this is just in the "easy case" of things declared in the interface itself.

[Ms2ger]

Fwiw, I'm pretty sure Gecko defines methods and attributes separately, so we'd have to look at the relative order within those groups.

[domenic]

I think the spec also defines methods and attributes separately!

[littledan]

Yeah, looks like attributes come before operations, per step 10 of https://heycam.github.io/webidl/#dfn-interface-prototype-object , so I guess the spec disagrees with Firefox here. Was this intentional? I wonder if this text was written before we cared about enumeration order.

I would've expected that things are interspersed. In a JavaScript class, with methods, getters and setters, they would be.

[domenic]

The history here is, to my recollection, that originally all attributes and operations were added declaratively, in no specified order. I introduced the first imperative installation of properties with namespaces, and encouraged @tobie to modernize other constructs in that way. Recently @Ms2ger has pushed that work much further. (Perhaps finished it?) I don't recall us ever stopping to consider enumeration order.

I think the extra thing to consider here is implementation complexity or precedent. I think many implementations may have monomorphic, type-by-type loops like the spec does, instead of for (member of members) { member.define(theClass) where .define() is polymorphic on the type of member.

[Ms2ger]

Gecko's implementation has been doing methods, then attributes (and then constants) since first using WebIDL in 2012 (https://hg.mozilla.org/mozilla-central/rev/1bdb337e3136#l27.69; I don't think we cared about enumeration order at the time.

[bzbarsky]

Switching the relative order of attributes and methods easy in Gecko.

Ordering each of those sets in some order like what #432 (comment) describes is work (we'd have to have a lot more complexity and state in the "build the interface member list" code than we do now), but pretty doable.

Having to have an order that interleaves attributes and methods (whether that's on a per-source basis or within sources, where a source is a mixin or partial interface definition) would be a big pain implementation-wise.

I guess mixins can't include other mixins, right? That simplifies things a bit.

Within a mixin, presumably partials are also handled by the same approach as for interfaces, but affecting the overall order of mixins?

Concatenation of names does not provide unique sort keys. Consider:

  partial interface A {
    readonly attribute long a;
    readonly attribute long bc;
  };
  partial interface A {
    readonly attribute long ab;
    readonly attribute long c;
  };

What we could do instead if sort by just first member name, which should be unique. Especially if we prefix the name with "Regular", "Static", "Both" (for constants) or something, to deal with the static and non-static sharing names case.

One thing worth thinking about here: Ideally the order of partials would not change when new things are added to the partials. I think we get that if we use the first member as key, as long as people add things to the end...