[charter] Define the Success Criteria

[siusin]

In the draft charter,

The Working Group is successful if it has been able to produce specs and standardize browser behavior so that it is possible to write a basic JS text editor that works across scripts with no more than 3 years development effort.

Could you clarify the term "across scripts", what script does it refer to?

It'd be also helpful to explain the rationale of "a JS text editor with no more than 3 years development effort".

[johanneswilm]

Hey @siusin ,
this is to refer to Latin script, CJK, Arabic script, etc. . If the term is not correctly used or confusing, feel free to replace it with a more accurate term. The point of including it is that historically, JavaScript editors have tended to only work well with Latin script and have been buggy when used with other scripts.

[siusin]

Thank you @johanneswilm .

/cc @r12a and @wseltzer

[r12a]

Big thank you @johanneswilm ! :). Might i suggest using the term 'writing system' rather than 'script' to reduce the ambiguity.

[BoCupp-Microsoft]

possible to write a basic JS text editor that works across writing systems with no more than 3 years development effort.

I think the success criteria should be easily measurable and preferably not so monolithic. The statement above seems ambiguous (what's the definition of "basic JS text editor") and the criteria: "3 years of development effort", seems like by definition could take a while to measure: up to 3 years :-).

Also the criteria seems like it encompasses too much. A JS text editor requires many pieces beyond the scope of this working group. I'd prefer criteria that can be evaluated per specification so we can measure our progress in a more incremental way.

I like the language from the CSS Working Group charter.

Putting its language in my own words I see that it focuses on:

• Whether we're producing stable documents that implementers can follow
• Whether we're producing test collateral to measure how well those implementations conform to our specifications
• Whether there are independent implementations in existence
• Whether there has been broad adoption of our APIs by the web development community

Those all seem easily measurable and like important indicators of success. The first two are measures of how well we're executing and second two are measures of how well we served our audience (the implementors and the web developers).

@johanneswilm and @gked, what do you think about changing the language in the proposed charter?

[johanneswilm]

Hey @BoCupp-Microsoft ,
actually I thought it was better to have not so measurable targets. As it is now, we are very much restricted by the requirements of having 2 implementations lined up right from the start of the spec development process - which for example means that EditContext is kind of out of scope for us as it is now. The point is that it doesn't matter whether we are producing 1 spec or 150 specs over the next few years. If 1 spec is enough to make it be possible to create a text editor, then that is enough and the WG has been successful in its task. On the other hand, if we produce 500 specs yet even with all those it still is way too complicated to create a richtext editor on the web, then we have failed.

That being said, I don't mind changing the success criteria to something else. It should just a) not prevent us from working on specs that are important to achieve said goal and b) it should not make us working on countless specs that achieve basically nothing just to keep the numbers up. The way you have stated the goals from the CSSWG, that sounds fairly general and something like that we could adopt as long as it stays general and doesn't get into too many specifics.

[BoCupp-Microsoft]

@johanneswilm the success criteria for CSS says:

in order to advance to Proposed Recommendation, each module is expected to have at least two independent implementations

It does not say that two implementations are necessary before a spec can be developed. Let me know if I'm misunderstanding this concern:

As it is now, we are very much restricted by the requirements of having 2 implementations lined up right from the start of the spec development process

In terms of what we work on, my read of the CSS WG charter, which I think has a good model, is that we can work on anything within our scope that the group agrees to work on. I'm seeing that defined by the Scope and Deliverables section of their charter.

[johanneswilm]

@BoCupp-Microsoft
Yes, that is different from what is in the Web Apps Working Group that we are currently under, which specifies:

Additionally, support from two or more web platform implementers is required before a substantive change can be made to a specification. This is enforced by a pull request template on GitHub, which Editors must fill out as a public record before a substantive contribution can be merged. [1]

That is the part that basically means we should not make any change to any draft without two implementers agreeing to implement it like that. So whenever you make any change to the EditContext spec, technically, you'd need promises from at least Firefox or Safari that they will implement it also in the new way.

I think the CSS WG works well for what it does. But it's currently different from this Editing Workgroup in that it is formed as a permanent working group that has no particular start or end. It's possible that the Editing WG also will end up like that as we find more and more things of interest to standardize in this general area. But for now, at least for the duration of this charter, the goal is still to make it easier to create webbased richtext text editors, right? And so therefore the success criteria we set for ourselves should be whether we are able to deliver this feature to the web in some form or other or not. The period of "3 years" is picked rather arbitrary though.

That being said, I can see how "success criteria" can be understood in multiple ways, and so if it's more common to put some general text there that just kind of described what we are doing, then that is fine with me as well.

[1] https://www.w3.org/2019/05/webapps-charter.html

[BoCupp-Microsoft]

Additionally, support from two or more web platform implementers is required before a substantive change can be made to a specification. This is enforced by a pull request template on GitHub, which Editors must fill out as a public record before a substantive contribution can be merged. [1]

That is the part that basically means we should not make any change to any draft without two implementers agreeing to implement it like that. So whenever you make any change to the EditContext spec, technically, you'd need promises from at least Firefox or Safari that they will implement it also in the new way.

@johanneswilm I agree with your process observations for specs on the recommendation track. The EditContext isn't yet a spec on a recommendation track, it's an explainer in the incubation phase. Maybe it shouldn't be part of the working group's charter yet, but wherever it lives it isn't reasonable to ask for implementer commitments for the changes were making to the proposal at this stage of its development.

Let's discuss scope in a separate issue and stay focused on success criteria in this issue.

That being said, I can see how "success criteria" can be understood in multiple ways, and so if it's more common to put some general text there that just kind of described what we are doing, then that is fine with me as well.

The success criteria of the Web Apps WG, the CSS WG, the Accessibility WG, the Media WG and other working groups I've randomly clicked on all have very similar wording.

The common components of the success criteria seem to be:

• Getting two implementations of specs
• Enumeration of accessibility, privacy and security concerns in specs
• Tests for specs

It sounds like your open to adopting this more general wording. How about we use the Web Apps WG success criteria as its currently written:

In order to advance to Proposed Recommendation, each specification must have at least two independent implementations in wide use.

Each specification must have an accompanying test suite, which is ideally developed in parallel to the specification. The test suite will be used to produce an implementation report before the specification transitions to Proposed Recommendation.

Where there are implications for implementors, developers, or users, in the areas of accessibility, internationalization, privacy, and security, each specification must have a section that describes relevant benefits, limitations, and best practice solutions for that particular area.

[johanneswilm]

@johanneswilm I agree with your process observations for specs on the recommendation track. The EditContext isn't yet a spec on a recommendation track, it's an explainer in the incubation phase. Maybe it shouldn't be part of the working group's charter yet, but wherever it lives it isn't reasonable to ask for implementer commitments for the changes were making to the proposal at this stage of its development.

Hitherto it applied to every document we were working on - no matter where in the process. It's the reason why the contenteditable spec [1] is now empty. The only reason that it hasn't applied to the EditContext document is - as far as I can tell - that we didn't tell anyone we started working on that, nor asked permission.

Let's discuss scope in a separate issue and stay focused on success criteria in this issue.

The reason why this is relevant here is that part of my reason why I thought the current text was a good idea is that we needed to make clear we are working on old and new specs trying to solve a specific problem (text editing on the web) and so we are trying to drive specs in coordination with development in the browsers. That's different from what we currently have, as the restrictions basically limit us to just speccing features that already exist, possibly coordinating the change of some behavior mildly for the browsers to behave more similarly, but not developing and implementing entirely new APIs.

Given that the sentence that causes this is not part of this new charter, I don't see a particular reason to start a separate conversation on this.

The success criteria of the Web Apps WG, the CSS WG, the Accessibility WG, the Media WG and other working groups I've randomly clicked on all have very similar wording.

If all charters have the same success criteria, then maybe it would make sense to move this section to some general W3C documents rather than have all the charters repeat one-another.

I still think we are different from the WGs that you mention in that all of them are basically constantly ongoing efforts with no end date in sight, whereas we might actually decide not to recharter if we think our problem has been resolved. We are not really successful if we manage to get 500 specs through that process, yet they are just 500 more useless APIs that do not help JS devs much in creating text editors.

So removing the information on what we are actually aiming to do while adding some general text that is in all the charters, doesn't seem that helpful to me. But maybe we can move the text on what the actual goal is to somewhere else in the document? An explainer text somewhere possibly?

[1] https://github.com/w3c/contentEditable/blob/gh-pages/index.html

[siusin]

How about --
In order to advance to Proposed Recommendation, each specification is expected to have at least two independent implementations of each feature defined in the specification, which may include in a basic JS text editor that works across writing system, based on tests. The Working Group is therefore expected to proactively work on writing, reviewing, and maintaining tests.

[johanneswilm]

@BoCupp-Microsoft what do you think? How can we include both the original intention and these criteria of what we expect of specifications we develop?

[gked]

@johanneswilm, I agree, it is not about the specs per se but rather about whether they solve developer problem.
As W3C working group we should agree on the successful flow which I think, we are all aligned on:

1. Developer presents a problem.
2. Someone (can be invited experts or browsers themselves) propose solution that solves the problem via an explainer
3. People discuss high level solutions via the explainer.
4. Once there is a general agreement on the shape of an API, spec editor is determined and the spec is drafted.
5. Spec editors get a commitment from at least two implementers.
6. Spec draft is moved into a official recommendation draft.

I think what we are talking about here is the commitment on delivery of steps 5 - 6. Steps 5 and 6 are naturally out of control for WG. If chairs also happen to be spec editors then 6 can be mitigated.
And I think the wording below matches the flow I outlined with minor corrections:

How about --
In order to advance to Proposed Recommendation, each specification is expected to have at least two independent implementations of each feature defined in the specification, which may include in a basic JS text editor that works across writing system, based on tests. The Working Group is therefore expected to proactively work on writing, reviewing, and maintaining tests.

In order to advance to Proposed Recommendation, each specification is expected to have at least two independent implementations of each feature defined in the specification, which may include in a basic JS text editor that works across writing systems and with a sufficient number of tests. The Working Group is therefore expected to proactively work on writing specifications, reviewing issues, and maintaining tests.

[johanneswilm]

Ok, that's fine with me. This part is a bit unclear to me. Is that grammatically correct or something missing:

which may include in a basic JS text editor that works across writing systems

[BoCupp-Microsoft]

@BoCupp-Microsoft what do you think? How can we include both the original intention and these criteria of what we expect of specifications we develop?

I think the original intention is well captured by our proposed scope and proposed deliverables section. I would support amending those sections further if you think they do not.

I'm opposed to the phrase "basic JS text editor" as a success criterion because I don't think it's measurable.

I would prefer just these three sentences:

1. In order to advance to Proposed Recommendation, each specification is expected to have at least two independent implementations of each feature defined in the specification.
2. Each specification must have an accompanying test suite, which is ideally developed in parallel to the specification.
3. Where there are implications for implementors, developers, or users, in the areas of accessibility, internationalization, privacy, and security, each specification must have a section that describes relevant benefits, limitations, and best practice solutions for that particular area.

[siusin]

Indeed, the group don't have to explain what is an independent implementation at this stage.