TAG and AB should be able to formally review charters #328
Comments
|
A middle position would be to get on charters to what I'd also like to get on specifications: that the TAG, AB, and cross-functional groups can give a red/yellow/green light, with associated opinions and reasoning, for the AC to consider when voting. |
|
Simply allowing AB and TAG members to vote on AC ballots raises the usual concerns about one member with successful AB and/or TAG candidates effectively getting multiple votes in the AC. I'm intrigued by the idea of various cross-functional groups being able to give a "red/yellow/green light" in advance of an AC ballot, but only if that were a consensus (or at least supermajority) decision by the group, and put forth points within that group's scope. For example, the AB could weigh in on process considerations, TAG on web architecture considerations, PING on privacy, etc. |
|
I don't like Vote; I do like horizontal review of charters, and I think it's way overdue that AB+TAG be formally notified of proposed charters before they go for ballot, and their input requested. There should be no delay, unless they push back. Is this a Process question? |
Maybe. In the present world, charter objections get tossed to the Director, usually with little opportunity for the community to first try to find its own consensus. I would love to have a community forum for trying to find consensus on charter issues - one that the TAG, AB, horizontal reviewers, and others could all participate in. I imagine that a clear community consensus would emerge from most of those discussions, which could then leave the Director - or post-Director structure - merely judging the community's consensus. I'd prefer the AC be included in that community discussion and not have a special role (like a vote), but even if the AC must remain special, I think changing the objection process to focus more on the community will help us. So the Process questions might involve:
|
|
Who evaluates the traffic lights? Is a red light automatically blocking? Modelling this as an extension of AC voting is not really workable IMO. The AC is near-dysfunctional in its silence, and currently represents only the folks (usually, companies) who have an interest in particular work progressing. |
AC Reviews are not votes. We're not counting support votes and comparing that number to the number of FOs. The number of ballots one way or the other is not what determines the outcome. What we're doing is assessing whether people see something wrong with the proposal. If nobody does (and at least some people like it), it goes forward. If anybody has an issue, we assess the validity of their concerns. If they're bunk, they get overruled. If they have a point, we listen to it. To the extent that TAG and AB members are considered to have relevant and informed opinions on charters and specs (I'd say that this is no less true that the average AC Rep), giving them a chance to voice that opinion in case they see something wrong seems like a good idea. Also, I'll note that by the Process, FOs are not restricted to AC Reviews. Theoretically, AB and TAG members (and anybody else) can already object to whatever it is they want to object to. But I agree it would be more convenient if they were called upon at the same time as the AC. That said, I also agree that FOs should be a tool of last resort, and there typically are ways to address concerns earlier and more effectively. That too should be discussed, but I think that's a separate issue. |
|
My traffic-light proposal was to provide a visibility to the community - members, team, Director - of the overall feeling of the horizontal review groups. They should be able to see if any advise that the train should stop. I don't think we need to, or should, give them formal stopping power. |
+1 How much time should the TAG + AB have to respond to a proposed charter, should they have a required response such as:
I think getting this up front will help substantially, event a default of "if no response is received in the time window, a no comment review is explicitly provided". No need to give them formal power, but I do think they ought to be required to take a position on charters they will adjudicate, before the charter is voted on. |
|
Earlier, I said:
In light of the discussion happening in #580, I am now thinking that it may be better if they were called upon before the AC reps are, so that the AC review can take that feedback (if any) into account. |
|
To me, it's a function of how formal we want to make the AB+TAG review prior to the AC review. Right now, we do horizontal reviews of charters , prior to the AC review, it has been effective in general, even if it's informal. So, this may well be that it's a guide issue if we keep it informal. If we make it formal, then it does require more Process modifications indeed. |
|
wow, if it's happening, as an ex-AC Rep, I was unaware of the results of such horizontal review when reviewing charters. I still dream of my 'traffic lights' on horizontal review questions, where we could see from each possible horizontal:
|
|
While we don't have a green/yellow/red/blank, we do have green/red horizontal review boards for each specification. It's still not perfect, since review outcome can be confusing sometimes. But it is the current starting for the Director to check things out. It does not work for charters however. One immediate way to improve this would be to link to where the horizontal reviews for charters are happening from the AC reviews. Those are happening in w3c/strategy. While the horizontal groups don't have a formal role, the team horizontal reviewers do ensure that the groups get involved in general. |
|
Does "No open horizontal issues found. No closed horizontal issues found." mean that a review was done and nothing found, or that nothing happened? But overall, that needs summarizing or linking to the AC when they are asked to vote!! |
It could mean either way. The Director does not allow, in general, a transition forward without proper horizontal reviews. But yes, it should be made clearer in the boards.
I concur. cc @koalie |
Actually, let me put some nuance. Today, for specifications, we're linking to the transition request, which contains all of those information already. So the AC is one click away from finding the information. I do concur that, for charters, we should add the link. |
|
Overall, I think that it would be desirable for AC reps voting on a charter to know if any AB or TAG member has raised concern over that charter. I don't care that strongly about what way we achieve this. It could be something like any of the following:
|
|
It would be helpful to be clear on what the problem is that we're trying to solve. I think it's something like "Groups may be chartered to produce specs that are incompatible with the web architecture principles or community values. Once upon a time there was a Director to make judgments on how to maintain the conceptual integrity of the w3c's standards system, but now that role is essentially vacant. So we need some mechanism for the TAG, AB, and the horizontal review groups to evaluate such matters at charter time, and have any concerns explicitly raised during the approval process." Is that about right? |
|
+1 to having a clear definition of the problem. Replying to Florian's list of options, input should be consensus views of the TAG and/or AB rather than on an individual TAG/AB member basis. |
Yes. (The way you've expressed it seems somewhat TAG centric. For AB feedback, it might be along the lines of "the procedures proposed for this Working Group seem at odds (or in clear violation of) the Process", or on the contrary "… seem far more stringent that the Process and usual practice calls for, with no clear reasoning supporting that". Or maybe "…could be seen as hostile by the members of the ***-WG, or by organization XYZ with which with have a cooperation agreement". )
Overall, yes. However, getting to formal consensus in these groups is typically very time consuming even when we do agree, and I'd be wary of creating too many bottlenecks and cause delays. To the extent that these reviews aren't themselves blocking, and merely inform the AC, maybe we don't need to have formal TAG/AB consensus, and can take individual comments (marking them as such) and make them visible? Coordination would still be welcome, but I'd like to find a way not to block on that. Also, note that the existing W3C process allows anyone at all to make formal objections, without restriction. People other than AC Reps aren't surveyed during AC reviews, but still have the right to object. |
|
@frivoal noted above
Which makes me think this isn't a valuable thing to work on. |
|
@chaals Doing so is allowed, but:
Which is why I think it might be a good idea nonetheless |
|
@frivoal wrote:
+1 thanks @chaals wrote
It seems to me:
|
|
It might be more palatable to the TAG and AB if, rather than having to review all Charters, there could be some way for any member of the community to say "there's a procedural/architectural concern here and we would like AB/TAG input on this charter". I can't see how to make it an opt-in without incurring delay, though. |
|
@dwsinger wrote:
@plehegar wrote:
So long as PLH cc'ed me 4 days ago, and that I've now read the proposals, the comments, the related issues/comments elsewhere, my overall recommendation would be to distinguish changing or not the role of the AB (in the case of the TAG the roles as described in the Process today already includes reviews under " to help coordinate cross-technology architecture developments inside and outside W3C. " of section 3.3.2.1), and what kind of information the Advisory Committee reps would find useful when the Team sends group charters for AC review. On the former (formal review role of AB and TAG), which is precisely the focus of the original issue raised in September 2019, I disagree that the AB needs to formally be participating in such AC reviews; and in the case of the TAG, I see this as a non-issue (cf. reference in my previous paragraph.) Furthermore, since the issue was raised in the context of "Director-Free", my preference would be to set expectations at the council level but not at the AB or TAG level, so that if the conditions which led to setting expectations were to disappear or change, the expectations would not stick and carry over without purpose. On the latter (review information sent to the AC in charters calls for review), I'm happy to augment what the Comm team sends, BUT, I disagree this requires any Process Document change. Furthermore, we regularly get complaints from AC Reps that our communications are too frequent, too verbose. |
|
@koalie I don't understand this:
The Council only gets involved when there is a formal objection. I think part of the goal here is to reduce the likelihood of formal objections. Can you help me understand what you wrote here? I'm realizing that I had not realized that the HR results were linked from the AC review, probably because I never thought I needed to review the formal request. In the review to AC it says
and if you don't think you need to review the formal request, you might not click on the link of 'transition request'. I'm not sure how better to phrase this to alert AC Reps that there is more than the triggering request behind the link. For Charters, however, there is nothing to suggest that HR has been done. There's a section 'Review Materials' in the AC review request, but I sampled a few recent ones and none contained any hint of HR. Even if HR review has happened, it has not included TAG or AB. I'm not sure I see any harm in including TAG and AB in the HR notifications for charters and at least giving them the opportunity to review and provide comment. (AB for any procedural concerns, TAG for technical/architectural.) |
|
The proposals so far seem to focus on surfacing more information to the AC, to inform any feedback/objections it makes. While that has the merits of an incremental approach, I think it might be sweeping some signficant problems under the carpet. Firstly, it relies on the AC's expressed support and FOs as the primary mechanism for assuring the quality of a charter. However, the AC is not well-suited to do that, even with optional well-meaning advice (like 'traffic lights'); it has notorious participation problems, and as we've seen attempting to use FOs to steer standards activity is fraught. Second, we currently have multiple layers of review / approval to assure quality and alignment -- horizontal, AC, and Director. In a Director-free W3C, one of those layers is removed, and this approach does not account for that loss. While the Team has often performed this function on his behalf in the past, it's not appropriate ot continue that practice in a Director-free W3C - it needs to come from the community and be accountable to it. Third, it reinforces the bias against saying 'no' to charters that don't fit the mission of the organisation, because that will require not only someone to make an effort to build a case for it, but also enough AC reps to decide to object (and pay the cost of following through on it). If we're to be a values-driven organisation this seems like a pretty open door for things that will cause issues down the road. A much more successful model would be to make the TAG responsible for new charters (much as in the IETF). AC reps would be able to rely on them to assure that quality in an accountable way (because they're elected by the AC). One objection to giving the TAG firmer responsibility for chartering is that it will take too much of their time. I'd argue that assuring that charters are aligned with our mission, value, and goals as an organisation is a primary responsiblity for the TAG in a director-free W3C, and if necessary we should restructure them to accommodate it. |
|
Perhaps we should make it clear that the TAG/AB have an opportunity to review, and if they do, where they can post the results, without imposing an obligation since they are both already very busy. I don't see a problem with the chairs asking "we have XX charter out for review, does anyone feel we need to review it?" or even "could I have a volunteer to skim it and see if there is anything that suggests we should review it?". So let's consider adding TAG and AB to charter HR notifications? |
|
The Revising W3C Process CG just discussed The full IRC log of that discussion<fantasai> Topic: TAG and AB formally reviewing charters<fantasai> github: https://github.com//issues/328 <fantasai> plh: 2 questions <fantasai> plh: how formal do we want those reviews to be? the more formal it is, the more it puts pressure on the timeline <fantasai> ... today we do informal horizontal review of charters <fantasai> ... we do them, but it's not encoded into the Process, just encoded in the Guide <fantasai> ... and they do time out <fantasai> ... if we don't get review from Privacy in a month, we'll still move forward <florian> q+ <fantasai> ... because after that there's review from W3M ... <fantasai> ... sometimes charters can have months of conversation in community as well, so by the time they get to this stage they're wanting to go <fantasai> ... so question of whether TAG/AB is able to do in a timely fashion is a question <fantasai> plh: Second question is how much should we inform the AC of this? <fantasai> ... e.g. for specicifcation transition, we link to the transition request <dsinger> q+ to make a confession <fantasai> ... if the AC wants to review that, and how the Director arrived ad conclusion, they can follow the links <fantasai> ... That's not necessary to answer in the Process, but would welcome input <fantasai> ... how can we make it clearer that there's information in those links to follow? <fantasai> florian: for this horizontal review phase on charters, it would be good to visibly survface to the AC if any HR Group has raised concerns, AC will have that information in their vote <fantasai> ... I suspect includeing TAG/AB review in that is reasonable <fantasai> ... but both the TAG and AB have a very heavy schedule <fantasai> ... so it will be difficult to include in their schedule <plh> q+ <plh> ack florian <fantasai> ... so suggestion from someone was to flag if any individual on the AB or TAG has raised a concern <plh> ack ds <Zakim> dsinger, you wanted to make a confession <fantasai> dsinger: First, a confession, when AC review linked to transition <fantasai> ... I thought that was a proof of the request <fantasai> ... rather that there was information there <fantasai> ... I suspect we need to ask AB and TAG how involved they want to be <fantasai> ... I suspect AB and TAG in a less stressful and more organized world would be able to assign an individual to skim through charters and check if it needs a more thorough review <fantasai> ... but I think the AB and TAG need a community discussion <fantasai> ... IIRC someone (mnot?) mentioned that someone needs to take a global look at what are we doing procedurally and technically <fantasai> ... if not the AB and TAG, is it the Team? Other members of the community? <fantasai> ... I don't want to add to the AB and TAG workload by mandating something <fantasai> ... but I think there's a reasonable question here <plh> ack fantasai <plh> fantasai: when I filed the issue, I was just asking for the ability for the TAG and AB to participate in the AC review <plh> ... for earlier review, that might still be valuable <dsinger> q+ to say that there is a difference between an *ability* and opportunity for TAG/AB to review, and a *requirement* that they do so <florian> q+ <plh> ... we don't have to include things in the process yet <plh> ... for a requirement, it would have to be included. <plh> ... +1 on this being an increase in the workload <plh> ... the chartering process needs to be easier to ensure specs to be on the rec track under a formal process <plh> ... if the AB and the TAG want to get involved, they can ask <plh> ... but I don't think we should be requiring it for now <plh> ... one: consider whether the AB/TAG to formally review alongside of AC reps <plh> ... this might be useful <plh> ... second: it might be useful for the Team Contact to update the AB and the TAG on charters being worked on <plh> ack plh <fantasai> plh: +1 to what you just said <plh> https://github.com/w3c/strategy/projects/2 <fantasai> ... as a reminder, the work we do on Charters is documented in GH <fantasai> ... in the strategy pipeline repo <fantasai> ... If tomorrow you want us to send notice to AB/TAG every time we start review of a charter <fantasai> ... [missed something about Dom and software] <fantasai> ... Can send notice for horizontal review of a charter <fantasai> ... can also make part of Team update <fantasai> ... unsure how much it's needed to mention thosethings <fantasai> ... but anyone can see that <fantasai> ... it's a public repo <plh> ack ds <Zakim> dsinger, you wanted to say that there is a difference between an *ability* and opportunity for TAG/AB to review, and a *requirement* that they do so <fantasai> dsinger: Want to distinguish between requirement vs opportunity for AB and TAG to review <fantasai> ... want to be clear that this is opportunity, not requirement <fantasai> ... so questions are, how do we make them aware of the opportunity <fantasai> ... and how do we accept their feedback? <fantasai> ... Giving them a formal notice gives them an opportunity <fantasai> ... and giving them a place to file issues solves the second problem <fantasai> plh: There's a template for where to raise issues <fantasai> dsinger: Then as part of AB and TAG meetings, we can say, we had these three charters come across our plate, does anyone feel need to review? <fantasai> plh: We send emails when we start a review, so we can add AB and TAG to those <fantasai> dsinger: So it would be easy for AB and TAG to add that to the formal part of the meeting <dsinger> ack ds <fantasai> florian: Anybody can file FO about anything <fantasai> ... so technically AB and TAG members and members of my neighborhood association can file FOs <fantasai> ... but not being *asked* to do so during AC review <fantasai> ... You can send an email, but don't get a form <fantasai> ... given that you can already do it, not a new power... <fantasai> dsinger: Sense of "AB feels there's problems" is different from "Florian has a problem" <fantasai> florian: I can still do that, but there's no form <fantasai> plh: How do you formally object before review? <fantasai> florian: You send an email to the Team <fantasai> plh: In the Process? <fantasai> florian: It says you can, just doesn't say how <fantasai> [florian goes to look for quotes] <fantasai> florian: We routinely have ppl in WGs saying they would FO, and chairs continue the discussion <fantasai> plh: That's a spec, not a charter <fantasai> plh: let's not get into that today <fantasai> florian: it's this issue, kinda <plh> fantasai: let's not resolve today who is allowed to raise formal objections on charters <plh> ... we can add AB and TAG in the loop informally <florian> +1 <plh> dsinger: let's propose adding the notifications next week to the AB and then ask the TAG if the AB agrees <florian> q- <plh> ack florian <plh> ack fantasai <Zakim> fantasai, you wanted to comment on repo <plh> fantasai: the charters are in the strategy repos <plh> https://github.com/w3c/strategy/projects/2 <plh> https://github.com/w3c/strategy/issues?q=is%3Aissue+is%3Aopen+label%3A%22Horizontal+review+requested%22 <fantasai> ... but a lot of other things too, how to get a list just of charters open for review? <fantasai> plh: But if we add to notification list, can just look at emails <fantasai> fantasai: Seems that's it for this topic, any other action items? |
|
@dwsinger if we do that, who's accountable for chartering decisions? Who will stand up and defend them and face consequences if they get it wrong? I.e., where is the identifiable locus of the decision? In the current system Tim and the Team are -- it's a loose sort of accountability, in that both are unelected, but they feel the heat if they get things wrong, at least. They can at least be identified and asked. As far as I can see what's being proposed is making the AC accountable. Given that they're a large, loosely coordinated and diffuse body that's also unelected, and given that AC participation is an ongoing and ackowledged problem, I fear this is a recipe for some very bad outcomes, both in terms of internal strife and external perception. |
|
The membership and the AC are finally responsible, I think. I want their decision to be informed. I agree, we have a problem of engagement, partly because I fear that we don't make it very clear why input is important. |
I understand @mnot to be asking who is accountable for making good chartering decisions. Now it is essentially nobody. For example a charter that brings in new members will be treated as a a success by management with no mechanism to determine whether the charter helped “lead the web to its full potential”. An answer could be “management is accountable to the (future) Board for making good chartering decisions with the advice and consent of the AC🤔 But I And apparently Mark believe it would be better to have some person or elected group held directly accountable. |
|
I don't think it's nobody now; I think both the team and at least some AC Reps do try to have quality charters. I am fairly sure that adding a requirement to the AB and TAG that they review every charter would be onerous and unproductive, and I am not sure it provides the kind of cohesive vision oversight we assume the Director once gave. Which is why I keep coming back to trying to improve people's ability to comment and improve. If the team are aware of a 'soft area' and that the AB or TAG could comment, maybe they'd explicitly suggest so. If the AB or TAG do comment, then when AC Reps vote they could be informed by that comment. |
|
(Discussed without conclusion during AB - October 20, 2022) |
frivoal
removed
the
Agenda+
|
I think we need to open a separate issue (perhaps tying this to #620 ) about communication with/lack of responsiveness from the AC. I believe that the two issues are closely related. |
|
I'm a little confused about how this issue relates to #620; for what it's worth I replied to that issue with a proposal #620 (comment) that includes putting the TAG and AB in the review loop for charters. |
@mnot you would first need to ask "who can determine what wrong is?" If a WG has an AC-approved Charter and active participation, and is working towards stated deliverables, by definition its Charter is not "wrong". I'm troubled by the framing that an approved WG Charter can be "wrong" at all. We should both take some risks and expect a non-zero failure rate in terms of the collective set of WG's delivering their stated objectives in the Charter - deliverables can be late, or abandoned, or even early. The world can change, we don't have to hold someone or some group accountable, i.e. blame them or sanction them in some way. At the same time, we should also allow for a plurality of Charters and WGs covering a wide set of problem areas that affect some community of the web-connected world at large. Attempting to pre-judge how important every WG's work will be over the next 2/5/10/20 years is a fool's errand: there must be an appropriate level of "allowing the market to decide".
+1 @dwsinger The kinds of areas where TAG could usefully weigh in on proposed Charters:
I don't see why AB should be weighing in at all on Charters with any more strength than anyone else who can add a review comment: if an AB member has a view, they should be able to express it of course, but requiring AB consensus before doing so would not be helpful. |
|
The areas where the AB might want to consider reviewing are where there are challenging procedural questions; "what does interop mean for a spec of this type, and how will it be assessed?" for example. This charter raises interesting architectural questions -> TAG |
|
Now W3C are experimenting on the Council (AB+TAG) as the last resort on addressing FOs on charters, which seems a feasible approach in Director-free. If AC has concern on a charter, they can issue FO. That's where AB and TAG can weigh in. Of course, group can ask AB or TAG to review charter if they think it is helpful. But we don't have to make AB and TAG review a formal procedure. |
Currently AC members can formally review charters, seems like the TAG in particular and the AB probably also should be participating in such AC reviews, particularly given their role of technical and process oversight and the importance of charters in the technical direction and process handling of Working Groups.
See #316 (comment)
Proposed simply to allow AB and TAG members to vote on charters in the same survey as AC members.
The text was updated successfully, but these errors were encountered: