Use modern WebIDL and Infra standard conventions

[anssiko]

ℹ️ This is a meta issue. See the (living) task list for pointers to specific issues and PRs.

The WebIDL spec recommends algorithmic method steps for defining operations: https://heycam.github.io/webidl/#method-steps

This spec currently uses a WebGPU spec inspired convention that differs from this best practice. We should consider adopting the "method steps" convention.

(Here's one example of method steps in context of the WebNN API spec:
https://webmachinelearning.github.io/webnn/#dom-ml-createcontext)

[anssiko]

I labeled this issue with "cr" implying we should address this issue before our Candidate Recommendation publication to satisfy "The document is considered complete and fit for purpose" requirement.

[anssiko]

Updated patterns for defining getters, setters, methods and constructors:

https://webidl.spec.whatwg.org/#getter-steps
https://webidl.spec.whatwg.org/#setter-steps
https://webidl.spec.whatwg.org/#method-steps
https://webidl.spec.whatwg.org/#constructor-steps

[anssiko]

@zolkis we discussed this issue on our call with a recommendation on tasks where to start:
https://www.w3.org/2022/09/22-webmachinelearning-minutes.html#t04

This issue is tagged as "cr" so we'll allocate time to discuss this on our bi-weeklies. Please bring any open issues or questions either to this issue or to the call. There is likely a lot of details that need to be clarified as you translate the current text into an algorithmic form. Thanks for your contributions!

[anssiko]

(API review questions that'll also inform this work on method steps discussed in #298)

[anssiko]

(Renamed the issue to better reflect the scope of this work.)

@zolkis I will document here the spec sections where we need to do editorial work such as add algorithmic method steps, add internal slots, use getters, setters, define methods and constructors using the modern WebIDL and Infra standard conventions.

This helps the WG understand the scope of this effort and areas where contributions are welcome. I would focus this task on editorial changes and only propose normative changes when they are needed to fix a spec bug, or to clarify an aspect where the spec and implementation disagree.

Integration branch

January 2024 update: Review feedback status (🟡 WIP, 🟢 fixed) updated.

August 2023 update: The big PR #446 has been merged. The remaining "review feedback" issues to be addressed separately.

June 2023 update: We use the zk-conventions-integration integration branch for this conventions alignment task.

PRs related to this conventions update will be merged to this integration branch. We merge the integration branch into main when all the related issues are addressed in the integration branch to keep main in a cohesive state and changes to it atomic.

Issues and PRs

zk-conventions-integration

Global changes

• MLActivation, MLOperand, input(), constant(), batchNormalization(), clamp(), concat()
  • fixed by Align algorithms defined so far #405
• Style changes to hide large algorithms and examples, frames for internal slots, etc.
  • fixed by Add stylistic changes #401
• Any bugs identified after merge of the PRs enumerated in this issue
  • fixed by Fix merge errors #437
• 🟡 review feedback Use deep copy for setting internal slots? #395
• 🟢 review feedback Improve algorithmic steps definitions #447
• 🟡 review feedback Use web spec best practices #450
• 🟡 review feedback Reserve asserts only for non-obvious things #455

MLOperand, MLActivation

• Add internal slots to MLOperand, MLActivation and basic algorithms #336
  • fixed by Add internal slots to MLOperand, MLActivation and basic algorithms #336 (comment)
• Clarify intermediate values (activations) vs MLActivation vs operations #335
  • fixed by ???
• 🟢 review feedback (bikeshed naming) MLOperandDataType #454
  • fixed by MLOperandType -> MLOperandDataType #464

MLGraphBuilder

• Add the build() method, the build() and buildSync() algorithms, MLBufferResourceView dictionary definitions
  • fixed by Add the build() and buildSync() algorithms #406
  • 🟡 review feedback Add traversal algorithm to buildSync() #448

Ops

• The batchNormalization() method
  • Improve batchNorm #334 fixed by Add the batch normalization algorithm #397
• The clamp() method
  • Add the clamp() algorithm #347 fixed by Clamp algorithm #399
• The concat() method
  • fixed by Add algorithm to concat() #398
• The conv2d() method
  • fixed by Add the conv2d and convTranspose2d algorithms #407
  • 🟢 review feedback Add missing conv2d validation steps #439
  • fixed by Fix #439: add missing conv2d() and convTranspose2d() validation steps #441
• The convTranspose2d() method
  • fixed by Add the conv2d and convTranspose2d algorithms #407
• Element-wise binary operations
  • fixed by Add element-wise binary operation algorithms #408
• Element-wise unary operations
  • fixed by Add element-wise unary algorithms #409
• The elu() method
  • fixed by Add the elu() algorithm #410
• The gemm() method
  • fixed by Add the gemm() algorithm #411
• The gru() method
  • fixed by Add the gru() and gruCell() algorithms #412
• The gruCell() method
  • fixed by Add the gru() and gruCell() algorithms #412
• The hardSigmoid() method
  • fixed by Add the hard-sigmoid algorithms #413
• The hardSwish() method
  • fixed by Add the hard-swish algorithm #414
• The instanceNormalization() method
  • fixed by Add the instance normalization algorithm #415
• The leakyRelu() method
  • fixed by Add the Leaky RELU algorithm #416
• The matmul() method
  • fixed by Add the matmul() algorithm #419
• The linear() method
  • fixed by Add the linear function algorithm #417
• The lstm() method
  • fixed by Add the LSTM and LSTM Cell algorithms #418
• The lstmCell() method
  • fixed by Add the LSTM and LSTM Cell algorithms #418
• The pad() method
  • fixed by Add the pad() algorithm #420
• Pooling operations
  • fixed by Add the pooling algorithms #421
  • 🟡 review feedback Fix description and formula in pooling operations #449
• The prelu() method
  • fixed by Add the prelu() algorithm #422 depends on Add element-wise binary operation algorithms #408
• Reduction operations
  • fixed by Add the reduction algorithms #423
  • 🟢 review feedback The default values of |options|.{{MLReduceOptions/axes}} should be {0, ...., N-1} rather than null. #451
• The relu() method
  • fixed by Add the relu() algorithm #424
• The resample2d() method
  • related Define algorithms for dictionaries with lists as default values #211
  • fixed by Add the resample2d() algorithm #425 and Wording change: Prefix literal lists with "the list" #515
• The reshape() method
  • fixed by Add the reshape algorithm #426
  • 🟡 review feedback Reshape: what if |newShape| is a scalar number #452
• The sigmoid() method
  • fixed by Add the sigmoid algorithm #427
• The slice() method
  • fixed by Add the slice algorithm #428
• The softmax() method
  • fixed by Add the softmax algorithm #429
• The softplus() method
  • fixed by Add the softplus algorithm #430
• The softsign() method
  • fixed by Add the softsign method #431
• The split() method
  • fixed by Add the split algorithm #432
• The squeeze() method
  • fixed by Add the squeeze() algorithm #433
• The tanh() method
  • fixed by Add the tanh algorithm #434
• The transpose() method
  • fixed by Add the transpose algorithm #435

main

• Validate MLContext #308
  • fixed by Fix #308: refer to MLContext validation steps from MLGraphBuilder #332
• Add MLGraphBuilder constructor steps
  • fixed by Add constructor steps to MLGraphBuilder #309
• Improve spec prose on "exists" #311
  • fixed by Fix #311: use [=map/exist=] instead of exist #313
• Improve createContext method steps
  • fixed by MlContext improvements #310
• Add method steps for createContextSync
  • fixed by MlContext improvements #310
• Handle errors according to Web IDL conventions #314
  • fixed by Fix error handling #315
• Add the builder.input() method steps
  • fixed by Add the builder.input() method steps #364
• Add the constant() method steps
  • fixed by Add the constant() method steps. #365
• Review sync vs async compute differences #316
  • fixed by Rework the sync async algorithms based on PR 323 #329

Other

Proposals for normative changes that were unearthed as part of the editorial work.

• API simplification: context types, context options, createContext() #302
• API simplification: context owns builder, graph becomes internal slot #303
• Clarify/simplify graph execution on GPU / MLCommandEncoder #333
• Ensure the programming model section reads well and is up to date with the latest changes

[zolkis]

This issue is good aggregate tracker. Issues might be spanned on need, if the discussion is expected to be contained, or too big.

[zolkis]

I have been developing the features from the TODO in separate private branches in my fork. Some of them were published in PRs (some merged, some open). Most of them are still in private branches in my local repo. I will push them to my fork (without making PRs) so they become visible.

I was asked to present all upcoming changes in one fork/branch, so that all changes could be reviewed with the full context.
The tip of my integration branch, containing all changes from the TODO above, is now visible on https://zolkis.github.io/webnn/
(index.bs and index.html are being kept up to date, rebased from main and including any upcoming fixes to my changes).
The changes from the current open PRs (clamp, concat, batch norm, sync-async execution, input fix) have been included.

@huningxin : the improved steps that we worked out in the clamp PR (explicit steps to create platform objects for operands and activations) has been included. Should we next update the rest of the algorithms based on that?

@wchao1115 you can take a look on the current state, and if you have any feedback, it would be good to fix and include those as early as possible (so far you had feedback on types used in internal slots, not using note style for argument descriptions and remove styling from titles). I have fixed the latter two.

On how to make feedback: either in one or more issues, or I could make PRs to an upstream integration branch to be made for this purpose (e.g. zolkis-algorithms-integration or any other preferred name in the upstream repo) from which we could rebase/PR to main when the time comes. Could someone with access rights create that target integration branch?

[wchao1115]

Thanks @zolkis. Can you please remove your current set of related PRs in the mainline PR queue, now that you have the entire change hosted in your fork?

With the expectation that the change in this fork will be scoped to

1. A global doc style change, and
2. Additional implementation sections as required by the convention,

the reviewers will review the entire change in the fork and once approved, merge it all at once when it's all ready to go. Any additional change after the merge is expected to be minimal and scoped to adjusting to the set of changes previously merged. Your private fork should therefore no longer be needed once the merge happens.

This request was made with the intention to provide the reviewers with enough context behind the whole change, which will unavoidably affect the entire spec as a whole and not incremental in nature.

The reviewers expect this entire change to be atomic, meaning that it will leave no follow-up action items after it is merged to the main content. It is important to remember that the change in this private fork is strictly scoped to the two types of changes noted above in order to avoid having a separate parallel content that is in the unmergeable state to the mainline content.

[zolkis]

@wchao1115: affected PRs will be closed when the alternative will have been set up. Right now it's not yet finished setting up.

the reviewers will review the entire change in the fork

How exactly these changes are proposed to be reviewed?
Reviews need PRs.
PRs need a target branch, which (if not the main branch) is currently missing. I don't have access rights to make it. I guess @anssiko can make it.

IMO the most practical way to review and integrate these changes, without affecting the main branch, is using normal github practices, as follows:

• create an integration branch in this (upstream) repo
• I will make PRs to that branch from my fork (and redirect the related, currently open PRs there, after which they will be closed on the main branch). With all PRs, the full set of changes are always visible at https://zolkis.github.io/webnn/ , solving the problem of viewing the whole context.
• There can be also a single big PR with multi-thousand lines of changes, which is very impractical to review (imagine hundreds or thousands of comments in the PR) and is not really a recommended github practice.
• These PRs can be merged to the integration branch one by one.
• Optionally, issues can be created that refer to that integration branch and can have their own label.
• When all changes will be done and approved, then a single PR can be made to the main branch, so the whole change set is integrated "atomically".
  This would be simple and flexible, would solve the concerns, remain practical for reviewing, while following github regular practices.

About the content of the changes: bear in mind that these are actually adding algorithms, i.e. normative part of the spec. It might result in effects on the current text (so far not much, only arguments descriptions have been moved). Stylistic changes come as needed, like adding new (sub)sections and titles, collapsible algorithm blocks, visual framing of certain blocks (internal slots, validation, algorithms, etc.) IIRC no other changes were needed, and the existing changes fit pretty well in your requested policies. Any further, deeper changes can be done later via issues and PRs in the main branch (regular spec work).

[anssiko]

The zk-conventions-integration integration branch is now ready for review.

These changes align the entire specification with modern specification conventions and add stylistic improvements on top that make navigating this specification more delightful experience.

The following resources are made available to the group to assist in this review task:

• Preview of zk-conventions-integration
• Diff between main and zk-conventions-integration (note: diff is slow to load)
• Issues addressed and PRs merged to zk-conventions-integration
• Commits ahead of main

Thanks @zolkis for this significant effort!

[wchao1115]

@anssiko and @zolkis, do we know the ETA to merge zk-conventions-integration branch to main? I've spent some good amount of time on the change, and as far as I can tell, I'm signed off on it. If there's additional change to be made, please let me know how you plan to make them, either in this staging branch, or directly targeting main as needed, and if so, by when. Thanks!

[zolkis]

@wchao1115 we agreed in the WG meeting to spend the time until the next WG meeting for proofreading, sanity check etc. Our implementation team helps in doing that, and I am making extra effort to handle all comments possible, and make issues for the larger or more complex changes to be handled later. The target is that by the next WG meeting, it will be in good enough shape to be merged, and then handle the rest soon after.

[anssiko]

@wchao1115, we expect to merge latest after our next meeting 24 Aug. We have a good momentum with reviews with an extended team pulled in to help with PR #446 review and @zolkis is focused on addressing the review comments directly in the integration branch (see commits). We open new issues for proposed changes more appropriate to be addressed after this big PR has been merged.

Thanks for your efforts in driving this to completion, everyone!

[anssiko]

The big PR #446 has been merged, thank you all! Your contributions have been acknowledged in the specification.

The modernized specification is now published at its official location (https://www.w3.org/TR/webnn/) as a Candidate Recommendation Draft to signal there have been significant changes to the previous published document 🚀

I'll leave this tracker issue open for a while to help the WG take to completion the few "review feedback" issues (noted in my comment above) we wanted to address on top of this big change.

[anssiko]

Per our discussion on today's call, I'd propose to retire this tracker issue and move to a GH label-based tracking of standards convention updates.

I've asked @inexorabletash to help me with this task. Let me know when you've absorbed the useful information from this big meta issue so we can close this.

[inexorabletash]

Thanks @anssiko

So far as I can tell, all of the topics mentioned here are either already fixed or are tracked by open issues. Here's how I'd categorize what's still open:

• Label these with something like "operator-specific"

  • Fix description and formula in pooling operations #449
  • Improve batchNorm #334
  • Add the clamp() algorithm #347
  • Reshape: what if |newShape| is a scalar number #452

• Label these with something like "spec quality"

  • Use deep copy for setting internal slots? #395
  • Use web spec best practices #450
  • Reserve asserts only for non-obvious things #455
  • Define algorithms for dictionaries with lists as default values #211

• Finally, label these with something like "normative" and/or "spec completeness" and/or "discussion needed". I think this is the most important category of issues.

  • API simplification: context types, context options, createContext() #302
  • API simplification: context owns builder, graph becomes internal slot #303
  • Clarify/simplify graph execution on GPU / MLCommandEncoder #333
  • Add traversal algorithm to buildSync() #448
  • File a new issue to track: Ensure the programming model section reads well and is up to date with the latest changes

Further labels e.g. "PR needed" "has PR", "needs WPT", "has WPT" might be useful but that can be discussed separately.

[anssiko]

@inexorabletash much thanks for the label proposals and triage.

I think we could document the proposed labels and their semantics in a .md file alongside code conventions and contrib guidelines. An idea up for grabs, PRs welcome :)

[anssiko]

Thanks everyone for your work on this WebIDL and Infra standard conventions alignment effort that has resulted in a significantly improved and more precise specification. Special thanks to @zolkis and @inexorabletash for this effort!

I opened #549 and triaged the remaining open issues referenced here. We're now ready to close this tracker 👋

From now on standards conventions related issues can be found via this label: conventions