Updated review of WebNN API

[dontcallmedom]

(extracted from #771 (comment))

I'm requesting an updated TAG review of WebNN API - previous TAG review: #771

Since the initial Candidate Recommendation Snapshot and the previous TAG review, the Working Group has gathered further implementation experience and added new operations and data types needed for well-known transformers webmachinelearning/webnn#375. In addition, the group has removed selected features informed by this implementation experience: higher-level operations that can be expressed in terms of lower-level primitives in a performant manner, and support for synchronous execution. The group has also updated the specification to use modern authoring conventions to improve interoperability and precision of normative definitions and is developing a new feature, a webmachinelearning/webnn#482, to improve performance and interoperability between the WebNN, WebGPU APIs and purpose-built hardware for ML.

The removal of support for synchronous execution is in-line with TAG's guidance (removal discussed in #531 and moving toward JSPI that is coming finally.

• Explainer¹ (minimally containing user needs and example code): https://github.com/webmachinelearning/webnn/blob/main/explainer.md
• Specification URL: https://www.w3.org/TR/webnn/
• Tests: https://github.com/web-platform-tests/wpt/tree/master/webnn
• User research: N/A
• Security and Privacy self-review²: Self-Review Questionnaire: Security and Privacy webmachinelearning/webnn#119
• GitHub repo (if you prefer feedback filed there): https://github.com/webmachinelearning/webnn/
• Primary contacts (and their relationship to the specification):
  • Anssi Kostiainen, @anssiko, Intel (chair)
  • Dominique Hazael-Massieux, @dontcallmedom, W3C (staff contact)
  • Ningxin Hu, @huningxin, Intel (editor)
  • Dwayne Robinson, @fdwr, Microsoft (editor)
• Organization(s)/project(s) driving the specification: W3C Web Machine Learning Working Group
• Key pieces of existing multi-stakeholder (e.g. developers, implementers, civil society) support, review or discussion of this specification:
  • Chromium comments: https://chromestatus.com/feature/5738583487938560 (in development)
  • Mozilla comments: N/A
  • WebKit comments: N/A (but Apple recently joined the WG)

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• Relevant time constraints or deadlines: expecting to republish as Candidate Recommendation Snapshot in Q1 2024
• The group where the work on this specification is currently being done: W3C Web Machine Learning Working Group
• Major unresolved issues with or opposition to this specification:
• This work is being funded by:

You should also know that...

[please tell us anything you think is relevant to this review]

We'd prefer the TAG provide feedback as open issues in our GitHub repo for each point of feedback

[anssiko]

We've issued a CfC to advance with the CR Snapshot publication mid-March noting in this CfC the TAG delta review is currently in flight. We expect this issue to be looked at in the context of the transition request.

As outlined in this issue, your earlier feedback (removal of sync APIs) has been addressed. The rest of the changes since your last review are evolutionary informed by implementation experience. Specifically, we are not expecting you to do another "full" review.

If the group doesn't hear any concerns from you it plans to proceed with the publication. Thank you for your review comments (#771 (comment)) that motivated the removal of the sync APIs.

[matatk]

Thanks @anssiko, @dontcallmedom for the review request. We were wondering, could you clarify the changes around transformers? We note you've added new data types and operations in support of them (is there a list?) - did you also add/remove any transformers?

[dontcallmedom]

the list of operators added (and the removal of a redundant one) is in webmachinelearning/webnn#478 (comment) based on the detailed analysis made in webmachinelearning/webnn#375 (comment)

[anssiko]

@matatk you may also find the updated use cases for transformers webmachinelearning/webnn#507 helpful -- these use cases motivated the new ops discussed in the above-mentioned issue, also linked from the SOTD.

To provide further context on the removal: one op (squeeze) was removed from the initial list of considered transformer ops because it was found out it can be expressed in terms of an existing lower-level op (reshape) in a performant manner. The emulation path for squeeze is presented informatively in the specification.

Please let us know if you have any further questions.

[anssiko]

For full disclosure and to close the loop on this review:

A new CR Snapshot (history) was published recently. Thank you for your questions and reviews (plural). We've already received two rounds of reviews from the TAG given we've hit the CRS milestone twice for this spec and appreciate your insights and persistence in working with us as we further evolve this specification. We look forward to another delta review with you as appropriate.

If you have further review comments now or at any time do not hesitate to reach out to our group. We will consider all suggestions regardless of the spec milestone we're targeting. We're currently iterating on CRDs and plan to publish a new CRS approximately every 6-12 months.

[matatk]

Hi @anssiko. Thank you for providing the context and info on recent changes, and for the publishing and cadence info. We are still looking into a few things on this review (noting that the 2024-04-29 version is now the current one, as you mentioned). We'll reply on this thread with any additional thoughts.