Web machine learning @ W3

[DanielMazurkiewicz]

Guys (and Girls? :-) ),

There is W3 community that tries to work out a standard for machine learning for web. It is great idea, but I'm strongly afraid that they will make another crappy standard that we all will have to live on later. I find Brain.js API very close to something I would expect from standards for web scripting - simple to use, simple to learn, doing its work. I'm worried also that W3 group will overcomplicate standard so we will have to wait for it couple of years to see in browsers. I think it would be much better to make firstly quick basic ML standard and then second version that extends it with all kind of "low level" and "super scientific" ML capabilities.

I want you all to get involved to avoid mentioned problems. You're doing great work with Brain.js! With your experience and expertise (or just voting power) we could push it forward so standard could be quickly adopted as something that is close to Brain.js. Something that developers could quickly adopt without tons of code and studying.

If you decide to get involved go to:
https://www.w3.org/community/webmachinelearning/
https://github.com/webmachinelearning/webnn/issues

I personally see ML api for www as something like that: https://github.com/DanielMazurkiewicz/WebAI, but anything close to Brain.js should be simply good.

Don't wait for the standard, make it! :-)
Cheers!

[robertleeplummerjr]

Hi @DanielMazurkiewicz! This does sound awesome. Have you looked at https://github.com/BrainJS/brain.js/wiki/Roadmap as well? This is our 2.0 roadmap. We're currently implementing it, and we're starting planning on v3, here: #317.

I'm interested in this, but it will take more energy than just me. If we are going to do this, we need to form a team of willing volunteers, that can help respect the ideas of both simplicity and the needs that such a standard would require.

Just curious, and wanted to start the discussion.

[DanielMazurkiewicz]

Hi @robertleeplummerjr !

Have you looked at https://github.com/BrainJS/brain.js/wiki/Roadmap as well? This is our 2.0 roadmap. We're currently implementing it, and we're starting planning on v3, here: #317.

Yes, I've seen it. I don't know how about you, but I would like NN to be easily exchangeable not only within JavaScript runtimes, but also with other programming languages, and for that JSON format of storing NN architecture (also for recurrent NN) is much better in my opinion. So as for the new versions of Brain.js I would prefer more JSON descriptive formats instead of any sort of callback functions or somewhat "layer objects". That is why I proposed to feed NN constructor with property "layers" that looks like that: layers: [8, 14, 8]. It gives an opportunity to introduce later named layers with sort of pipes for any sort of recurrent nets. For example:

DanielMazurkiewicz/WebAI@a5578b1

As for using OpenCL plans, that sounds great!
In couple of weeks I plan to start to work on vulkan compute module on top of NAPI for nodejs. With vulkan it should be easy to use all available GPUs simultaneously - even from multiple vendors. Coding NN directly in SPIR-V should give performance gains even over OpenCL. Moreover, I think it might be even bigger performance gain if making entire training process without need of involving CPU, SPIR-V should be able handle that. Moreover - intel offers spir-v to cpu efficient compiler that uses CPU accelerations. So once written same code could be used very efficiently on CPU and GPU.
So if I could wish myself something here it would be brain.js api ported to spir-v and available as one of brain.js backends (openCl can be another one) :-)

If we are going to do this, we need to form a team of willing volunteers, that can help respect the ideas of both simplicity and the needs that such a standard would require.

Totally agree! For me it is not a question "if" - I've already started supporting and promoting brain.js like api for W3 NN standard. I think the more people supporting brain.js will sign to W3 to that project, the more chances to get the standard falling into Simplicity and Performance First phrase. It doesn't require much to sign and sometimes a share opinion.

Is it ok for you to place sort of appeal to participate in standard preparations in read.me of Brain.js? So users of brain.js could also share their opinion?

[robertleeplummerjr]

Yes, I've seen it. I don't know how about you, but I would like NN to be easily exchangeable not only within JavaScript runtimes, but also with other programming languages

You share my desire, and I feel a large part of the community's as well. I've always wanted to do a quick port to c++, c#, java, and php, just for the sake of offering an easy api to everyone.

So as for the new versions of Brain.js I would prefer more JSON descriptive formats instead of any sort of callback functions or somewhat "layer objects".

Short answer: I love this strategy! This actually is already mostly adopted in the fromJSON method, but I agree, it should be very easy to invoke. As for instantiation, that'd easily be part of the options. The main thing is that GPU or CPU (or other?) be accessible quickly without much effort to get at or define the thing desired.

Further reading: Something I want to be clear about with the roadmap for v2 and why v2 is important: It describes one of the simplest means of instantiating the net directly, as I'm sure you know. This allows one to even build their own layers. The layers, in other words, aren't dependent on the net, and the net on the layers. You can shim in your own, or use what is there.

In couple of weeks I plan to start to work on vulkan compute module on top of NAPI for nodejs

Why not become familiar with gpu.js, the underlying engine with brain.js, and provide an architecture layer that does this, then offer it to the world? That would be awesome! There are already 3 backends, with one coming (Headless WebGL) here: https://github.com/gpujs/gpu.js/tree/develop/src/backend

Simple js in, GPU computation out. Wicked fast, wicked easy.

Coding NN directly in SPIR-V should give performance gains even over OpenCL.

OpenCL in GPU.js was seen as simply another backend, or driver. That isn't to say you that if we had an OpenCL module, everything node side would have to go through OpenCL. It is simply a driver. If another backend was provided, we'd simply use that, like Vulken. All the hard stuff is worked out, like type detection, javascript transpilation to whatever language you like, etc.

So once written same code could be used very efficiently on CPU and GPU.

GPU.js already does this.

For me it is not a question "if"

Very encouraging to hear!

I think the more people supporting brain.js will sign to W3 to that project, the more chances to get the standard falling into Simplicity and Performance First phrase.

I agree! One of the biggest needs is being open to new ideas from the community. They have some pretty fantastic ones. Also, not to be afraid of sticking up for an idea with logic backing it. Although simple, it has been one of the neatest things to see in practice for brain.js and open source.

Is it ok for you to place sort of appeal to participate in standard preparations in read.me of Brain.js? So users of brain.js could also share their opinion?

I welcome it.

[DanielMazurkiewicz]

Further reading: Something I want to be clear about with the roadmap for v2 and why v2 is important: It describes one of the simplest means of instantiating the net directly, as I'm sure you know. This allows one to even build their own layers. The layers, in other words, aren't dependent on the net, and the net on the layers. You can shim in your own, or use what is there.

We are on same page here. But if it comes to standardization that additionally should work among variety of different hardware (cpu, gpu, dsp, fpga...) I think it is easier to stick to JSON that can be quite easily "compiled" to work on selected hardware, than having a custom JS code that will have to run somewhere in between hardware and JS runtime. I think better to focus on JSON descriptive format to be as flexible as possible (if it comes to standardization of course). In some later phases of upgrading standard it could add some more sophisticated techniques that could use custom JS inside standardized ML.

Vulkan and deep down performance is entire big another topic :-) , so maybe lets split here and summarize what I/we expect from W3C standardized API:

1. Simple to use, simple to learn API (lets make a proposal draft of it or evolve my proposal: https://github.com/DanielMazurkiewicz/WebAI)
2. Allowing to use all available hardware and hardware accelerations simultaneously
3. Neural Network architecture should be described entirely in standardized JSON (keeping in mind also a flexibility, which is a possibility to mimic in JSON all sort of NN architectures: https://towardsdatascience.com/a-deeper-understanding-of-nnets-part-1-cnns-263a6e3ac61) - NN could be easily exchanged not only between javascript based runtimes, but also between other languages / ML runtimes / ML macro compillers / so on...
4. Safety that will guarantee JS runtime exclusive access to its NeuralNetwork object
5. Splitting standardization process to parts, so it could be quickly adopted and later extended:
   a) Basic ML functionality/API - widely used high level well proven algorithms (keeping in mind that it is gonna be extended)
   b) Extended ML functionality - extending JSON architecture capabilities, training algorithms, access to neural network internals, all highly scientific orientated API, custom JS injections...
   c) Providing standardized set of ML models (and dedicated APIs) that could be widely used (like body parts recognition - face, hands, pose, gestures ...; characters recognition, voice recognition, shapes recognition - triangle, rectangle, circle, square ...; objects recognition; ... TBD)

[DanielMazurkiewicz]

Just a small reminder to anyone who want to share his knowledge/opinion/give likes/dislikes ;-):
webmachinelearning/webnn#3

[robertleeplummerjr]

Ty @DanielMazurkiewicz for the update!