Get rid of npm and node

[KOLANICH]

npm, node and whole JS ecosystem are fatally flawed. npm tends to put all the submodules into a subfolder (if one doesn't set it explicitly to use the system folder with --global flag) and many JS developers use packages recklessly, which causes bloat. Lot of malicious JS packages have been spotted in npm, some of them were once benign packages bought from their original authors and implanted with malware (python packages don't have better security, but much less malicious packages have been spotted in pypi).

All of these are good reasons to stay away of npm, node packages and whole ecosystem centered around them and do the JS development basing only on ECMA and W3C standards (only ECMA modules (they are supported by browsers), no bundlers, babel, other similar things and no bloaty unneeded frameworks like React or Angular).

So I ask you to reconsider usage of Node and its packages, or at least provide the basics, like ipywidgets, without using anything from npm.

[bollwyvl]

No arguments on many of the points, though the tone might be a bit aggressive.

I am both one of the loudest proponents of standards, and a begrudging detractor of node/npm for end users (#2065). However, for developers, I think this is a bit impractical at this point, which we wouldn't have reached without the millions of nerd-hours in development, testing, ops, etc. that went into what is there.

On this project, we've spent the last few years trying to bring the project more in line with the workflows that are used by the millions of developers that have chosen to build useful things on top of the node ecosystem, and not requiring lots of special tricks, as well as deal with the practical elements of allowing every user to customize their environment with a rich plugin system that is at least as complex as any advanced business web app.

do the JS development basing only on ECMA and W3C standards (only ecma modules (they are supported by browsers)

This is mostly accurate at this point, and we're starting to explore it #5500.

https://caniuse.com/#feat=es6-module

Firefox ESR 60 just started supporting modules a few months ago, and is an important yardstick for compatibility. Otherwise: indeed, the outlook is rosy.

Actually getting these modules to the browser is another story. Even if we went back to square one and rebuilt everything in this repository from the ground up in your standards-based vision, we'd still be looking at thousands of files. So all the authors of Core and any extensions would have to agree on a file structure so that we could resolve them. Then, we'd have to distribute them somewhere. I guess we'd just stand up another global repository and CDN capable of withstanding DDoS attacks. Oh, and pay people to do manual package analysis. Now once you, and all your users, have them, you get them to the browser. This is thousands of http requests, even if (especially if) we rewrite everything. Aha! But what about http/2? Sure, so we'll go to a pre-release version of tornado that supports it, and require every user to run their local app under https, and get letsencrypt certs for everyone. Or instruct users to ignore https certificate warnings.

Next, there would be the question of keeping it all working. So we'd build a new test runtime (can't use one from npm). No typescript, I'm assuming (not a standard) so we'd have to go back and write bound tests for every feature that presently is known to just work.

Then we'd document it. I guess write it by hand.

Anyhow, perhaps a bit extreme, but these are some of the issues we're dealing with. If you have actionable steps we can take to start dealing with some of these issues, they would be greatly appreciated.

So I ask you to reconsider usage of Node and its packages, or at least provide the basics, like ipywidgets, without using anything from npm.

This is a honking good idea. The pattern, first demonstrated by https://github.com/jonmmease/jupyterlab_delux shows how a developer can build a custom jupyterlab and distribute it in another package manager format for a tailored purpose, not requiring node at all on the user machine, who kinda gets what they get. The approach demonstrated there uses conda, but could be done with pip and an out-of-band node, which could then be consumed downstream.

We're starting to explore that with other things like:

• https://github.com/bollwyvl/robotlab (very minimal, just some syntax highlighting and widgets)
• https://github.com/bollwyvl/cadlab (widgets and pythreejs and bqplot and itkwidgets and...)

Really the only thing missing is an end-user, in-browser "stage 0" UI that would let you switch between these, rather than requiring restarting the service with a full path, or shipping a custom entry point. The other would be a "blessed" set of extensions, which we feasibly could curate for inclusion. The biggest challenge here is figuring out how to gate it to manage end-user file size, so that the load doesn't take any longer than it already does.

[KOLANICH]

Thank you for the answer and for the links.

This is thousands of http requests, even if (especially if) we rewrite everything.
Aha! But what about http/2? Sure, so we'll go to a pre-release version of tornado that supports it, and require every user to run their local app under https, and get letsencrypt certs for everyone.

Not needed for a local app, file:/// can be used. No http requests at all involved for static content. For interaction with API CORS can be used.

[minrk]

Thanks @bollwyvl for the explanation of why we can’t and shouldn’t abandon node and npm at this point. Installing bundles via conda is an option for those who want to avoid node at runtime, and targeting ES6 modules is a separate and ongoing discussion, unrelated to node and npm. I think this can be closed.

[jasongrout]

Thank you @bollwyvl and @minrk for elaborating on this.