[Proposal] Publish /lib/js/* as an npm package

[jchavarri]

The seems to be no way at the moment to use BuckleScript generated JS code in apps or libraries without forcing consumers to include bs-platformin their dependencies, or authors to bundle their code.

A real world case can be found in VS Code extensions (node.js apps). Publishing a VS Code extension with bs-platform in package.json would mean hundreds of MBs are sent to Microsoft publishing servers. Actually, any regular node.js app will have a similar problem when being deployed.

I'm pretty sure this has been discussed before, but I'm interested in the minimum work required to make this happen:

• Use lerna or a custom script to automate this new package publishing
• Add an option in bsconfig.json package-specs so the output JS requires from this new package optionally. Or alternatively, make the separate package the only option.
• Change the compiler so the new package name is used on each module that depends on /lib/js

Happy to help with this.

Related:

• Generated code requires modules from bs-platform, forcing an extra production dependancy #2082: Generated code requires modules from bs-platform, forcing an extra production dependancy
• Treat stdlib as normal package #1187: Treat stdlib as a normal package
• [Discussion] Start of the stdlib infra #1860: [Discussion] Start of the stdlib infra

[bobzhang]

I tend to make stdlib a separate package, if we make it a separate package, how do we make sure stdlib is always there for the convenience of users?

If we extract alone, I am not sure what's the best way to do it since the path may be incorrect, there are two work arounds:

1. use rollup
2. has a script to remove those unneeded binaries when publish to production

I am unclear what's the best way to do it

[jchavarri]

I would definitely vote for 1 (with rollup or any other tool), because there are myriads of edge cases between OSs, environments, etc. that npm is already solving, and these cases would need to be considered with option 2.

The minimal approach that I can think of would be something like this:

• Create a new package (let's say bs-stdlib), that publishes the contents of /lib/js to npm
• Add /lib/js to .npmignore so that folder doesn't get included in bs-platform package any longer
• Add bs-stdlib to bsb/templates package.json files (as dependency).
• Require bs-stdlib in the generated js files

There's the issue of users having one version for bs-platform and another one for bs-stdlib, but I guess each dev is responsible of keeping the projects updated. To keep things simple, the best would be probably to lock bs-stdlib versioning to be the same as bs-platform.

cc @chenglou @bassjacob @jaredly as you've discussed this topic on Discord at different times in the past.

[TheSpyder]

I’m greatly in favour of having a separate stdlib npm module. I’m hoping to start adding BuckleScript generated code into my es6 projects before the end of the year and having code that doesn’t depend on bs-platform is on my list of requirements before I can do that.

[bobzhang]

If you make stdlib a separate package, how do you make sure it’s always included when you start a new project?

[jaredly]

If we maintain a separate repo (say bs-core) that is a mirror of the stdlib w/ tags for versions, then when someone wants to deploy to a nodejs env, they replace "bs-platform": "1.2.3" with "bs-platform": "git+https://github.com/Bucklescript/bs-lite.git#1.2.3"

[TheSpyder]

Well it would go in the bsb init template, which is a good start.

There’s already a custom error message for when bs-platform isn’t found, couldn’t a new one be added for when bs-core isn’t found?

Or maybe bs-core is what we tell devs to install and it has a peer dependency on bs-platform 🤔

[jaredly]

Here's my proposed solution in action:
If you replace, in your package.json

  "bs-plaftorm": "1.8.3",

with

  "bs-platform": "git+https://github.com/jaredly/bs-core#1.8.3",

npm will install my hollowed-out version of bs-platform (that doesn't try to build the ocaml compiler) into node_modules/bs-platform, and all your bucklescript-produced javascript files should Just Work™
@TheSpyder want to try that out?

[TheSpyder]

My code compiles as es6-global, so no that doesn't work ;)

I think we're all on roughly the same page; we want to be able to compile to JS, check that JS in, and then have other developers work with the codebase without requiring them to install bs-platform.

What you've done is apparently similar to how facebook does it, according to a quick chat I had with @chenglou on discord they maintain their own custom copy of the stdlib for this purpose.

[jchavarri]

I've created a proposal more or less inline with what @jaredly is proposing in #2171.

On prepublish the main bs-platform it would automatically build the whole JS lib (cd jscomp && make world) and then publish it as the bs-stdlib package. It checks both packages have the same version + update the templates to include bs-stdlib as dependency.

The downside is that if / when this was shipped, the existing projects would need to manually include bs-stdlib as dep.

[jaredly]

Bucklescript could easily check package.json to verify that the dep is there & fail with a helpful error message. Doesn't seem like too bad of an upgrade path

[bobzhang]

how about encouraging people to not use stdlib by default? the ocaml stdlib is not very attractive, we can basically copy a stdlib as it is and evolve separately

[TheSpyder]

That seems separate to this goal; you'd have as much problem referring to any replacement stdlib in compiled JS as you would with the default implementation.

Unless you mean leave the default as-is for compatibility and require people to opt in to the replacement stdlib to avoid depending on bs-platform?

[bobzhang]

Unless you mean leave the default as-is for compatibility and require people to opt in to the replacement stdlib to avoid depending on bs-platform?

Yes, I think it is not a bad idea. you have more flexibility to evolve the api.
It is non trivial work to factor out stdlib,

• stdlib Makefile is a mess (not portable across OSes, I used a hack to make it work on Win)
• stdlib Format is not real Js friendly, stdlib polluted namespace too much
• stdlib requires special runtime support (for those c polyfill)

With a new stdlib alternative, we can use bsb for that stdlib and introduce conditional compilation etc. The downside is that it may not work with existing ocaml libraries, but we still keep the old ocaml stdlib for compatibilitiy reasons