Make meteor publish-for-arch unnecessary in most cases

[benjamn]

Currently, if you are a Meteor package author, and your package depends on an npm package that uses native modules, running meteor publish will prompt you to build and publish your package on Mac (64-bit), Linux (32-bit and 64-bit), and Windows (32-bit), using the meteor publish-for-arch command.

Though MDG provides free build machines via https://build.meteor.com and meteor admin get-machine, it would be an overstatement to say that building packages on these machines (especially Windows) is easy or quick. To make matters worse, the upgrade to Node 4 in Meteor 1.4 imposes additional burdens for package authors who want their packages to work for Meteor 1.4 developers as well as pre-1.4 developers.

Fortunately, since Meteor 1.3.3, it has been technically possible to avoid running meteor publish-for-arch, because Meteor 1.3.3+ has the ability to recompile binary npm dependencies found in Meteor packages. You could trigger this behavior by running METEOR_FORCE_PORTABLE=1 meteor publish when you published your package, though that was never recommended, and is likely not documented anywhere (except in the code).

This pull request disables the pre-publish portability check, thereby deferring compilation to package installation time. This is similar to setting METEOR_FORCE_PORTABLE=1 by default. Automatic rebuilding requires the end-user developer to have a compiler toolchain installed on their development machine, whereas meteor publish-for-arch was meant to avoid that requirement. However, having a basic compiler toolchain is already a soft prerequisite for using many packages from npm, and it's easier for the end-user developer to configure a toolchain once than it is for a package author to do that work four (or even eight) times, every time she publishes the package.

Once this is merged and released (likely in Meteor 1.4.1), it will still be possible to pre-build binary dependencies and publish them using meteor publish-for-arch if you really want to, either by specifying an older version of Meteor when you publish (e.g., meteor --release 1.4 publish) or by setting an environment variable: METEOR_ALLOW_NON_PORTABLE=1 meteor publish.

We believe this change will reduce demand for https://build.meteor.com sufficiently that we may consider turning off the build farm. If you are a package author who uses the build farm, we would love to hear your thoughts on this change, especially if you think you would continue running meteor publish-for-arch after you are no longer expected to do so.

[sebakerckhof]

As a package author this is a very welcome change. Just a bit afraid I'm going to see lots of issues when people who don't have the necessary toolchain installed get errors. Maybe meteor tool could show some useful messages to guide these people to the installation instructions.

Anyway, If I understand correct, I can already do this today by running METEOR_FORCE_PORTABLE=1 meteor publish and it will work for meteor 1.3.3+ users?

[benjamn]

Anyway, If I understand correct, I can already do this today by running METEOR_FORCE_PORTABLE=1 meteor publish and it will work for meteor 1.3.3+ users?

This is almost true, except that we need to record a bit more information in .meteor-last-rebuild-version.json so that packages will be rebuilt reliably. That's what 1d47395 does. Without that change, .meteor-last-rebuild-version.json files look the same for packages built on different platforms (Mac, Linux, Windows), as long as the Node/V8/etc. versions match. So, for example, a Linux user would not always rebuild a package that you published using a Mac. Adding process.{platform,arch} to .meteor-last-rebuild-version.json fixes that.

[tmeasday]

See #7634

[mitar]

We believe this change will reduce demand for https://build.meteor.com sufficiently that we may consider turning off the build farm. If you are a package author who uses the build farm, we would love to hear your thoughts on this change, especially if you think you would continue running meteor publish-for-arch after you are no longer expected to do so.

It would be OK if it would work.

But I think you should keep binaries around for packages which have heavy dependencies. Like pdf.js package I have is pretty tricky to get all dependencies installed.

[mitar]

If you want to deprecate build machines as well, maybe you should document better what are configuration of each machine so that people can run it themselves. (You are saying in documentation that it is important that they are precisely configured.) Maybe you could even provide virtual images or vagrant images to build with a command.

[c9s]

Can we make this option inside the package.js to mark a package as portable? because sometimes it could be forgot to pass the option.