New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make `meteor publish-for-arch` unnecessary in most cases #7608

Merged
merged 6 commits into from Aug 11, 2016

Conversation

Projects
None yet
5 participants
@benjamn
Member

benjamn commented Aug 8, 2016

Currently, if you are a Meteor package author, and your package depends on an npm package that uses native modules, running meteor publish will prompt you to build and publish your package on Mac (64-bit), Linux (32-bit and 64-bit), and Windows (32-bit), using the meteor publish-for-arch command.

Though MDG provides free build machines via https://build.meteor.com and meteor admin get-machine, it would be an overstatement to say that building packages on these machines (especially Windows) is easy or quick. To make matters worse, the upgrade to Node 4 in Meteor 1.4 imposes additional burdens for package authors who want their packages to work for Meteor 1.4 developers as well as pre-1.4 developers.

Fortunately, since Meteor 1.3.3, it has been technically possible to avoid running meteor publish-for-arch, because Meteor 1.3.3+ has the ability to recompile binary npm dependencies found in Meteor packages. You could trigger this behavior by running METEOR_FORCE_PORTABLE=1 meteor publish when you published your package, though that was never recommended, and is likely not documented anywhere (except in the code).

This pull request disables the pre-publish portability check, thereby deferring compilation to package installation time. This is similar to setting METEOR_FORCE_PORTABLE=1 by default. Automatic rebuilding requires the end-user developer to have a compiler toolchain installed on their development machine, whereas meteor publish-for-arch was meant to avoid that requirement. However, having a basic compiler toolchain is already a soft prerequisite for using many packages from npm, and it's easier for the end-user developer to configure a toolchain once than it is for a package author to do that work four (or even eight) times, every time she publishes the package.

Once this is merged and released (likely in Meteor 1.4.1), it will still be possible to pre-build binary dependencies and publish them using meteor publish-for-arch if you really want to, either by specifying an older version of Meteor when you publish (e.g., meteor --release 1.4 publish) or by setting an environment variable: METEOR_ALLOW_NON_PORTABLE=1 meteor publish.

We believe this change will reduce demand for https://build.meteor.com sufficiently that we may consider turning off the build farm. If you are a package author who uses the build farm, we would love to hear your thoughts on this change, especially if you think you would continue running meteor publish-for-arch after you are no longer expected to do so.

benjamn added some commits Aug 6, 2016

Include process.{platform,arch} in .meteor-last-rebuild-version.json.
This supports rebuilding binary npm packages published by completely
different architectures, not just different versions of Node.

@benjamn benjamn added this to the Release 1.4.1 milestone Aug 8, 2016

@benjamn benjamn self-assigned this Aug 8, 2016

@sebakerckhof

This comment has been minimized.

Contributor

sebakerckhof commented Aug 8, 2016

As a package author this is a very welcome change. Just a bit afraid I'm going to see lots of issues when people who don't have the necessary toolchain installed get errors. Maybe meteor tool could show some useful messages to guide these people to the installation instructions.

Anyway, If I understand correct, I can already do this today by running METEOR_FORCE_PORTABLE=1 meteor publish and it will work for meteor 1.3.3+ users?

@benjamn

This comment has been minimized.

Member

benjamn commented Aug 8, 2016

Anyway, If I understand correct, I can already do this today by running METEOR_FORCE_PORTABLE=1 meteor publish and it will work for meteor 1.3.3+ users?

This is almost true, except that we need to record a bit more information in .meteor-last-rebuild-version.json so that packages will be rebuilt reliably. That's what 1d47395 does. Without that change, .meteor-last-rebuild-version.json files look the same for packages built on different platforms (Mac, Linux, Windows), as long as the Node/V8/etc. versions match. So, for example, a Linux user would not always rebuild a package that you published using a Mac. Adding process.{platform,arch} to .meteor-last-rebuild-version.json fixes that.

benjamn added some commits Aug 9, 2016

Permit patch-compatible versions in .meteor-last-rebuild-version.json.
This is a more general solution than simply discarding the patch version
for currentVersions.versions.node, as I did in a previous commit.

@benjamn benjamn merged commit a78dde0 into devel Aug 11, 2016

4 checks passed

CLA Author has signed the Meteor CLA.
Details
ci/circleci Your tests passed on CircleCI!
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
@tmeasday

This comment has been minimized.

Contributor

tmeasday commented Aug 12, 2016

See #7634

@mitar

This comment has been minimized.

Collaborator

mitar commented Aug 20, 2016

We believe this change will reduce demand for https://build.meteor.com sufficiently that we may consider turning off the build farm. If you are a package author who uses the build farm, we would love to hear your thoughts on this change, especially if you think you would continue running meteor publish-for-arch after you are no longer expected to do so.

It would be OK if it would work.

But I think you should keep binaries around for packages which have heavy dependencies. Like pdf.js package I have is pretty tricky to get all dependencies installed.

@mitar

This comment has been minimized.

Collaborator

mitar commented Aug 20, 2016

If you want to deprecate build machines as well, maybe you should document better what are configuration of each machine so that people can run it themselves. (You are saying in documentation that it is important that they are precisely configured.) Maybe you could even provide virtual images or vagrant images to build with a command.

@c9s

This comment has been minimized.

Contributor

c9s commented Dec 3, 2016

Can we make this option inside the package.js to mark a package as portable? because sometimes it could be forgot to pass the option.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment