Instructions in testing guide fail to run client tests from the command line

[danstiner]

While following the instructions in the testing guide for configuring command line execution using PhantomJS (https://guide.meteor.com/testing.html) I consistently see the error message cannot execute binary file when running client tests. This seems to be because the phantomjs being installed is for MacOS but I am running Linux x64.

Steps to reproduce on a Linux machine:

curl https://install.meteor.com/ | sh
meteor create phantomjs-tests
cd phantomjs-tests/
meteor npm install --save babel-runtime
meteor add dispatch:mocha-phantomjs
meteor test --once --driver-package dispatch:mocha-phantomjs

Resulting output:

--------------------------------
----- RUNNING CLIENT TESTS -----
--------------------------------

/home/dan/.meteor/packages/dispatch_phantomjs-tests/.0.0.7.1k3403j++os+web.browser+web.cordova/npm/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs: /home/dan/.meteor/packages/dispatch_phantomjs-tests/.0.0.7.1k3403j++os+web.browser+web.cordova/npm/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs: cannot execute binary file

Running file on the binary indicates it is actually a Mach-O executable meant for MacOS:

$ file /home/dan/.meteor/packages/dispatch_phantomjs-tests/.0.0.7.1k3403j++os+web.browser+web.cordova/npm/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs
Mach-O 64-bit x86_64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|BINDS_TO_WEAK|PIE>

This can be worked around by running the following, which leads me to believe this is an issue with meteor not correctly doing the rebuild, and that this is not an issue with the phantomjs-tests package.

cd /home/dan/.meteor/packages/dispatch_phantomjs-tests/.0.0.7.1k3403j++os+web.browser+web.cordova/npm
npm rebuild

Related discussion on Dispatch-Archive/meteor-mocha-phantomjs#35

Meteor 1.4.2.3
dispatch:mocha-phantomjs@0.1.9
dispatch:phantomjs-tests@0.0.7
x86_64 GNU/Linux

[aldeed]

@benjamn I have been getting a lot of complaints about this. All I did was publish dispatch:phantomjs-tests with a couple small changes from Mac, but it was done on latest Meteor for the first time so this effectively caused a breaking change in a non-breaking version. It seems that meteor npm rebuild, as suggested in https://forums.meteor.com/t/whats-happening-to-meteor-publish-for-arch-in-meteor-1-4-1/28209 and the guide, does not actually rebuild the non-local cached packages, or maybe they are overwritten after being rebuilt. I haven't investigated much but I did run into this myself on Travis.

I thought maybe I could fix by doing publish-for-arch anyway, but it would not let me.

[abernix]

@aldeed Just out of curiosity, which version of Meteor were you previously publishing dispatch:phantomjs-tests from? I'll try to take a look at this today.

[aldeed]

@abernix I think 1.3.x? Not sure exactly since there was a long period between publishing the last working version and the patch that started breaking things.

[aldeed]

@abernix There may be other info helpful for you to investigate in the linked issue Dispatch-Archive/meteor-mocha-phantomjs#35

[danstiner]

After some digging I have found what I think to be the issue. Pardon my unfamiliarity with meteor internals.

In commit 861880b the behavior of meteorNpm.rebuildIfNonPortable was changed by @benjamn from looking at a packages' npm modules and rebuilding all of them if any was non-portable, to a more "aggressive" approach of only rebuilding individual non-portable npm modules.

The old behavior worked because dispatch:phantomjs-tests has the module meteor-force-non-portable which would force a rebuild of all modules under ~/.meteor/packages/dispatch_phantomjs-tests/0.0.7/npm/node_modules. Under the new behavior only that specific non-portable module is rebuilt. The npm package phantomjs-prebuilt is ignored because the module does not contain any files ending in ".node" and thus meteor thinks it is portable. I think looking for ".node" files is a heuristic for packages that use node-gyp, but it fails to flag phantomjs-prebuilt because it uses a non-portable install script to pull down arch-specific binaries. Full logic is in the function isPortable: https://github.com/meteor/meteor/blob/devel/tools/isobuild/meteor-npm.js#L441

I confirmed that changing the code to see phantomjs-prebuilt as non-portable caused the correct rebuild to happen and fix the reported issue.

I think this could be fixed by additionally look in each package to see if the scripts section of package.json contains a install or preinstall script. The npm docs say "[t]he only valid use of install or preinstall scripts is for compilation which must be done on the target architecture." This will add some file read and parsing time in additional to causing a few more rebuilds to happen but it should give correct behavior. Additionally I suspect instead of heuristically looking for .node files a check could directly be made for a .gyp config file, but that may not be true in all cases and would at least break the meteor-force-non-portable package.

Note: I noticed while debugging the dispatch:phantomjs-tests package was actually packaged with cached portable status files like ~/.meteor/packages/dispatch_phantomjs-tests/0.0.7/npm/node_modules/phantomjs-prebuilt/.meteor-portable which contained the value true. I'm not sure if that is intended behavior, but it does suggest two things:

1. Any fix will require publishing updated versions of dispatch:phantomjs-tests and other affected packages with fixed or removed .meteor-portable files
2. An ugly short-term workaround: simply re-publish the package with the cache file for phantomjs-prebuilt manually edited to contain false. That should force a rebuild on install, the cached value will override any portability logic.

[aldeed]

Wow, that seems promising @danstiner. Thanks for investigating! I will see if I can get a fixed version published using the workaround.

@abernix

[benjamn]

@danstiner That analysis checks out with me (and I wrote the logic that you're analyzing)!

I think this could be fixed by additionally look in each package to see if the scripts section of package.json contains a install or preinstall script.

I think this is a totally reasonable approach. I would be cautiously open to going back to the old behavior of rebuilding everything, but this middle ground strikes me as a safe compromise. If it's safe to rebuild everything, then it's safe to rebuild a few additional packages.

Would you have time to submit a pull request for that change? Specifically, make isPortable return false if a package.json file containing an install or preinstall script is found?

[danstiner]

Thanks so much for taking care of this @benjamn! Sorry I wasn't able to make time to send a pull request myself.

[0o-de-lally]

@benjamn I can confirm that the meteor update --release 1.4.3-rc.3 runs dispatch:mocha-phantomjs smoothly, and fixes issues in meteor/todos#198 , Dispatch-Archive/meteor-mocha-phantomjs#33 and possibly Dispatch-Archive/meteor-mocha-phantomjs#35

@abernix If MDG is updating the testing guide: could you focus more on dispatch:mocha-phantomjs, rather than on the other dispatch options ? Despite this error it seems like dispatch:mocha-phantomjs seems to be currently the most maintained package of them all.