[1.4] mup: errors in production (AssertionError: "undefined" === "function") #7475

Closed
WayneUong opened this Issue Jul 26, 2016 · 94 comments

Projects

None yet
@WayneUong
WayneUong commented Jul 26, 2016 edited

Upgraded 1.3.5.1 to 1.4, it works fine locally, but when I deployed to EC2 using mupx

[52.35.45.35] x Verifying deployment: FAILED

-----------------------------------STDERR-----------------------------------
package.json meteor-dev-bundle@0.0.0 No description
npm WARN package.json meteor-dev-bundle@0.0.0 No repository field.
npm WARN package.json meteor-dev-bundle@0.0.0 No README data
npm WARN cannot run in wd meteor-dev-bundle@0.0.0 node npm-rebuild.js (wd=/bundle/bundle/programs/server)
=> Starting meteor app on port:80

assert.js:93
  throw new assert.AssertionError({
        ^
AssertionError: "undefined" === "function"
    at wrapPathFunction (/bundle/bundle/programs/server/mini-files.js:77:10)
    at Object.<anonymous> (/bundle/bundle/programs/server/mini-files.js:108:24)
    at Module._compile (module.js:456:26)
    at Object.Module._extensions..js (module.js:474:10)
    at Module.load (module.js:356:32)
    at Function.Module._load (module.js:312:12)
    at Module.require (module.js:364:17)
    at require (module.js:380:17)
    at Object.<anonymous> (/bundle/bundle/programs/server/boot.js:9:13)
    at Module._compile (module.js:456:26)

=> Redeploying previous version of the app
@Nexus7
Nexus7 commented Jul 26, 2016

Have you tried using a version of node which is 4.x.x?

@minderov
minderov commented Jul 26, 2016 edited

I had exactly the same problem. Go to your meteor app folder and check the version of node that meteor uses meteor node -v. It might be different from the node version that node -v would show. In my case meteor node version was v0.10.45, so running meteor update helped to fix the problem. Make sure that node -v shows 4.x.x too.

UPDATE:
Although I should point out that I am not using mupx. I am checking out git repo and building it on server. If I am building it locally as a tarball and use it to deploy the app on VPS, I get the same error. Was working fine on 1.3.5.1.

@WayneUong

mupx uses a docker container to run meteor and all of the dependencies. My guess is that the node version inside the docker image is not 4.x.x. I just can't figure out how to update node with mupx yet, or even check its version.

@Nexus7
Nexus7 commented Jul 26, 2016

I think the node version for mupx is in the mup.json file?

@WayneUong

I don't you can specify node version in the json settings. You probably have to hard code it in the installation script here:
https://github.com/arunoda/meteor-up/blob/master/scripts/linux/install-node.sh

What's a good node version I should use?

@Nexus7
Nexus7 commented Jul 26, 2016

See here: https://github.com/arunoda/meteor-up

You have a setting for nodeVersion in there... You could try and use v4.2.2 but just need to make sure it is on the machine in question... I believe Meteor 1.4 is now on 4.4.7

@WayneUong

I tried these settings:

  "setupNode": true,
  "nodeVersion": "4.4.7",

Still having the same error. Also note that I use mupx (https://github.com/arunoda/meteor-up/tree/mupx) instead of mup (https://github.com/arunoda/meteor-up)

@Ben305
Ben305 commented Jul 26, 2016

I'm using a docker image with Node 4.4.7, but my deployment fails with:

/bundle/bundle/programs/server/node_modules/fibers/future.js:280
                                                throw(ex);
                                                ^
Error: Module did not self-register.
    at Error (native)
    at Object.Module._extensions..node (module.js:434:18)
    at Module.load (module.js:343:32)
    at Module.Mp.load (/bundle/bundle/programs/server/npm/node_modules/meteor/babel-compiler/node_modules/reify/node/runtime.js:16:23)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at bindings (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bindings/bindings.js:76:44)
    at Object. (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bcrypt/bcrypt.js:3:35)
    at Module._compile (module.js:409:26)

If anyone want's to try my docker image, just use ben305/meteord:node4, which is a fork from meteorhacks:meteord base image.

@evanoralph

Im having this problem too just updated 1.4 and i cant deploy now Im getting this error

AssertionError: "undefined" === "function"

@abernix abernix changed the title from [meteor 1.4] AssertionError: "undefined" === "function" to [1.4] mupx: errors in production Jul 26, 2016
@walliski

Had this problem when updating to Meteor 1.4 (not docker/mupx), just had to update Node version on the server to 4.x.x. Now it works fine 🎉

@abernix
Collaborator
abernix commented Jul 26, 2016 edited

Ok, in case this issue gets out of control, this issue is going to be reserved for the original issue reported is which is Meteor 1.4 not working in production with MupX You'll probably find your own answer by reading along but if you're going to +1 this issue, ask for help in this issue, or paste in your error, make sure your environment matches what this issue is about!

To start off, let's clarify that there are three versions of "mup":

  1. @arunoda's original mup: https://github.com/arunoda/meteor-up
  2. The mupx branch of that project: https://github.com/arunoda/meteor-up/tree/mupx (THIS issue)
  3. The even newer kardirahq mup: https://github.com/kadirahq/meteor-up

Meteor 1.4 now uses Node 4.4.7 and it is absolutely necessary to update your Node version on your production environment when moving to Meteor 1.4. This is not something that Meteor handles or has ever handled.

Solutions?

"original" mup (number 1 above – NOT mupx!)

I believe you can get away with changing the nodeVersion in your mup.json to 4.4.7 and running mup setup to reinstall the NEW Node version, then mup deploy to deploy the 1.4 version.

The mupx branch (number 2 above)

The nodeVersion in your mup.conf IS NOT USED. Changing it will do nothing! In fact, it's not supposed to be there if you're using mupx. If it's there, it means it's probably left over from an older mup upgrade.

Unfortunately, mupx is a dead-end at the moment since it is hard coded to use the meteorhacks/meteord:base docker image and that Docker image explicitly uses Node 0.10.43.

The best advice I could give is to switch to using the NEW kardirahq mup as it supports specifying a dockerImage.

The new kadirahq/mup

The default dockerImage is meteorhacks/meteord:base and thus still uses Node 0.10.43. If someone builds and publishes a new image from the base, it could be easily swapped in. I've gone ahead and provided an abernix/meteord:base image with the necessary changes to run Node 4.4.7 + the benefit of NPM 3 on an updated distro (jessie instead of wheezy). I'm not sure if this is something that kadira will provide soon themselves or not.

You can use my image by adding the following in your mup.js, inside the meteor object (see docs for details), and re-deploying with mup deploy:

dockerImage: 'abernix/meteord:base',

Your own Node deployment!

Update your node to node 4.4.7! On Ubuntu, this can be done with apt-get, but I'll leave you to your own devices to figure that out; there is plenty of information out there on how to do this. A popular option for easily changing node versions is n or nvm, though you can generally find what you need from Node's website.

Also Important!

If you're not building your project on the exact same architecture as you're deployment (e.g. you run meteor build on your Mac, but you run node main.js on Ubuntu), it's important that you have the ability to compile NPM binary dependencies on your Ubuntu machine otherwise you will end up with errors (Module did not self-register is one of them).

Ensure that you meet the requirements for node-gyp installation! and make sure you're running npm install in the programs/server folder of your Meteor bundle, before you start the server with node main.js. I believe that meteord would have the requirements as it installs build-essentials and python (which in the wheezy repository I think is version 2.7)

You might also try running npm rebuild instead of npm install within your programs/server directory if you're dealing with an existing project.

@Ben305
Ben305 commented Jul 26, 2016

@abernix I'm using kadirahq/mup with my own docker-image, that includes Node 4.4.7. But even with that image, I get the error message I posted above.

@abernix
Collaborator
abernix commented Jul 26, 2016 edited

@Ben305 What happens if you run npm rebuild inside your bundle's programs/server directory ON the machine the bundle is being run on ?

@Ben305
Ben305 commented Jul 26, 2016 edited

@abernix I added npm rebuild to my docker image (ben305/meteord:node4), but the error is still there

@abernix
Collaborator
abernix commented Jul 26, 2016

@Ben305 When you are usingkadirahq:mup it builds on your local machine – not inside the docker image. The change you made to your latest Docker image was the addition of the npm rebuild command in the meteord build_app.sh file but that file is not called from the image you created, which is based on meteorhacks/meteord:base. The build_app.sh script is only called on the meteorhacks/meteord:devbuild or meteorhacks/meteord:onbuild images. So npm rebuild is still not getting called in your case.

If you were to put it anywhere in your image, you'd need to put it in run_app.sh.

However, to debug this, I'd suggest an easier option than building a new image for now...

On your docker host, run the following:

$ sudo docker exec -it containerId /bin/bash # where containerId is what you get from `docker ps`

Then, at the prompt inside your Docker image:

$ (cd /bundle/bundle/programs/server && npm rebuild)

Then run the following back on your own terminal in your .deploy directory:

mup restart

If you're still having problens, debugging wise, it'd be nice to see the output of:

$ sudo docker logs containerId # again, containerId from before
@dr-dimitru
Contributor
dr-dimitru commented Jul 26, 2016 edited

For everyone who expecting Module did not self-register issue on dev-stage and having locally installed NPM packages used in app, run (note you need NVM intalled):

nvm install v4.4.7
npm rebuild
npm update
meteor reset
meteor update
meteor run

As unrelated, moved to separate ticket

@abernix
Collaborator
abernix commented Jul 26, 2016

@dr-dimitru Not only does that not relate to the original issue (production), but those are a lot of unnecessary commands which do a lot of silly things including installing a Node binary that Meteor will not use itself and wiping your database.

It's best to use meteor npm which is already the correct bundled version of Node for the version of Meteor that you are using. Quite simply, I believe meteor npm rebuild would have likely fixed your issue. But really, please open a new issue if you are experiencing that issue.

@catin-black

I have the same problem (with mupx)
Installation of new version of node did not help.

@dr-dimitru
Contributor

@abernix created separate issue, ty

@cstrat
cstrat commented Jul 26, 2016 edited

I am having this issue bundling from my Mac and deploying to NodeChef.
Guessing that my issue is what @abernix pointed about where you mentioned bundling on a system which is different to where you're deploying...

What should I be doing then? Bundling on my system for Ubuntu?
Currently I run:
meteor build ../_builds/ --architecture os.linux.x86_64

NodeChef seems to use Ubuntu though... should I be bundling for that?


EDIT - in case others are hosting on NodeChef and having the same issue.
Checked with the NodeChef team - sounds like they just need to update their PaaS to support the new version of Node. Should be fixed shortly.

@rajpa
rajpa commented Jul 26, 2016

Can confirm I got this error with mupx, meteor v1.4.
Node 4.4.7, npm 3.10.5 both on dev and prod.

Pls help to resolve this!!!

-----------------------------------STDERR-----------------------------------
package.json meteor-dev-bundle@0.0.0 No description
npm WARN package.json meteor-dev-bundle@0.0.0 No repository field.
npm WARN package.json meteor-dev-bundle@0.0.0 No README data
npm WARN cannot run in wd meteor-dev-bundle@0.0.0 node npm-rebuild.js (wd=/bundle/bundle/programs/server)
=> Starting meteor app on port:80

    assert.js:93
      throw new assert.AssertionError({
            ^
    AssertionError: "undefined" === "function"
        at wrapPathFunction (/bundle/bundle/programs/server/mini-files.js:77:10)
        at Object.<anonymous> (/bundle/bundle/programs/server/mini-files.js:108:24)
        at Module._compile (module.js:456:26)
        at Object.Module._extensions..js (module.js:474:10)
        at Module.load (module.js:356:32)
        at Function.Module._load (module.js:312:12)
        at Module.require (module.js:364:17)
        at require (module.js:380:17)
        at Object.<anonymous> (/bundle/bundle/programs/server/boot.js:9:13)
        at Module._compile (module.js:456:26)

    => Redeploying previous version of the app
@Ben305
Ben305 commented Jul 26, 2016

@abernix I don't have the directory /bundle/bundle/programs/server in my docker container.

@CarlQLange

does using https://github.com/jshimko/meteor-launchpad instead of meteorhacks/meteord solve this problem?

@abernix
Collaborator
abernix commented Jul 26, 2016

@Ben305 I'm certain you do. In fact, that path is actually in the error message you pasted above. I just reproduced this by deployed a new 1.4 project all the way through, successfully (because of no binary dependencies). Please try my suggestion on the docker host (the machine that is running your docker container).

@rajpa If you are using mupx you are NOT using Node 4.4.7 – this is because Node 0.10 is baked into the docker image that mup is using. My advice would be to downgrade your project to 1.3 if you are experiencing this in production and wait for the mup project to come up with a recommendation.

@CarlQLange It would appear that that docker image has been updated to support Node 4, but I do not believe that docker image is compatible with mup. (my quick test shows it is not).

@CarlQLange

@abernix thanks a lot for the response, and your help in this thread generally :) I didn't get to test it myself.

@abernix
Collaborator
abernix commented Jul 26, 2016

I really wish I had a more appropriate solution. Unfortunately, this is not a problem with Meteor (mup is not a Meteor provided or supported tool) and any solution requires changes to your deployment environment – plus it might need to be carefully integrated into the meteord image to avoid unnecessary rebuilds, etc.

@Ben305, since you're the only person who has identified using kadirahq:mup (versus meteorhacks:mup), can you try using the image I just built? I had to do it quickly, but I was able to start a server with it. Simply add the following line to your mup.js (from your .deploy folder), inside the meteor object:

dockerImage: 'abernix/meteord:base', 

...then run mup deploy and see how it works.

@rajpa
rajpa commented Jul 26, 2016

The docker image on prod keeps restarting.. not sure if that is because of mupx or something else...

@WayneUong
WayneUong commented Jul 26, 2016 edited

@abernix I hard coded abernix/meteord:base into mupx because I needed to install wkhtmltopdf. It managed to get passed the AssertionError. But now I'm encountering the Module did not self-register. Seems like a step forward. I know this issue is no longer relevant here, but do you know how to solve it in the context of mup/mupx?

For reference, this is my start.sh for mupx:

#!/bin/bash

APPNAME=<%= appName %>
APP_PATH=/opt/$APPNAME
BUNDLE_PATH=$APP_PATH/current
ENV_FILE=$APP_PATH/config/env.list
PORT=<%= port %>
USE_LOCAL_MONGO=<%= useLocalMongo? "1" : "0" %>

# Remove previous version of the app, if exists
docker rm -f $APPNAME

# Remove frontend container if exists
docker rm -f $APPNAME-frontend

# We don't need to fail the deployment because of a docker hub downtime
set +e
docker build -t abernix/meteord:app - << EOF
FROM abernix/meteord:base

RUN apt-get update
RUN curl -O http://download.gna.org/wkhtmltopdf/0.12/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz
RUN tar -xf wkhtmltox-0.12.3_linux-generic-amd64.tar.xz
RUN apt-get install libXrender1 -y
RUN mv ./wkhtmltox/bin/wkhtmltopdf /bin
EOF
set -e

if [ "$USE_LOCAL_MONGO" == "1" ]; then
  docker run \
    -d \
    --restart=always \
    --publish=$PORT:80 \
    --volume=$BUNDLE_PATH:/bundle \
    --env-file=$ENV_FILE \
    --link=mongodb:mongodb \
    --hostname="$HOSTNAME-$APPNAME" \
    --env=MONGO_URL=mongodb://mongodb:27017/$APPNAME \
    --name=$APPNAME \
    abernix/meteord:app
else
  docker run \
    -d \
    --restart=always \
    --publish=$PORT:80 \
    --volume=$BUNDLE_PATH:/bundle \
    --hostname="$HOSTNAME-$APPNAME" \
    --env-file=$ENV_FILE \
    --name=$APPNAME \
    abernix/meteord:app
fi

<% if(typeof sslConfig === "object")  { %>
  # We don't need to fail the deployment because of a docker hub downtime
  set +e
  docker pull meteorhacks/mup-frontend-server:latest
  set -e
  docker run \
    -d \
    --restart=always \
    --volume=/opt/$APPNAME/config/bundle.crt:/bundle.crt \
    --volume=/opt/$APPNAME/config/private.key:/private.key \
    --link=$APPNAME:backend \
    --publish=<%= sslConfig.port %>:443 \
    --name=$APPNAME-frontend \
    meteorhacks/mup-frontend-server /start.sh
<% } %>
@WayneUong
WayneUong commented Jul 26, 2016 edited

@abernix more info: I deploy to a linux EC2 instance from a mac osx. But I don't think the architecture is the problem here since it worked fine pre-1.4. Also, I believe mupx defaults the architecture to linux:
https://github.com/arunoda/meteor-up/blob/mupx/lib/build.js#L22

@indesignlatam

I just tried this "dockerImage: 'abernix/meteord:base'" with no success using kadira version, i also ran meteor npm rebuild before deploying but no success.

Verifying Deployment: FAILED

    -----------------------------------STDERR-----------------------------------
    undle/programs/server/node_modules/supports-color
    chalk@0.5.1 /bundle/bundle/programs/server/node_modules/chalk
    => Starting meteor app on port:80
    /bundle/bundle/programs/server/node_modules/fibers/future.js:280
                            throw(ex);
                            ^

    Error: Module did not self-register.
        at Error (native)
        at Object.Module._extensions..node (module.js:434:18)
        at Module.load (module.js:343:32)
        at Module.Mp.load (/bundle/bundle/programs/server/npm/node_modules/meteor/babel-compiler/node_modules/reify/node/runtime.js:16:23)
        at Function.Module._load (module.js:300:12)
        at Module.require (module.js:353:17)
        at require (internal/module.js:12:17)
        at bindings (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bindings/bindings.js:76:44)
        at Object.<anonymous> (/bundle/bundle/programs/server/npm/node_modules/meteor/npm-bcrypt/node_modules/bcrypt/bcrypt.js:3:35)
        at Module._compile (module.js:409:26)

    => Redeploying previous version of the app
@Ben305
Ben305 commented Jul 26, 2016

@abernix Tried your image, same problem: Error: Module did not self-register.

@abernix abernix changed the title from [1.4] mupx: errors in production to [1.4] mup: errors in production Jul 26, 2016
@abernix
Collaborator
abernix commented Jul 26, 2016

@Ben305 @indesignlatam Can you try again with my abernix/meteord:base image? If you still have it in place then just running mup deploy again should do the trick.

@abernix
Collaborator
abernix commented Jul 26, 2016

(Link to my changes, and explanation)

I was able to reproduce the problem and actually fix it (for myself, at least) by changing npm install to use the --unsafe-perm flag, as found by @markoshust in #7339 (comment). This is due to the fact that npm is installing as root on Docker. I also went ahead and upgraded debian from wheezy to jessie (thanks, me!)

Contrary to the flag name, I don't believe the permissions are actually unsafe since it's within the Docker container. 😉

@indesignlatam

@abernix I can confirm it now works without problems. Thanks for taking time to fix this.

@WayneUong

@abernix confirmed working. I had to install a ubuntu virtual machine to avoid messing too much with docker. thanks!

@evanoralph
evanoralph commented Jul 27, 2016 edited

@WayneUong can i ask what method did you use to fix the error that @abernix provide still new in this

@WayneUong

@evanoralph I setup a Ubuntu virtual machine with virtualbox and deploy from there to make sure that it has the same environment as my server. I also make sure that both node and meteor node are at 4.4.7.

You can use kadirahq/mup and specify the dockerImage as abernix/meteord:base. But I am using a custom mupx with the modified start.sh above, and it works fine for me. You can find the mupx folder at /usr/local/lib/node_modules/mupx/ (mac osx).

@evanoralph

Thank you @WayneUong I have look the abernix dockerImage, Can you give me example how to specify it Im still learning but its ok if not if only you have free time Thanks

@markoshust

For those using Docker, I needed to add the --unsafe-perm flag to npm i to get things installing correctly. See here for example https://github.com/markoshust/docker-meteor/blob/master/Dockerfile#L10

@Ben305 Ben305 referenced this issue in meteorhacks/meteord Jul 27, 2016
Open

Update for Meteor 1.4 #108

@Ben305
Ben305 commented Jul 27, 2016 edited

@abernix I can confirm your image works! Thank you!!!

I've updated my fork with your changes and created a PR for the meteord image

@abernix
Collaborator
abernix commented Jul 27, 2016 edited

All: I've gone ahead and made one more change to my abernix/meteord:base image which updates it to use npm version 3. While not critical I figure it would be more ideal to use the same version that Meteor uses in development, and also provide the flatter dependency-tree, which is sometimes desirable.

@markoshust Thanks for stumbling across that fix. I actually don't use mup but I saw your update about this recently and I included that in my docker image above. Worked well! (Your image is great too, this is just a drop-in replacement for mup)

@evanoralph Please see the new kadirahq/meteor-up project and follow their instructions for setting up and deploying. It's a small migration but I believe you should be able to replace mupx with it quite simply. There's a slight difference in where the mup configuration lives (now inside a .deploy folder) and also the format is now JavaScript (.js) instead of .json – but you should consult their docs.

@Ben305 Very glad it's working! A PR for the meteord image could potentially break a lot of peoples installations (e.g. those who are running pre-Meteor 1.4), so I'm not sure that PR should/will be accepted? It will likely need to remain a separate image. Either way, good to record the issue for others to see.

I'm closing this issue. For the record, I don't use this image and I don't have much intention in keeping the abernix/meteord:base image up to date so I encourage those who need it to take appropriate measures. You can see my changes here.

Lastly, if you're in this issue and you still can't resolve your issue, please take appropriate actions to update to Node 4. If you're not familiar with what it takes to maintain your own Node+Meteor deployment I highly recommend looking at paid options out there which manage this for you! (cough, cough I dunno, Meteor Galaxy for MDGs own sake!)

@abernix abernix closed this Jul 27, 2016
@abernix abernix added a commit to abernix/meteor-guide that referenced this issue Jul 27, 2016
@abernix abernix Provide correct mup options for various Meteor versions
Meteor 1.4 does not work with `mupx` without forking it and providing your own docker image.  A better option is to use the kadirahq/mup developement version (also named `mup`).  A dockerImage is still necessary but hopefully the project will provide an option.  A work around is available in meteor/meteor#7475 but I'd like to avoid suggesting it publically at the moment.

Closes meteor/guide#529
3d586d4
@patrickml

@abernix do you see anything wrong with this config? I cant seem to get the application to load

module.exports = {
  servers: {
    one: {
      host: '192.25.151.925',
      username: 'root',
      pem: './key.pem',
    },
  },

  meteor: {
    name: 'myapp',
    path: '../../',
    servers: {
      one: {},
    },
    buildOptions: {
      debug: true,
    },
    env: {
      PORT: 80,
      ROOT_URL: 'http://mydomain.com',
      MONGO_URL: '<my_url>',
      MONGO_OPLOG_URL: '<my_url>',
    },
    dockerImage: 'abernix:meteord:base',
    deployCheckWaitTime: 60,
  },
};
@zeroasterisk

@abernix @Ben305 I'm using mup (the new one) & dockerImage: 'abernix:meteord:base' but still getting the following error when attempting to start up :(

> fsevents@1.0.14 install /bundle/bundle/programs/server/npm/node_modules/fsevents
> node-pre-gyp install --fallback-to-build

sh: 1: node-pre-gyp: not found

npm ERR! Linux 4.4.0-31-generic
npm ERR! argv "/opt/nodejs/bin/node" "/opt/nodejs/bin/npm" "rebuild" "--no-bin-links" "--update-binary"
npm ERR! node v4.4.7
npm ERR! npm  v3.10.5
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! fsevents@1.0.14 install: `node-pre-gyp install --fallback-to-build`
npm ERR! spawn ENOENT

ideas? I'm happy to fork and customize the image, but not sure what I should be doing.

building on mac, deploying to ubuntu server (on docker, so it shouldn't matter)

@abernix
Collaborator
abernix commented Jul 27, 2016
@xergioalex
xergioalex commented Jul 27, 2016 edited

Thanks @abernix, i'am using a custom mupx with your docker container, and it's working.

If anyone want test my custom mupx (https://github.com/RockaLabs/meteor-up/tree/muprockanew) just install with:

npm -g install git+https://github.com/RockaLabs/meteor-up.git#muprockanew

Then use normally muprockanew setup or muprockanew deploy

@ouuyeah
ouuyeah commented Jul 28, 2016

Muchas gracias @xergioalex , me ha funcionado :D !

@millerone

Thank you so much @abernix for the help and @xergioalex for your custom mupx (it's the only one that has work perfectly for me)!!
Finally i can deploy again :)

@abernix
Collaborator
abernix commented Jul 28, 2016

@zeroasterisk Can you come up with a basic reproduction repo that this happens to you on? I simply lack the time to dig around at the moment and this might be due to one of your NPM dependencies.

@abernix
Collaborator
abernix commented Jul 28, 2016

@patrickml Looks reasonable, assuming your Meteor application is two full levels above this file? (path: '../../'??) The error(s) you're receiving would be more helpful.

@patrickml

@abernix i have my deploy script in .deploy/beta/mup.js

@zeroasterisk
zeroasterisk commented Jul 28, 2016 edited

@abernix I think it's "redux": "^3.5.2" -- i'll be posting a test repo soon...

@evlrbot
evlrbot commented Jul 28, 2016

My app is on Meteor 1.4, node 4.4.7, and will run on my deployment server. I modified the mupx start.sh script to use abernix/meteord:base. Then ran mupx setup, then mupx deploy, then get the following error:

x Verifying deployment: FAILED

    -----------------------------------STDERR-----------------------------------
    0.5
    npm ERR! code ELIFECYCLE
    npm ERR! meteor-dev-bundle@0.0.0 install: `node npm-rebuild.js`
    npm ERR! Exit status 1
    npm ERR! 
    npm ERR! Failed at the meteor-dev-bundle@0.0.0 install script 'node npm-rebuild.js'.
    npm ERR! Make sure you have the latest version of node.js and npm installed.
    npm ERR! If you do, this is most likely a problem with the meteor-dev-bundle package,
    npm ERR! not with npm itself.
    npm ERR! Tell the author that this fails on your system:
    npm ERR!     node npm-rebuild.js
    npm ERR! You can get information on how to open an issue for this project with:
    npm ERR!     npm bugs meteor-dev-bundle
    npm ERR! Or if that isn't available, you can get their info via:
    npm ERR!     npm owner ls meteor-dev-bundle
    npm ERR! There is likely additional logging output above.

    npm ERR! Please include the following file with any support request:
    npm ERR!     /bundle/bundle/programs/server/npm-debug.log
    sudo: unable to resolve host ip-172-30-0-47

    => Redeploying previous version of the app

Does anyone know what the cause may be?

@micahalcorn

Thanks @abernix for the fix. I am able to deploy my app using your branch of the new mup unless I import gcloud-node, which seems to struggle with gRPC and related dependencies. Since I have no experience with Docker and don't really understand how Meteor Up works, I don't know what I'm looking at on this stack trace. I tried manually installing grpc and node-pre-gyp both prior to my build on my local machine and also on the production instance. I suppose that I might need to do this from within the Docker container or I might have a permissions issue or something? Thanks for any advice.

@thsowers
thsowers commented Jul 29, 2016 edited

@micahalcorn Have you tried sudo apt-get install build-essential

I ran into this issue and this fixed it, read more here: kelektiv/node.bcrypt.js#90

Edit: Formatting

@zeroasterisk
zeroasterisk commented Jul 29, 2016 edited

Thanks @abernix & @Ben305 --- Does the following nearly empty repo deploy for you?

https://github.com/zeroasterisk/mup-meteor-1-4-demo

cd /tmp
git clone https://github.com/zeroasterisk/mup-meteor-1-4-demo.git
cd mup-meteor-1-4-demo
(cd webapp && npm install --production)
vim deploy/mup.js
(cd deploy && mup deploy)
@evanoralph

@abernix and all tnx for the help!! cheers

@micahalcorn
micahalcorn commented Jul 29, 2016 edited

@thsowers "already the newest version". Thanks though.

Edit note: I don't have any problem with bcrypt when gcloud is not included.

@aslafy-z

I had to install npm package before meteor build without what it was failing with : "Unable to resolve modules: [insert long list of modules]".

This is my (working) fork: https://github.com/aslafy-z/meteord, it's on docker hub too: https://hub.docker.com/r/zadki3l/meteord/ (binbuild failing :/)

Thanks to @Ben305, @abernix & @zhaoyao91 !

@abernix
Collaborator
abernix commented Jul 29, 2016

@micahalcorn On the gcloud issue, maybe chime in specifically about that on this related issue: #7301

@aslafy-z I'm not super clear on what you're saying, but in general, I think if you make sure that you run meteor npm install in development and run your project at least once before you meteor build you should be able to avoid that. Modules are copied from your development process during the meteor build process so that might explain your problem? Not sure. Hope things are working for you in general – your changes to your docker image look almost identical to mine.

Also, I also had problems with the binbuild image initially, but I think I resolved it and pushed it to Docker hub – though I have no way to test the image as I'm not sure what/who binbuild applies to or if anyone uses it? I think it's relatively uncommon. Anyhow, I just pushed my changes to Git if you want to take a look. Also, feel free to test the abernix/meteord:binbuild image if it applies to you, as that's the one that uses the build_app script.

@aslafy-z

@abernix: In fact, i'm building my docker image from a CI, that's why I need to install dependencies at build time. Is this a bad pattern ?

@abernix
Collaborator
abernix commented Jul 29, 2016

@aslafy-z That's fine but if your Docker architecture is not exactly the same as your production environment, you will likely run into issues. You should make sure that your CI is running the exact same version of Node and in some (more extreme) cases, things like zlib, openssl, etc. or other libraries included in the output of node -p process.versions.

Ideally, you would build your Meteor on the same exact OS/arch as the Docker that is going to run it. If this isn't possible, then it's important that your Docker image have the compilation tools (g++, gcc, make, etc.) in order to rebuild things itself. There is really no avoiding this as binary dependencies are arch/platform specific.

@evlrbot
evlrbot commented Jul 29, 2016 edited

Perhaps this should find a different thread, but @micahalcorn @zeroasterisk I think you and I may be having similar issues with package dependencies. My app will not deploy with a customized mupx if it includes the meteorhacks:kadira-profiler package, but will deploy if I remove that package (meteorhacks:kadira-binary-deps@1.5.0, meteorhacks:kadira-profiler@1.3.0).

The app runs fine on my dev aws ec2 ubuntu instance with kadira-profiler enabled. My deploy target is exactly the same type of instance, so I assume there is something different about my dev environment and the abernix/meteord:base docker image. I don't know much about docker though.

Seems like I need to install my failing dependencies either in the start up script or in the docker image, but I'm out of my depth as to how.

@bruceLeeOfMeteor

@abernix Many thanks, kadirahq/mup works fine. I have a question though is it possible to update phantoms to version 2 within your dockerImage: 'abernix/meteord:base' ?

@abernix
Collaborator
abernix commented Jul 30, 2016
@c9s
Contributor
c9s commented Aug 1, 2016 edited

I've ended up using mupx and mup, the issues are just like "never ends"... I forked mup and maintained/customize here https://github.com/c9s/typeloy, rewritten in TypeScript with some special fix for the binary issues.

@nizulzaim
nizulzaim commented Aug 1, 2016 edited

i still have this problem. Anyone can help me?

-----------------------------------STDERR-----------------------------------
 erm"
 npm ERR! node v4.4.7
 npm ERR! npm  v3.10.5
 npm ERR! code ELIFECYCLE
 npm ERR! meteor-dev-bundle@0.0.0 install: `node npm-rebuild.js`
 npm ERR! Exit status 1
 npm ERR!
 npm ERR! Failed at the meteor-dev-bundle@0.0.0 install script 'node npm-rebuild.js'.
 npm ERR! Make sure you have the latest version of node.js and npm installed.
 npm ERR! If you do, this is most likely a problem with the meteor-dev-bundle package,
 npm ERR! not with npm itself.
 npm ERR! Tell the author that this fails on your system:
 npm ERR!     node npm-rebuild.js
 npm ERR! You can get information on how to open an issue for this project with:
 npm ERR!     npm bugs meteor-dev-bundle
 npm ERR! Or if that isn't available, you can get their info via:
 npm ERR!     npm owner ls meteor-dev-bundle
 npm ERR! There is likely additional logging output above.

 npm ERR! Please include the following file with any support request:
 npm ERR!     /bundle/bundle/programs/server/npm-debug.log

 => Redeploying previous version of the app

 -----------------------------------STDOUT-----------------------------------

 To see more logs type 'mup logs --tail=50'

 ----------------------------------------------------------------------------
@pykiss
pykiss commented Aug 1, 2016

Now I am using my own deploy system

I have written it in bash, and It does everything I need.

I think it is easy to use, and to customize.

I hope it will be usefull for you.

https://github.com/pykiss/simple-meteor

@evlrbot
evlrbot commented Aug 1, 2016

@nizulzaim I was getting that same error. See my earlier post on this thread. I don't understand the root problem very well, but removing packages at least got me deploying again.

If you read the FULL error log, mupx logs --tail=all, you may find the output is quite lengthy but should have the original error near the top. Read it all in order to find the package(s) causing the issue, then remove it if possible.

I had to remove meteorhacks:kadira-profiler in order to deploy. I don't understand exactly what the issue is, but I'm sure some smarty out there does.

This was just a temporary fix for me, as I ultimately reverted to meteor 1.3.4. to not lose any progress.

@nizulzaim
nizulzaim commented Aug 1, 2016 edited

I found the root cause. It is npm-bcrypt version 0.8.7 that being use my main project package. Reverting to 1.3.5 is the best option right now. Thank you @evlrbot

@Twisterking Twisterking referenced this issue in c9s/typeloy Aug 2, 2016
Closed

SSL support and maintaining #21

@abernix abernix changed the title from [1.4] mup: errors in production to [1.4] mup: errors in production (AssertionError: "undefined" === "function") Aug 2, 2016
@abernix abernix referenced this issue in kadirahq/meteord Aug 3, 2016
@madushan1000 madushan1000 upgraded node to 4.4.7 38dbdf5
@mmaguire400

xergioalex, nice work, thanks

@CaptainChemist

@xergioalex I was stuck on this for 3 days and trying the new mup didn't work but your solution worked beautifully, thank you so much!

@abernix
Collaborator
abernix commented Aug 5, 2016

For those saying that the new MUP doesn't work, I'm absolutely positive that it does work when used with my docker image. If you're having issues, I think you should report what problems you encountered before going down the route of a completely different MUP fork from a third-party (though of course, do as you wish).

My personal opinion: From a community standpoint, I just don't think that yet another MUP fork was a good idea since this whole situation will repeat itself again when Meteor updates versions again. I realize it was the "easiest" move but I think the forking of the already-deprecated mupx project into muprockanew was not the right advice when MUP v2 was already in beta, almost released and actually did work (granted it took a couple of changes).

The MUP project has been supported and maintained for years now by Kadira and the MUPv2 came out of beta this week. Granted, I think you still need to use my abernix/meteord:base image either way – I've already passed my suggestions onto the authors for how to solve that and keep everything under one roof.

Anyhow, keep on Muppin'.

@lesliechw
lesliechw commented Aug 5, 2016 edited

@abernix , MUP (the new one) with your dockerimage, confirmed work and it's my only way to make it work.

My Environment, Dev Meteor 1.4.0.1, Node 4.4.7 (both dev and production).

Thank you very much.

@twavis
twavis commented Aug 5, 2016

I just want to add my personal experience just to possibly save somebody the headache I've gone through the past day. In an attempt to fix deployment with meteor 1.4, I tried all of the options discussed in these various threads.

The new mup has several issues and it fails silently whether you have a ~ in pem path or improper path to settings file. Be careful if you blindly try to migrate from mupx to new mup!

By the time I sorted all of that out, well, I ran out of disk space from all of the docker images I tried and this silently failed as well with new mup but had new 'Copying MongoDB config FAILED' error with the modified mupx. It wasn't until I ssh'd into the ec2 instance and manually tried to copy files did I realize that I was out of disk space.

If you use AWS ec2 instances, and like me, are running the t2.micro for the free tier, be aware of the small space and how quickly these docker images will fill it.

@pykiss
pykiss commented Aug 5, 2016

We are indeed javascripters, but in order to deal with servers and deploys
we should learn and use a bit of bash.

mi approach https://github.com/pykiss/simple-meteor, always cleans the old
containers when it success or the new containers when it fails, and it is
just a 100 lines very simply bash script with done with my little bash
knolowdgement.

KISS

2016-08-05 17:09 GMT+02:00 Travis White notifications@github.com:

I just want to add my personal experience just to possibly save somebody
the headache I've gone through the past day. In an attempt to fix
deployment with meteor 1.4, I tried all of the options discussed in these
various threads.

The new mup has several issues and it fails silently whether you have a ~
in pem path or improper path to settings file. Be careful if you blindly
try to migrate from mupx to new mup!

By the time I sorted all of that out, well, I ran out of disk space from
all of the docker images I tried and this silently failed as well with new
mup but had new 'Copying MongoDB config FAILED' error with the modified
mupx. It wasn't until I ssh'd into the ec2 instance and manually tried to
copy files did I realize that I was out of disk space.

If you use AWS ec2 instances, and like me, are running the t2.micro for
the free tier, be aware of the small space and how quickly these docker
images will fill it.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#7475 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AC9PlgaqZs9N6mZx2LYnTDiyJhahPyJPks5qc1I1gaJpZM4JUz5e
.

@noahsw noahsw referenced this issue in winsleague/winsleague Aug 10, 2016
Merged

Revert "`meteor update` (#189)" #190

@Cyval
Cyval commented Aug 11, 2016 edited

Im having a weird result when installing on semaphore ci server. instead of muprockanew being installed the old instance mup is being installed. im currently stucked with this issue.

please see : http://screencast.com/t/4nNRArt4qAG
Thanks guys

@sonjaysarmiento

Hi @Cyval im currently experiencing that on circle ci too. any clue on how to move forward?

@StephenHHuang

@abernix Thanks, ran into this issue and was able to deploy with Meteor 1.4.0.1 using your dockerimage abernix/meteord:base

@boulakar

hi i have this prb i'm on ubuntu amazon aws

ubuntu@ip-10-0-1-68:~/Rocket.Chat$ node main.js

assert.js:93
throw new assert.AssertionError({
^
AssertionError: "undefined" === "function"
at wrapPathFunction (/home/ubuntu/Rocket.Chat/programs/server/mini-files.js:77:10)
at Object. (/home/ubuntu/Rocket.Chat/programs/server/mini-files.js:108:24)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/home/ubuntu/Rocket.Chat/programs/server/boot.js:9:13)
at Module._compile (module.js:456:26)

@clayne11

@boulakar did you try following the instructions in the thread? You need to use the abernix:meteord:base docker image.

@babenzele babenzele added a commit to babenzele/meteor-up-x that referenced this issue Oct 19, 2016
@babenzele babenzele Update README.md for users of Meteor 1.4
We need to inform users running Meteor 1.4 and above that they must use the newest version of mup as described by @abernix:

meteor/meteor#7475 (comment)
3275de8
@babenzele babenzele referenced this issue in arunoda/meteor-up Oct 19, 2016
Open

Update README.md for users of Meteor 1.4 #1134

@gitMonks

Hi,
I have managed to get rid of this error by using the dockerImage: 'abernix/meteord:base', docker image thanks to @abernix. BUT even though I get Verifying Deployment: SUCCESS message my app is still not live. All I can see is Welcome to nginx page.

Please help me end this nightmare!

@gitMonks gitMonks referenced this issue in kadirahq/meteor-up Oct 20, 2016
Open

Deployment Failuire #334

@babenzele
Contributor

What's in your logs? mup logs --tail=50

Does your app load locally with no problems?

@gitMonks

@babenzele I have found out that its actually live but only if the url is followed by :3000. So I have definitely messed up my nginx settings somehow. Now, I have to hunt for the solution of this new issue. Thanks for your help. Here is the log so it might help someone else.
image

@keyscores

@abernix Is there a reason MDG doesn't provide a canonical docker image? I understand it conflicts with the galaxy strategy. But docker is a very key piece of modern infrastructure, and it's aggravating to rely on abandoned community projects. Can we get an MDG maintained docker image?

@stubailo
Contributor

That docker image would just fall out of date because we wouldn't be using it internally. So I think it's best for someone in the community using docker in production to maintain it.

@keyscores

@stubailo I understand you may not use meteord. But I'm guessing galaxy has a reference implementation. Maybe you could make that available? This doesn't keep me from using galaxy. I use galaxy for the web front end, and docker elsewhere for task queue and workers.

Or worst case would be to offer someone a fellowship to maintain a docker implementation. Arunoda had a good start, but he's clearly too busy to maintain MUP and Meteord.

Otherwise we'll keep bugging you here with silly deployment questions :)

@abernix
Collaborator
abernix commented Oct 24, 2016

@keyscores I don't work for MDG so I have no comment. The MUP project has continued on the kadirahq repo and seems to actively be taking pull requests. I would encourage anyone who is interested in its upkeep to participate on that project.

I'll do my best to keep the image I offered above up-to-date (and to my knowledge it still works perfectly fine), but for the same reason that @stubailo mentioned above, it's very likely that it will eventually fall into disrepair as I don't actively use it myself.

@c9s
Contributor
c9s commented Oct 25, 2016

you have the wrong node version.

install node 4.6 for m 1.4

gitMonks notifications@github.com 於 2016年10月21日 星期五寫道:

@babenzele https://github.com/babenzele I have found out that its
actually live but only if the url is followed by :3000. So I have
definitely messed up my nginx settings somehow. Now, I have to hunt for the
solution of this new issue. Thanks for your help. Here is the log so it
might help someone else.
[image: image]
https://cloud.githubusercontent.com/assets/18562694/19574364/dabef874-9701-11e6-84ec-6af48d74dba0.png


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#7475 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AADGzrUQvgJHtIxz-xdHOfB_vbR1sWnmks5q173tgaJpZM4JUz5e
.

@keyscores

@abernix Have you submitted a pull request for meteorD to the new meteorD repo? I agree that the docker implementation is getting too fragmented. This is why I wish there could be an MDG reference implementation that is kept up to date.

@babenzele
Contributor

Wouldn't a better idea be to simply share the process to create an appropriate docker image? Then anyone could make one and it wouldn't be any one entity's responsibility.

@abernix
Collaborator
abernix commented Oct 28, 2016

@keyscores I have not because because at the time I made my image I didn't make it with the intention of it becoming a popular Docker image; I made it as a drop in semi-emergency replacement for MUP with the belief that the still-developing kadirahq/meteor-up would adopt some of those changes. I offered up some of my changes to the kadirahq fork and meteorhacks/meteord#112 had me thinking it would go somewhere but I know they're very different with other projects.

@babenzele The code for my image and the others is all open source and you can build the abernix/meteord images with the ./build_it.sh script.

Let's see where this conversation on the forums goes. 😄

@wangleihd
wangleihd commented Nov 26, 2016 edited

Help me! Thx

$ mup deploy
Building App Bundle Locally

Started TaskList: Pushing Meteor
x Verifying Deployment: FAILED

	-----------------------------------STDERR-----------------------------------
	-DEV",
	  "http_parser": "2.7.0",
	  "icu": "56.1",
	  "modules": "46",
	  "node": "4.6.2",
	  "openssl": "1.0.2j",
	  "uv": "1.9.1",
	  "v8": "4.5.103.42",
	  "zlib": "1.2.8"
	}
	npm WARN meteor-dev-bundle@0.0.0 No description
	npm WARN meteor-dev-bundle@0.0.0 No repository field.
	npm WARN meteor-dev-bundle@0.0.0 No license field.
	=> Starting meteor app on port:80
	/bundle/bundle/programs/server/boot.js:348
	}).run();
	   ^

	Error: $ROOT_URL, if specified, must be an URL
	    at Error (native)
	    at packages/meteor.js:1218:13
	    at packages/meteor.js:1233:4
	    at packages/meteor.js:1380:3
	    at /bundle/bundle/programs/server/boot.js:295:34
	    at Array.forEach (native)
	    at Function._.each._.forEach (/bundle/bundle/programs/server/node_modules/underscore/underscore.js:79:11)
	    at /bundle/bundle/programs/server/boot.js:128:5
	    at /bundle/bundle/programs/server/boot.js:344:5
	    at Function.run (/bundle/bundle/programs/server/profile.js:480:12)

	=> Redeploying previous version of the app

	-----------------------------------STDOUT-----------------------------------

	To see more logs type 'mup logs --tail=50'

	----------------------------------------------------------------------------
(node:7531) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error:
-----------------------------------STDERR-----------------------------------
-DEV",
  "http_parser": "2.7.0",
  "icu": "56.1",
  "modules": "46",
  "node": "4.6.2",
  "openssl": "1.0.2j",
  "uv": "1.9.1",
  "v8": "4.5.103.42",
  "zlib": "1.2.8"
}
npm WARN meteor-dev-bundle@0.0.0 No description
npm WARN meteor-dev-bundle@0.0.0 No repository field.
npm WARN meteor-dev-bundle@0.0.0 No license field.
=> Starting meteor app on port:80
/bundle/bundle/programs/server/boot.js:348
}).run();
   ^

Error: $ROOT_URL, if specified, must be an URL
    at Error (native)
    at packages/meteor.js:1218:13
    at packages/meteor.js:1233:4
    at packages/meteor.js:1380:3
    at /bundle/bundle/programs/server/boot.js:295:34
    at Array.forEach (native)
    at Function._.each._.forEach (/bundle/bundle/programs/server/node_modules/underscore/underscore.js:79:11)
    at /bundle/bundle/programs/server/boot.js:128:5
    at /bundle/bundle/programs/server/boot.js:344:5
    at Function.run (/bundle/bundle/programs/server/profile.js:480:12)

=> Redeploying previous version of the app

-----------------------------------STDOUT-----------------------------------

To see more logs type 'mup logs --tail=50'

----------------------------------------------------------------------------
@wangleihd

Error:

Start Meteor: FAILED

	-----------------------------------STDERR-----------------------------------
	Error response from daemon: no such id: app
	time="2016-11-26T10:39:21+08:00" level=fatal msg="Error: failed to remove one or more containers"
	Error response from daemon: no such id: app-frontend
	time="2016-11-26T10:39:21+08:00" level=fatal msg="Error: failed to remove one or more containers"
	docker: "run" requires a minimum of 1 argument. See 'docker run --help'.
	-----------------------------------STDOUT-----------------------------------
$ cat config/start.sh
#!/bin/bash

APPNAME=app
APP_PATH=/opt/$APPNAME
BUNDLE_PATH=$APP_PATH/current
ENV_FILE=$APP_PATH/config/env.list
PORT=3000

# Remove previous version of the app, if exists
docker rm -f $APPNAME

# Remove frontend container if exists
docker rm -f $APPNAME-frontend

# We don't need to fail the deployment because of a docker hub downtime
set +e
docker pull abernix/meteord:base
set -e
echo $logConfig.driver >> abc.txt

docker run \
  -d \
  --restart=always \
  --publish=$PORT:80 \
  --volume=$BUNDLE_PATH:/bundle \
  --hostname="$HOSTNAME-$APPNAME" \
  --env-file=$ENV_FILE \
  --link=mongodb:mongodb --env=MONGO_URL=mongodb://mongodb:27017/$APPNAME \
#  \
  --log-driver=syslog \
  --log-opt max-size=100m --log-opt max-file=10 \
  \
  \
  --name=$APPNAME \
  abernix/meteord:base
@hwillson
Collaborator

Hi @wangleihd - given that you're seeing:

Error: $ROOT_URL, if specified, must be an URL

I'd recommend taking a look at what you have your ROOT_URL set to in your mup.json.

The above being said, this isn't the best place to ask for mup usage help (since mup is managed and maintained completely separately from Meteor). You'll have better luck asking these types of questions on the Meteor Forums or Stack Overflow. Thanks!

@antoineclaval antoineclaval added a commit to antoineclaval/heroku-buildpack-meteorite that referenced this issue Nov 29, 2016
@antoineclaval antoineclaval change default node version. Why: meteor/meteor#7475 dc02196
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment