New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AWS Elastic Beanstalk Build Error: node-gyp requires that the user home directory is specified in either of the environmental variables HOME or USERPROFILE #414

Closed
deerawan opened this Issue Mar 5, 2014 · 23 comments

Comments

Projects
None yet
@deerawan

deerawan commented Mar 5, 2014

I have aws elastic beanstalk for node js application and I uploaded my node js app (zip file) to my instance but I saw this error in aws logs

The Error

node-gyp requires that the 'user'\''s' home directory is specified in either of the environmental variables HOME or 'USERPROFILE\n' at new Gyp '(/opt/elasticbeanstalk/node-install/node-v0.8.26-linux-x64/lib/node_modules/npm/node_modules/node-gyp/lib/node-gyp.js:54:11

Package.json

{
  "name": "testing-beanstalk",
  "description": "Testing Beanstalk",
  "author": "Bugan",
  "version": "1.0",
  "private": true,  
  "dependencies": {
    "express": "3.x",
    "socket.io": "0.9.6",
    "mysql": "2.0.1",
    "cron": "1.0.0",
  }
}

Any idea to solve this problem? How to set user home directory?

Thank you before

@chadkouse

This comment has been minimized.

Show comment
Hide comment
@chadkouse

chadkouse Mar 5, 2014

With eb you can define environment variables - is the question more about what directory to set it to?

chadkouse commented Mar 5, 2014

With eb you can define environment variables - is the question more about what directory to set it to?

@deerawan

This comment has been minimized.

Show comment
Hide comment
@deerawan

deerawan Mar 6, 2014

@chadkouse thank you for your response. :)

Where to define environment variables in EB? I tried to create .sh script in .ebextensions like this one below

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/05set_home_env.sh" :
    mode: "000775"
    owner: root
    group: root
    source: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars
      export HOME=/home/user-ec2
      echo "Set home user ec2"

But fail when EB execute the sh file. I also check through the EB instance via SSH and run command "set" to see all environment variables and I see "HOME" var is already set to /home/user-ec2. But still node-gyp can't rebuild because the HOME is not set.

deerawan commented Mar 6, 2014

@chadkouse thank you for your response. :)

Where to define environment variables in EB? I tried to create .sh script in .ebextensions like this one below

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/05set_home_env.sh" :
    mode: "000775"
    owner: root
    group: root
    source: |
      #!/usr/bin/env bash
      . /opt/elasticbeanstalk/support/envvars
      export HOME=/home/user-ec2
      echo "Set home user ec2"

But fail when EB execute the sh file. I also check through the EB instance via SSH and run command "set" to see all environment variables and I see "HOME" var is already set to /home/user-ec2. But still node-gyp can't rebuild because the HOME is not set.

@deerawan

This comment has been minimized.

Show comment
Hide comment
@deerawan

deerawan Mar 14, 2014

Finally I solved it, there is no node-gyp error related anymore. The solution is I set HOME variable in the script below. This script will be executed in EB. :)

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: root
    content: |
      #!/bin/bash          
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      export HOME=/home/ec2-user
      echo "export home"

      OUT=$(/opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

it can be closed now. Thank you

deerawan commented Mar 14, 2014

Finally I solved it, there is no node-gyp error related anymore. The solution is I set HOME variable in the script below. This script will be executed in EB. :)

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: root
    content: |
      #!/bin/bash          
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      export HOME=/home/ec2-user
      echo "export home"

      OUT=$(/opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

it can be closed now. Thank you

@chadkouse

This comment has been minimized.

Show comment
Hide comment
@chadkouse

chadkouse Mar 14, 2014

@deerawan did you find a way to script this change or you are manually ssh'ing to your instances and making it?

chadkouse commented Mar 14, 2014

@deerawan did you find a way to script this change or you are manually ssh'ing to your instances and making it?

@deerawan

This comment has been minimized.

Show comment
Hide comment
@deerawan

deerawan Mar 16, 2014

Yes, I find a way to insert this script without ssh-ing my instance. In EB, you can make configuration for deployment. It is stated in folder .ebextensions. EB will automatically execute file with .config extension. So, I just made a config file named env.config

Here is my node js project structure

- .ebextensions
---- env.config
- node_modules
- package.json
- server.js 

There are some default files (mostly .sh) that EB always execute in deployment process. One of them is 50npm.sh. The default content of this file is

      #!/bin/bash          
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      OUT=$(/opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

So, in order to make node-gyp know the HOME environment variable, I just need to create new 50npm.sh in my config file which contain export home statement before npm install command. Below is my config file

env.config

packages:
  yum:
    git: []
    gcc: []
    make: []
    openssl-devel: []
    libxml2: []
    libxml2-devel: []    

option_settings:
  - option_name: NODE_ENV
    value: production
  - namespace: aws:elasticbeanstalk:container:nodejs
    option_name: NodeVersion
    value: 0.10.21

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: root
    content: |
      #!/bin/bash          
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      export HOME=/home/ec2-user # ADDED EXPORT COMMAND
      echo "export home" # JUST FOR REMARK

      OUT=$(/opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

The path "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" is important because it is the location of the default 50npm.sh. So, we need to replace the old one with ours.

The next step is deploy your node js to EB and voila it works :)

Hope it helps someone else who face the same problem.

deerawan commented Mar 16, 2014

Yes, I find a way to insert this script without ssh-ing my instance. In EB, you can make configuration for deployment. It is stated in folder .ebextensions. EB will automatically execute file with .config extension. So, I just made a config file named env.config

Here is my node js project structure

- .ebextensions
---- env.config
- node_modules
- package.json
- server.js 

There are some default files (mostly .sh) that EB always execute in deployment process. One of them is 50npm.sh. The default content of this file is

      #!/bin/bash          
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      OUT=$(/opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

So, in order to make node-gyp know the HOME environment variable, I just need to create new 50npm.sh in my config file which contain export home statement before npm install command. Below is my config file

env.config

packages:
  yum:
    git: []
    gcc: []
    make: []
    openssl-devel: []
    libxml2: []
    libxml2-devel: []    

option_settings:
  - option_name: NODE_ENV
    value: production
  - namespace: aws:elasticbeanstalk:container:nodejs
    option_name: NodeVersion
    value: 0.10.21

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: root
    content: |
      #!/bin/bash          
      function error_exit
      {
        eventHelper.py --msg "$1" --severity ERROR
        exit $2
      }

      export HOME=/home/ec2-user # ADDED EXPORT COMMAND
      echo "export home" # JUST FOR REMARK

      OUT=$(/opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install 2>&1) || error_exit "Failed to run npm install.  $OUT" $?
      echo $OUT

The path "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" is important because it is the location of the default 50npm.sh. So, we need to replace the old one with ours.

The next step is deploy your node js to EB and voila it works :)

Hope it helps someone else who face the same problem.

@davidsketchdeck

This comment has been minimized.

Show comment
Hide comment
@davidsketchdeck

davidsketchdeck Mar 16, 2014

Thank you for sharing! It's really kind of you

davidsketchdeck commented Mar 16, 2014

Thank you for sharing! It's really kind of you

@deerawan

This comment has been minimized.

Show comment
Hide comment
@deerawan

deerawan Mar 16, 2014

@davidsketchdeck you are welcome, david :)

deerawan commented Mar 16, 2014

@davidsketchdeck you are welcome, david :)

@robmaceachern

This comment has been minimized.

Show comment
Hide comment
@robmaceachern

robmaceachern commented Aug 8, 2014

Thanks @deerawan 👍

@et304383

This comment has been minimized.

Show comment
Hide comment
@et304383

et304383 Sep 22, 2014

This is not a valid fix. The module should be able to be installed without needing bash variables set.

The NPM package should be installable without any shell variables set. Please fix this. Nothing should be installed assuming a HOME directory or need a home directory value set. Nothing should be writing to the user home directory if I'm installing NPM packages globally.

et304383 commented Sep 22, 2014

This is not a valid fix. The module should be able to be installed without needing bash variables set.

The NPM package should be installable without any shell variables set. Please fix this. Nothing should be installed assuming a HOME directory or need a home directory value set. Nothing should be writing to the user home directory if I'm installing NPM packages globally.

@et304383

This comment has been minimized.

Show comment
Hide comment
@et304383

et304383 Sep 23, 2014

Also, running Chef through CloudFormation using the nodejs_npm resource of the nodejs cookbook (https://github.com/redguide/nodejs) there is no opportunity to set bash variables.

Fix this properly - $HOME should NOT be required to install ANY software.

et304383 commented Sep 23, 2014

Also, running Chef through CloudFormation using the nodejs_npm resource of the nodejs cookbook (https://github.com/redguide/nodejs) there is no opportunity to set bash variables.

Fix this properly - $HOME should NOT be required to install ANY software.

@ajf

This comment has been minimized.

Show comment
Hide comment
@ajf

ajf Jan 29, 2015

@eric-tucker 👍

ajf commented Jan 29, 2015

@eric-tucker 👍

@smoya

This comment has been minimized.

Show comment
Hide comment
@smoya

smoya Mar 11, 2015

What is the current status of this issue? Because i have the same problem.

smoya commented Mar 11, 2015

What is the current status of this issue? Because i have the same problem.

@kylev

This comment has been minimized.

Show comment
Hide comment
@kylev

kylev Jul 23, 2015

FWIW, I deployed a Node app to EB based on Amazon Linux 2015.03 today and this problem is no longer present. I poked around a bit and /opt/elasticbeanstalk/containerfiles/ebnode.py (which is used to drive the npm install) now sets $HOME.

(I'm not a fan of build tools depending on a homedir, but it does now seem to work.)

kylev commented Jul 23, 2015

FWIW, I deployed a Node app to EB based on Amazon Linux 2015.03 today and this problem is no longer present. I poked around a bit and /opt/elasticbeanstalk/containerfiles/ebnode.py (which is used to drive the npm install) now sets $HOME.

(I'm not a fan of build tools depending on a homedir, but it does now seem to work.)

@chadkouse

This comment has been minimized.

Show comment
Hide comment
@chadkouse

chadkouse Jul 23, 2015

good news. Maybe I will try eb again. I gave up after many, many confusing
and stalled deployments.

On Thu, Jul 23, 2015, 5:46 PM Kyle VanderBeek notifications@github.com
wrote:

FWIW, I deployed a Node app to EB based on Amazon Linux 2015.03 today and
this problem is no longer present. I poked around a bit and
/opt/elasticbeanstalk/containerfiles/ebnode.py (which is used to drive
the npm install) now sets $HOME.

(I'm not a fan of build tools depending on a homedir, but it does now seem
to work.)


Reply to this email directly or view it on GitHub
#414 (comment)
.

--chad

chadkouse commented Jul 23, 2015

good news. Maybe I will try eb again. I gave up after many, many confusing
and stalled deployments.

On Thu, Jul 23, 2015, 5:46 PM Kyle VanderBeek notifications@github.com
wrote:

FWIW, I deployed a Node app to EB based on Amazon Linux 2015.03 today and
this problem is no longer present. I poked around a bit and
/opt/elasticbeanstalk/containerfiles/ebnode.py (which is used to drive
the npm install) now sets $HOME.

(I'm not a fan of build tools depending on a homedir, but it does now seem
to work.)


Reply to this email directly or view it on GitHub
#414 (comment)
.

--chad

@rhyeal

This comment has been minimized.

Show comment
Hide comment
@rhyeal

rhyeal Mar 8, 2016

We ended up using a slightly modified version for our problem. Leaving it here for anyone else who may need it (bcrypt specific):

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: root
    content: |
      #!/bin/bash
      #==============================================================================
      # Copyright 2013 Amazon.com, Inc. or its affiliates. All Rights Reserved.
      #
      # Licensed under the Amazon Software License (the "License"). You may not use
      # this file except in compliance with the License. A copy of the License is
      # located at
      #
      #       http://aws.amazon.com/asl/
      #
      # or in the "license" file accompanying this file. This file is distributed on
      # an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
      # implied. See the License for the specific language governing permissions
      # and limitations under the License.
      #==============================================================================

      export HOME=/home/ec2-user
      export USERPROFILE=/home/ec2-user

      sudo rm -rf /tmp/deployment/application/node_modules/bcrypt

      set -xe

      /opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install


rhyeal commented Mar 8, 2016

We ended up using a slightly modified version for our problem. Leaving it here for anyone else who may need it (bcrypt specific):

files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/50npm.sh" :
    mode: "000775"
    owner: root
    group: root
    content: |
      #!/bin/bash
      #==============================================================================
      # Copyright 2013 Amazon.com, Inc. or its affiliates. All Rights Reserved.
      #
      # Licensed under the Amazon Software License (the "License"). You may not use
      # this file except in compliance with the License. A copy of the License is
      # located at
      #
      #       http://aws.amazon.com/asl/
      #
      # or in the "license" file accompanying this file. This file is distributed on
      # an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, express or
      # implied. See the License for the specific language governing permissions
      # and limitations under the License.
      #==============================================================================

      export HOME=/home/ec2-user
      export USERPROFILE=/home/ec2-user

      sudo rm -rf /tmp/deployment/application/node_modules/bcrypt

      set -xe

      /opt/elasticbeanstalk/containerfiles/ebnode.py --action npm-install


@pauloaguiar91

This comment has been minimized.

Show comment
Hide comment
@pauloaguiar91

pauloaguiar91 Sep 4, 2016

Thank you so much! @rhyeal and all. Your solution did the trick. Relatively painless process thanks to this thread.

pauloaguiar91 commented Sep 4, 2016

Thank you so much! @rhyeal and all. Your solution did the trick. Relatively painless process thanks to this thread.

@joelwass

This comment has been minimized.

Show comment
Hide comment
@joelwass

joelwass Sep 7, 2016

Don't forget to have an .ebignore file. The .ebignore file should (generally) just be:

node_modules/

This was my error - I was on a wild goose chase for hours.

joelwass commented Sep 7, 2016

Don't forget to have an .ebignore file. The .ebignore file should (generally) just be:

node_modules/

This was my error - I was on a wild goose chase for hours.

@GuillaumeSarfati

This comment has been minimized.

Show comment
Hide comment
@GuillaumeSarfati

GuillaumeSarfati Oct 21, 2016

Hello,

Sorry I have the same issue , I have tried your solutions but nothing works..

I have tried deerawan's config
I have tried rhyeal's config

Do you have other solutions ?

node version: 4.4.3
os: 64bit Amazon Linux 2016.03 v2.1.1 running Node.js

Thanks !

GuillaumeSarfati commented Oct 21, 2016

Hello,

Sorry I have the same issue , I have tried your solutions but nothing works..

I have tried deerawan's config
I have tried rhyeal's config

Do you have other solutions ?

node version: 4.4.3
os: 64bit Amazon Linux 2016.03 v2.1.1 running Node.js

Thanks !

@whatch

This comment has been minimized.

Show comment
Hide comment
@whatch

whatch Nov 29, 2016

this is still an issue; deploying to aws elastic beanstalk, and running into this same issue. Also have issues with it on windows, docker, and on it goes. It literally has to be the largest point of failure in the node world. fix it for christ sakes

whatch commented Nov 29, 2016

this is still an issue; deploying to aws elastic beanstalk, and running into this same issue. Also have issues with it on windows, docker, and on it goes. It literally has to be the largest point of failure in the node world. fix it for christ sakes

@akaNightmare

This comment has been minimized.

Show comment
Hide comment
@akaNightmare

akaNightmare commented Jan 4, 2017

+1

@dankolesnikov

This comment has been minimized.

Show comment
Hide comment
@dankolesnikov

dankolesnikov Feb 14, 2017

@GuillaumeSarfati Were you able to resolve the issue?
This is still active:/

dankolesnikov commented Feb 14, 2017

@GuillaumeSarfati Were you able to resolve the issue?
This is still active:/

@florianbepunkt

This comment has been minimized.

Show comment
Hide comment
@florianbepunkt

florianbepunkt Apr 15, 2017

Also still having issues deploying to AWS EB. node-gyp rebuild fails… anyone came up with a solution?

florianbepunkt commented Apr 15, 2017

Also still having issues deploying to AWS EB. node-gyp rebuild fails… anyone came up with a solution?

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@bnoordhuis

bnoordhuis Apr 16, 2017

Member

A solution is outlined in #414 (comment). As to this:

Fix this properly - $HOME should NOT be required to install ANY software.

node-gyp needs a place to download the SDK to. If you don't pass --devdir, it defaults to $HOME/.node-gyp. See the README for more info.

Member

bnoordhuis commented Apr 16, 2017

A solution is outlined in #414 (comment). As to this:

Fix this properly - $HOME should NOT be required to install ANY software.

node-gyp needs a place to download the SDK to. If you don't pass --devdir, it defaults to $HOME/.node-gyp. See the README for more info.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment