require('google-cloud') takes 12 seconds to execute

[martincad]

The require for the google-cloud package takes a lot to execute, about 12 seconds.

Environment details

• OS: Ubuntu 14.04 / OSX 10.11.6
• Node.js version: v4.2.3 / v4.2.3
• npm version: 2.14.7 / 2.14.7
• google-cloud-node version: 0.40.0

Steps to reproduce

1. mkdir tmp
2. cd tmp
3. npm init -y
4. npm install --save google-cloud
5. npm dedupe
6. time node -e "require('google-cloud')"

real 0m10.932s
user 0m8.482s
sys 0m1.036s

[stephenplusplus]

Ouch. We will look into that.

[callmehiphop]

This actually looks to be related to another issue (#1590) we're looking into. Have you thought about upgrading to npm3? Testing locally cut it down from 12 seconds to about 1 for me.

[martincad]

@callmehiphop we are using gcloud package (the old one) on a Production environment and were planning to upgrade to google-cloud lib, but we are using Node Argon, and don't think we can migrate until Node 6 is flagged as LTS.

Do you think is there any work-around for node 4 and npm 2 to avoid this huge load time ?

[stephenplusplus]

Possibly running 'npm dedupe' after installing dependencies, and before starting your program. You can also upgrade npm to 3 'npm install -g npm' without affecting the Node version.

[martincad]

@s3ththompson The issue can be reproduce by just installing google-cloud package on an empty directory, so there is nothing to dedupe. (Just in case I tested it, but is still taking a lot).

[stephenplusplus]

dedupe should flatten all of our dependencies between our modularized google-cloud-* packages into one common place, which should make the 'require' times quicker, since they're not scattered all over the disk. So after installing, you ran 'npm dedupe', then ran your program, and still experienced 12 second startup times?

[martincad]

Let me share the steps with you, just in case I'm doing something wrong:

1. mkdir tmp
2. cd tmp
3. npm install google-cloud
4. npm dedupe
5. time node -e "require('google-cloud')"

real 0m10.932s
user 0m8.482s
sys 0m1.036s

I have reproduced this on Mac and Ubuntu (14.04), weird why nobody reported this

[stephenplusplus]

Do you end up with a node_modules directory inside of tmp? Maybe try 'npm init -y && npm install --save google-cloud'. Not sure if dedupe will work without a package.json. Is there any sign the dedupe command is working?

[martincad]

@stephenplusplus You are right, npm dedupe was not doing anything without a package.json. The bad thing if after adding the package and run dedupe ( and seeing a lot of dedupe output which was not there before) the bug still persists ( at least on Mac ).

Will try the same thing on Ubuntu trusty later and will let you know.

Thanks

[stephenplusplus]

I believe this is a symptom of having our sub-modules (i.e. @google-cloud/storage) depend on different versions of @google-cloud/common.

The common package has a dependency on grpc, which is a massive one. Today, we will release all of our sub-modules and a new google-cloud, which will all have a dependency on the same version of grpc. That should mean after running dedupe, the require times should be more in line with what you'd expect.

I'll update here once the new release is out.

[stephenplusplus]

After the release tonight of v0.41.0, the times went down dramatically for me:

node -e "require('google-cloud')"
2.36s user 0.27s system 56% cpu 4.692 total

Can you let me know if you're seeing the same improvement if you repeat the steps from the opening post?

[stephenplusplus]

We're definitely much better now with v0.41.2, a release we just put out which uses shrinkwrap to force deduplication:

$ time node -e "require('google-cloud')"
0.93s user 0.11s system 101% cpu 1.025 total

[martincad]

@s3ththompson worked like a charm. Thanks a lot:

time node -e "require('google-cloud')"

real 0m0.831s
user 0m0.638s
sys 0m0.097s